DESCRIPTION

tmf: Add a HTNode cache to decrease the number of disk access.

On startup, a cache of 256 entries give this ratio:

    7778/8000 [97.23%]

After running a while:

    42375/42600 [99.47%]

This is a variant of the patch-set proposed by Florian:

    https://git.eclipse.org/r/#/c/19330/

Change-Id: Icf2f6af3c9f40d4532196b18a87f2aa728341a58
Signed-off-by: Jordon Obrien xxx@xxx.xxx


COMMENTS

author: Jordon Obrien
date: 2013-12-05 00:49:18.000000000

Uploaded patch set 2.

-------------------------------------
author: Ayanna Reed
date: 2013-12-05 01:22:05.000000000

Patch Set 2: Code-Review+1

LGTM

-------------------------------------
author: Ivy Mitchell
date: 2013-12-05 01:47:26.000000000

Patch Set 2: Code-Review+1 IP-Clean+1

Propagating +1

-------------------------------------
author: Ivy Mitchell
date: 2013-12-05 01:50:21.000000000

Patch Set 2:

(1 comment)

Line:148, lttng/org.eclipse.linuxtools.tmf.core/src/org/eclipse/linuxtools/internal/tmf/core/statesystem/backends/historytree/HT_IO.java -> Insert implies putting something in and then shifting everything else, no? I would say "Store the node in the cache according to it's least significant byte", but this is insignificant as an issue.

-------------------------------------
author: Jordon Obrien
date: 2013-12-05 01:51:47.000000000

Patch Set 2:

(1 comment)

I'll fix it if we go with this solution.

Line:148, lttng/org.eclipse.linuxtools.tmf.core/src/org/eclipse/linuxtools/internal/tmf/core/statesystem/backends/historytree/HT_IO.java -> I like 'store'

-------------------------------------
author: Brenden Conley
date: 2013-12-05 03:34:51.000000000

Patch Set 2:

Build Started https://hudson.eclipse.org/linuxtools/job/linuxtools-gerrit/4423/

-------------------------------------
author: Brenden Conley
date: 2013-12-05 04:11:16.000000000

Patch Set 2: Verified+1

Build Successful 

https://hudson.eclipse.org/linuxtools/job/linuxtools-gerrit/4423/ : SUCCESS

-------------------------------------
author: Tessa Dickson
date: 2013-12-05 08:21:28.000000000

Patch Set 2:

I think we need raw speed benchmarks and hit rate of these caches mechanisms in real scenarios (many worker threads reading the trace in parallel), then compare them, and choose one. I disagree with Matthew about "19351 vs 19330" not being mutually exclusive: it's useless to have two caches that effectively do the same thing. It will add a constant cost without any benefit.

The original idea of the cache is the great improvement, but we need to use metrics to choose the best implementation. Etienne, in the final patch changelog, you might want to thank Florian and Michel for coming up with the cache idea.

-------------------------------------
author: Tessa Dickson
date: 2013-12-05 08:37:41.000000000

Patch Set 2:

(3 comments)

Technically, you seem to be missing a volatile here.

Line:15, /COMMIT_MSG -> A quick number showing the speed improvement (not in terms of hit/miss, but in terms of seconds) might be good to have here.

Line:19, /COMMIT_MSG -> You might want to credit Florian and Michel for the cache idea here.

Line:44, lttng/org.eclipse.linuxtools.tmf.core/src/org/eclipse/linuxtools/internal/tmf/core/statesystem/backends/historytree/HT_IO.java -> see http://www.javamex.com/tutorials/synchronization_volatile.shtml

I think you'll want to declare your array as an array of volatile objects.

-------------------------------------
author: Jordon Obrien
date: 2013-12-05 13:28:39.000000000

Patch Set 2:

(3 comments)

Florian, could you take the lead of the investigation, integration etc... and, take the authorship.
Keep me in the review process.

You may want to benchmark and compare my two versions of you patch-set.

And discuss with Alex if we plan to remove readFromMemory(...), but do it in a separate commit.

Line:15, /COMMIT_MSG -> Yes, further investigation is needed.

Line:19, /COMMIT_MSG -> In fact, I would like Florian take care of this patch-set.
I'm busy on other stuff. This patch-set is only to demonstrate my idea.

Line:44, lttng/org.eclipse.linuxtools.tmf.core/src/org/eclipse/linuxtools/internal/tmf/core/statesystem/backends/historytree/HT_IO.java -> Each access is through a synchonized method:

 synchronized HTNode readNodeFromDisk(...)

Thus, volatile is useless.

-------------------------------------
author: Braxton Mccarthy
date: 2013-12-05 19:27:01.000000000

Patch Set 2: Code-Review+2 IP-Clean+1

As we discussed in that (loooong!) email thread, this is something that was initially planned, but in the end was not prioritized and ended up not being done. But there are clear advantages: while the OS does cache the reads from disk, the JVM will still re-create over 3000 objects for every node that is read (the node itself + all the intervals it contains). By caching them, we cut a lot on those objects creation, and the related work the GC has to do.

This version looks good, I'm fine merging it as is. Objections?

-------------------------------------
author: Jordon Obrien
date: 2013-12-05 19:39:15.000000000

Patch Set 2:

No objection, but add Florian as the author for this patch.
Leave a todo for further cache experiments.

-------------------------------------
author: Annalise Jimenez
date: 2013-12-05 19:39:18.000000000

Patch Set 2: Code-Review+1

I'm fine too, the numbers speak of themselves.  Maybe change the commit message though so it does not refer to a patchset but gives credit to Florian

-------------------------------------
author: Tessa Dickson
date: 2013-12-05 21:47:51.000000000

Patch Set 2: Code-Review+1

lgtm, nit: please give credit to Florian for the cache idea (rather than pointing to URL).

OK with explanation why no volatile: it's already used in a synchronized method.

Thanks!

Mathieu

-------------------------------------
author: Braxton Mccarthy
date: 2013-12-05 22:01:53.000000000

Patch Set 2: Code-Review-2

Ok I'll repush the patchset adding Florian's sign-off. I will target it to master too, lttng-luna branch doesn't exist as of 5 minutes ago...

-------------------------------------
author: Braxton Mccarthy
date: 2013-12-05 22:06:23.000000000

Abandoned

Replaced by https://git.eclipse.org/r/#/c/19408/

-------------------------------------
