DESCRIPTION

Have mcsoda cache generated docs

Change-Id: I2b4bc7f8d8636da57c7e7bdb03c7c3c36e11f1b9


COMMENTS

author: Nora Conway
date: 2011-11-04 18:23:53.509000000

Uploaded patch set 2.

-------------------------------------
author: Zackery Barr
date: 2011-11-04 18:32:51.677000000

Patch Set 2: (1 inline comment)



Line:600, pytests/performance/mcsoda.py -> curious to know how large can this cache get ? ( and how much RAM does python use at that point )

I have noticed some slowness in accessing a very large dict/array in python when i used to do these caching as part of functional test

also we need to make sure we run these tests on a machine that has enough ram

-------------------------------------
author: Nora Conway
date: 2011-11-04 18:52:42.473000000

Patch Set 2: (1 inline comment)



Line:600, pytests/performance/mcsoda.py -> good questions.  I re-ran mcsoda with larger item counts (instead of the default 100K items)...

with 1M items, mcsoda takes awhile (>30secs) to build up its doc cache, and is at 15K ops/second.  So, that'll affect tests, unfortunately.  Also, after the doc cache gets filled up, then mcsoda jumps to 25+K ops/second (since it no longer has to generate docs), where memory stabilizes at around 2.7GB vsize and 395MB rsize.

will look for other ways to make doc generation faster / less memory, but no a-ha insights at the moment.

-------------------------------------
author: Reid Stokes
date: 2011-11-04 19:18:01.994000000

Patch Set 2: (1 inline comment)



Line:600, pytests/performance/mcsoda.py -> Cant we pre create these docs before... why do they have to be inflight?

If the memory usage is going to be some fixed size. 

THis would also impact perf run, since we run mcsoda locally against the test server.

-------------------------------------
