DESCRIPTION

helper class to put test results into couchdb

provides a function to add new performance results and to compare
versus old results

import random
from results_helper import ResultsHelper
results = xxx@xxx.xxx
data = {
    "version":"2.0.0r-37-gef01f43",
    "product":"couchbase",
    "nodes":1,
    "operation":"set",
    "items":1000,
    "value_size":100,
    "key_size":10,
    "json":False,
    "active_task":None,
    "ops_per_second":random.randint(4500,5500),
    }
print results.compare_perf(data,
                           limit=None,
                           same_keys=["product",
                                      "nodes",
                                      "operation",
                                      "value_size",
                                      "key_size",
                                      "json",
                                      "active_task"],
                           different_keys={},
                           result_key="ops_per_second")
change = results.compare_perf(data,
                              limit=1,
                              same_keys=["product",
                                         "nodes",
                                         "operation",
                                         "value_size",
                                         "key_size",
                                         "json",
                                         "active_task"],
                              different_keys={"version":"2.0.0r-37-gef01f43"},
                              result_key="ops_per_second")
results.add_perf(data)
for c in change:
    if c <= 0.9:
        raise Exception("test performace is less than 90% of a previous run")

Change-Id: I4f96b0db922d8022f28199ed27a8bad4a4457cbe


COMMENTS

author: Ayden Rivas
date: 2011-09-12 22:31:42.052000000

Patch Set 1:

right now it doesnt have testrunner or test integration.

todo:
add a "-p" parameter for the couchdb location
include this in the memcapable perf tests

-------------------------------------
author: Zackery Barr
date: 2011-09-12 22:37:00.584000000

Patch Set 1: (1 inline comment)



Line:33, lib/results_helper.py -> so the data object will have build# , cluster size and other attributes that are common between all tests and then in addition to common attributes we can add test specific stats?
for example for disk_drain_rate we can add disk_drain_rate or for warmup we can have warmup ?

-------------------------------------
author: Ayden Rivas
date: 2011-09-12 22:46:41.353000000

Patch Set 1: (1 inline comment)



Line:33, lib/results_helper.py -> build#, cluster size, and the others aren't enforced, you can add results that don't include that data. though the use of that data will then be pretty limited.

you can add as many test specific stats as you want. such as drain rate, warmup time, ops per second.

you can also add a task. so you could have your stat be ops per second, and your task be disk_drain, autocompaction, rebalance, etc. right now i only allow one task.

-------------------------------------
author: Zackery Barr
date: 2011-09-12 22:59:01.496000000

Patch Set 1: Verified; Looks good to me, approved



-------------------------------------
author: Zackery Barr
date: 2011-09-12 22:59:05.073000000

Change has been successfully cherry-picked as a9e949e89b484dec8a3f902ae3e8cf807e39fa48.

-------------------------------------
