DESCRIPTION

script to collect data files from every node

Change-Id: I2107e18f42997f4ab8adf2ef62593f657b9d7bf4


COMMENTS

author: Angelina Mayer
date: 2014-03-21 01:04:39.587000000

Uploaded patch set 2.

-------------------------------------
author: Angelina Mayer
date: 2014-03-21 01:05:58.479000000

Patch Set 2: Verified+1

A simple variation of collect_server_info.py to collect data files from every node.

Usage: python collect_data_files.py -i <ini>

-------------------------------------
author: Hugo Blankenship
date: 2014-03-21 01:58:49.777000000

Patch Set 2:

Build Started http://factory.couchbase.com/job/testrunner-Jasmin Rangel-master/759/

-------------------------------------
author: Hugo Blankenship
date: 2014-03-21 02:23:39.632000000

Patch Set 2: Verified+1

Build Successful 

http://factory.couchbase.com/job/testrunner-Jasmin Rangel-master/759/ : SUCCESS

-------------------------------------
author: Fisher Vargas
date: 2014-03-21 06:03:29.918000000

Patch Set 2:

Currently tearDown() function deletes all the buckets after test is failed, and we loose the data files. We need to make some provision to run this script before bucket deletion is called.

-------------------------------------
author: Angelina Mayer
date: 2014-03-21 23:49:27.342000000

Patch Set 2:

That's a good idea too. For now this is a standalone script. Any comments?

-------------------------------------
author: Esperanza Cherry
date: 2014-03-22 12:01:00.571000000

Patch Set 2:

hi Sangharsh,

we already have some params to do not clean cluster in teardown: skip_buckets_handle, stop-on-failure,  skip_cleanup
https://github.com/couchbase/testrunner/blob/master/pytests/basetestcase.py#L162

but i don't think that we should collect data after every failure in ordinary cases. it may be gigabytes of data per case. besides, there were individual cases when developers want to look at the data.

-------------------------------------
author: Fisher Vargas
date: 2014-03-23 12:35:18.735000000

Patch Set 2:

Andrei, XdcrBaseTests don't have such provision to skip tearDown? Also, we don't need to skip tearDown() function, while we need to run tearDown function after the log-collection is done. 

And, We need such data files specially in case of XDCR verification failed (MB-10494). So, we can add such checks that data files will be collected only if XDCR data verification check failed.

-------------------------------------
author: Esperanza Cherry
date: 2014-03-23 13:49:27.184000000

Patch Set 2:

Andrei, XdcrBaseTests don't have such provision to skip tearDown? 
-- not yet, but it's copy-paste


Also, we don't need to skip tearDown() function, while we need to run tearDown function after the log-collection is done.
-- currently we get log-collection after tearDown.

And, We need such data files specially in case of XDCR verification failed (MB-10494).
- seems like they did not need to Alk in the bug.
Also, Parag started to work with "compare data" scripts. but I did not see the script. purpose of obtaining data was to ensure that the data are synchronized, and the stats do not.

 So, we can add such checks that data files will be collected only if XDCR data verification check failed. 
--I think we will have more problems than the result.


I see this script will be useful only if: 
1. stop test on failure(don't run tearDown)
2. collect data files

as an improvement to the script:
1) location of data files could be different
2) doesn't work on windows(Mac?)
3) we can collect any folder ( logs,config, etc) via additional parameter

-------------------------------------
