DESCRIPTION

CBQE-1913: refactor failovertests

* Moved calls to rest/ssh into appropriate tasks sometimes borrowing from
implementation in basetestcase.

* Removed common method being used by all the tests with all its conditions
and mad each of the tests flatter having their own consice flow

* Added a swap orchestrator failover test

* I tried port/password tests - more debugging needed, they may be long somewhere else
but I'm leaving in case someone wants to implement

* The addback test needs to make use of its own addbacktask as full rebalance cannot be used in this case

Change-Id: Ife664e19a6e40d9d3ca5c09a6d31ed9ce550c930


COMMENTS

author: Titus Ho
date: 2014-01-30 22:07:26.015000000

Uploaded patch set 2.

-------------------------------------
author: Hugo Blankenship
date: 2014-01-30 23:07:25.654000000

Patch Set 2:

Build Started http://factory.couchbase.com/job/testrunner-Jasmin Rangel-master/293/

-------------------------------------
author: Hugo Blankenship
date: 2014-01-30 23:33:55.152000000

Patch Set 2: Verified-1

Build Failed 

http://factory.couchbase.com/job/testrunner-Jasmin Rangel-master/293/ : FAILURE

-------------------------------------
author: Linda Vaughan
date: 2014-02-07 22:55:16.269000000

Patch Set 2: Code-Review+1

-------------------------------------
author: Esperanza Cherry
date: 2014-02-09 08:28:02.254000000

Patch Set 2:

refactoring does not involve loss of functionality. Several scenarios are unaccounted for here. if you make these changes, it will be deleted steps / logic test for these cases.
we will have only:
"TODO: NEEDS ADD-BACK TASK  &
SHOULD BE ELSEWHERE". 
not determined when it will be done
who will do TODO (again, I believe that it is a waste of time), face the question: what does it mean, what steps are assumed to be, etc.


also we lose all workarounds and known bugs: for example, see comments related to MB-7168 in 
http://review.couchbase.org/#/c/33049/2/pytests/failover/failovertests.py

will spend time figuring out the problem, fill new bug and create new workaround

-------------------------------------
author: Titus Ho
date: 2014-02-11 21:53:19.022000000

Patch Set 2:

Thanks Andrei for going over this, I was under the impression that these tests are not being run today.  Yet point me to the jenkins job if I am incorrect here.  I am also very interested in how long this job runs today.

A refactor can in some cases can remove trivial cases and especially if very similar coverage exists elsewhere.  If you 'grep' for add_back in our repo you can see it being done in at least 3 different components and so there really is coverage for this scenerio elsewhere.  

Also I'm of the opinion that we should not ever code in workarounds for broken server code to get tests to pass.  Let it break until someone fixes it.

-------------------------------------
author: Esperanza Cherry
date: 2014-02-12 07:26:30.138000000

Patch Set 2:

Hi Tommie, these tests should be run http://qa.sc.couchbase.com/job/centos_x64--03_01--failover_tests-P0/32/consoleFull.

but I see now that cluster is not 'initialized' properly, we have only one node after setup. all the tests work wasted.

yes, we have some tests with "add_back" but they don't contain  cases like:
1)click falover(failed_over = rest.fail_over(node.id))
2)and then add back with rebalance

-------------------------------------
author: Titus Ho
date: 2014-02-12 15:46:37.894000000

Patch Set 2:

Yea that's what I was looking for - and looks like this together with autofailover tests the entire suite takes 7 hours.   But still, time reduction is a small goal of refactoring.  Our code base is out of control with duplicate logic and legacy cruft everywhere.  This repo is public and if anyone wanted to run our tests they wouldn't know where to start - let alone contribute.  Even in-house we are struggling with this.

2) Here's some examples of manually triggering failover then adding_back from existing code-base -
https://github.com/couchbase/testrunner/blob/master/pytests/swaprebalance.py#L449
https://github.com/couchbase/testrunner/blob/master/pytests/tuqquery/tuq_cluster_ops.py#L96
https://github.com/couchbase/testrunner/blob/master/pytests/xdcr/biXDCR.py

No doubt our code works today, and I'm in no rush to push this as you've been hands-on with testrunner more than anyone.   Would be great to have you in our mtgs - esp would like to know if you think this current model is sustainable for 3.0 and beyond, as I've been of the opinion we can do better.

-------------------------------------
author: Titus Ho
date: 2017-05-08 11:25:59.376000000

Abandoned

-------------------------------------
