DESCRIPTION

cbqe-2130 abort installation of couchbase
server if client could not ssh to any server (vm)

Change-Id: I13026dc865997d4cb9be1f6d4e4accb4609da231


COMMENTS

author: Tate Garrett
date: 2014-03-18 01:16:36.428000000

The change could not be merged due to a path conflict.

Please rebase the change locally and upload the rebased commit for review.

-------------------------------------
author: Tate Garrett
date: 2014-03-18 01:23:51.474000000

The change could not be merged due to a path conflict.

Please rebase the change locally and upload the rebased commit for review.

-------------------------------------
author: Frederick Booker
date: 2014-03-18 01:25:22.088000000

Patch Set 2: Verified+1

-------------------------------------
author: Hugo Blankenship
date: 2014-03-18 01:53:16.191000000

Patch Set 2: Verified+1

Build Successful 

http://factory.couchbase.com/job/testrunner-Jasmin Rangel-master/719/ : SUCCESS

-------------------------------------
author: Esperanza Cherry
date: 2014-03-18 08:39:40.665000000

Patch Set 2:

Hi Tony, with exit(1) we exit python and installation aborted

with current code when set wrong password:

xxx@xxx.xxx testrunner]# python scripts/install.py -i andrei.ini -p version=3.0.0-443-rel,product=cb,parallel=True
2014-03-18 01:35:23,870 - root - INFO - connecting to 10.3.4.144 with username : root password : couchbase1 ssh_key: 
2014-03-18 01:35:23,880 - root - INFO - connecting to 10.3.4.146 with username : root password : couchbase1 ssh_key: 
2014-03-18 01:35:23,881 - root - INFO - connecting to 10.3.4.145 with username : root password : couchbase1 ssh_key: 
2014-03-18 01:35:23,881 - root - INFO - connecting to 10.3.4.147 with username : root password : couchbase1 ssh_key: 
2014-03-18 01:35:26,417 - root - ERROR - Authentication failed
2014-03-18 01:35:26,421 - root - ERROR - Authentication failed
2014-03-18 01:35:26,457 - root - ERROR - Authentication failed
2014-03-18 01:35:26,459 - root - ERROR - Authentication failed
thread uninstaller-thread-10.3.4.144 finished
thread uninstaller-thread-10.3.4.146 finished
thread uninstaller-thread-10.3.4.145 finished
thread uninstaller-thread-10.3.4.147 finished
2014-03-18 01:35:26,558 - root - INFO - connecting to 10.3.4.144 with username : root password : couchbase1 ssh_key: 
2014-03-18 01:35:28,709 - root - ERROR - Authentication failed
xxx@xxx.xxx testrunner]# 

didn't catch why we need to raise Exception instead of exit.
This does not guarantee script abort

-------------------------------------
author: Frederick Booker
date: 2014-03-18 19:20:42.768000000

Patch Set 2:

With exit(1), it does not abort the installation scripts.  We had many times that the installation did not abort when one of vm could not login

-------------------------------------
author: Angelina Mayer
date: 2014-03-18 21:19:43.131000000

Patch Set 2: Code-Review+2

-------------------------------------
author: Tate Garrett
date: 2014-03-18 21:19:50.045000000

The change could not be merged due to a path conflict.

Please rebase the change locally and upload the rebased commit for review.

-------------------------------------
author: Angelina Mayer
date: 2014-03-18 21:22:13.124000000

Patch Set 2: -Code-Review

Tony, looks like you need to rebase again. And since Andrei had a comment, I defer the approval to him. Thanks.

-------------------------------------
author: Esperanza Cherry
date: 2014-03-18 22:11:10.998000000

Patch Set 2:

Tony, do you have an example when installation script is not aborted?
http://qa.hq.northscale.net/job/centos_x64--00_02--compaction_tests-P0/2/console

-------------------------------------
author: Frederick Booker
date: 2014-03-18 22:34:13.438000000

Patch Set 2:

It is aborted but not right after it could not ssh to vm.
Also I did saw some cases it was not aborted (1 or 2 vms did not install cbsv)
and ran till the end of installation.

-------------------------------------
author: Esperanza Cherry
date: 2014-03-25 10:16:47.607000000

Patch Set 2:

Hi Tomi,

example http://qa.hq.northscale.net/view/All/job/ubuntu_x86--13_02--swap_rebalance-P0/23/console

installation aborted after two failed attempts to connect

2014-03-24 17:55:33,516 - root - ERROR - Can't establish SSH session: Error reading SSH protocol banner[Errno 104] Connection reset by peer
2014-03-24 17:55:52,657 - root - ERROR - Can't establish SSH session: Error reading SSH protocol banner[Errno 104] Connection reset by peer

I think if we remove max_attempts_connect=2 script will be terminated imediately. 

what do you think?

-------------------------------------
author: Esperanza Cherry
date: 2014-03-25 10:47:21.628000000

Patch Set 2:

forgot link https://github.com/couchbase/testrunner/blob/master/lib/remote/remote_util.py#L146

-------------------------------------
