DESCRIPTION

JVMCBC-184: Prevent race condition on topology change with >1 buckets.

Motivation
----------
When more than one bucket is open at the same time a race condition combined
with our "safety rescheduling" logic prevents the system from getting back
into a stable state.

The two configurations for the buckets race each other with the reconfiguration
logic and the operations always get rescheduled.

In addition, the retry logic needs to be in place for memcached buckets as well,
since the gap can be there present too.

Modifications
-------------
The change is quite trivial in that the rescheduling logic is now only executed
as a fallback when the regular node could not be selected beforehand. This allows
the operations to pass while just to retry the subset which is affected by
the temporary topology difference between the config nodes and the actual
instantiated node list.

Also the retry logic is applied for memcached buckets.

Result
------
More stable behaviour during rebalance when more than one bucket is open.

Change-Id: If8697a4ce4f01e0f1e68a02d0b521c6631fdd7bc


COMMENTS

author: Natasha Sullivan
date: 2015-04-17 11:03:37.369000000

Uploaded patch set 2.

-------------------------------------
author: Natasha Sullivan
date: 2015-04-17 11:04:31.668000000

Patch Set 2: Verified+1

-------------------------------------
author: Estrella Humphrey
date: 2015-04-17 14:17:18.861000000

Patch Set 2: Code-Review+2

-------------------------------------
author: Gerrit Code Review
date: 2015-04-17 16:49:59.564000000

Change has been successfully cherry-picked as f63977c7f382c49239684dc0c53d27fb21272c70

-------------------------------------
