DESCRIPTION

JVMCBC-312: use the same logical connection for all DCP nodes

Motivation
----------
DCP connection have to be opened on each node separately, but from the
SDK perspective, there is only one logical connection and stream. But
current implementation was creating logical connection for every node,
and only first connection was actually visible to the consumer.

Modification
------------
Maintain map of logical connection, and reuse it when new DCP connection
with the same name has been requested. Also dispose the connection only
when all streams from all nodes has been closed.

Result
------
No missing messages anymore

Change-Id: Iedd8268eddd898677844a80c32cb184c5db9a7ad


COMMENTS

author: Cory Peck
date: 2016-03-22 10:33:31.368000000

Uploaded patch set 4.

-------------------------------------
author: Estrella Humphrey
date: 2016-03-23 09:20:20.273000000

Patch Set 4:

(10 comments)

Line:95, src/integration/java/com/couchbase/client/core/cluster/DCPConnectionTest.java -> the only assertion here is that the status is a SUCCESS, is that enough to assert a rollback took place?

Line:157, src/integration/java/com/couchbase/client/core/cluster/DCPConnectionTest.java -> not sure where the flow control happens when reading the test, could be worth putting a comment around the area of interest (as tests can be used to learn about a codebase)

Line:230, src/integration/java/com/couchbase/client/core/cluster/DCPConnectionTest.java -> maybe we should make that a utility function somewhere in xxx@xxx.xxx what do you think?

Line:886, src/main/java/com/couchbase/client/core/env/DefaultCoreEnvironment.java -> note: don't forget the associated change in java-client (and document this setter)

Line:378, src/main/java/com/couchbase/client/core/env/CoreEnvironment.java -> please document

Line:42, src/integration/java/com/couchbase/client/core/util/TestProperties.java -> please revert that

Line:101, src/main/java/com/couchbase/client/core/endpoint/dcp/DCPConnection.java -> double-checking: shouldn't the default be 0xffffffff for snapshotEndSequenceNumber as well?

Line:206, src/main/java/com/couchbase/client/core/endpoint/dcp/DCPConnection.java -> response status is the partition number??

Line:53, src/main/java/com/couchbase/client/core/endpoint/dcp/DCPConnectionHandler.java -> copy-pasta javadoc :)
make sure to redo the javadoc in this file, esp. any mention of SASL

Line:300, src/main/java/com/couchbase/client/core/CouchbaseCore.java -> does an OpenConnectionRequest implies an openBucket request?
so far IIRC we've been doing that sort of pre-requisites in the java-client layer, and core messages would just assume preconditions like xxx@xxx.xxx wdyt?

-------------------------------------
author: Marques Ellis
date: 2016-03-23 17:45:53.214000000

Patch Set 4: Verified-1

This patch does not compile with 2.2.5 client due to the changes in DefaultCoreEnvironment

-------------------------------------
author: Cory Peck
date: 2016-03-24 10:21:18.836000000

Patch Set 4:

(10 comments)

Line:95, src/integration/java/com/couchbase/client/core/cluster/DCPConnectionTest.java -> yes, just adding stream with incorrect range returns the error from server, addStream() wrapper will handle rollbacks here

Line:157, src/integration/java/com/couchbase/client/core/cluster/DCPConnectionTest.java -> I relies on these settings in environment

            .dcpConnectionBufferSize(1024)          // 1 kilobyte
            .dcpConnectionBufferAckThreshold(0.5)   // should trigger BUFFER_ACK after 512 bytes

How can I make explicit it? If flowcontrol fails, we get less messages. The tests counts the returned values.

Line:230, src/integration/java/com/couchbase/client/core/cluster/DCPConnectionTest.java -> I will move it into ClusterDependentTest

Line:886, src/main/java/com/couchbase/client/core/env/DefaultCoreEnvironment.java -> Done

Line:378, src/main/java/com/couchbase/client/core/env/CoreEnvironment.java -> Done

Line:42, src/integration/java/com/couchbase/client/core/util/TestProperties.java -> Done

Line:101, src/main/java/com/couchbase/client/core/endpoint/dcp/DCPConnection.java -> no, it's okay

Line:206, src/main/java/com/couchbase/client/core/endpoint/dcp/DCPConnection.java -> yes

https://github.com/couchbase/memcached/blob/master/include/memcached/protocol_binary.h#L534

Line:53, src/main/java/com/couchbase/client/core/endpoint/dcp/DCPConnectionHandler.java -> done, but there is currently limitation here, that I cannot pass connection name for DCP (like we are doing with username/password for SASL)

Michael, what we can do for it? At the moment I made it global for environment through the configuration.

Line:300, src/main/java/com/couchbase/client/core/CouchbaseCore.java -> okay, I will extract it from here

-------------------------------------
author: Gerrit Code Review
date: 2016-03-30 07:02:23.834000000

Change has been successfully cherry-picked as bddbdae7e92e13112e649eca4d35f786599f4493

-------------------------------------
