DESCRIPTION

JVMCBC-385: Implement Generic Service Pool

Motivation
----------
core-io had a longstanding limitation that the number of endpoints,
practically speaking, has always been fixed. While configurable, in
practice this means fiddling around with config settings and as a
result providing a non-optimal user experience. It also makes it
hard to shut off sockets if they are not needed during idle
periods.

A recent change which disables pipeling has further fueled the need
for such a change, since it led to decreased performance out of the
box.

Modifications
-------------
This, honestly quite big, changeset changes the situation. While it
touches many files it boils down to a couple of distinct changes
that ultimately leady to a flexible service pool implementation.

Here is how it works:

A ServiceConfig per service is created on startup in the environment
and then used to configure the ServicePool. KV service is the only
one which has always a fixed number of endpoints since right now
when pipelining is enabled only min==max works (since the pipelined
endpoints would always accept more requests, so when start adding
new ones? this will be fixed later).

At connect time, the minimum number of endpoints is established. When
a new messages comes in through send and no endpoint can be used
(through the selection strategy) the logic to grab a new endpoint is
executed. Of course if max is reached, the op is just retried. This
also happens when the new endpoint couldn't be opened.

Finally, log is in place that tears down sockets which are idle longer
than the configured period. If its 0 then its disabled. Note that
additional logic is in place which makes sure we never go below
the minimum, if this happens new endpoints are spawned proactively.

* Supporting changes
 - AbstractGenericHandler now tracks "KeepAlive" messages with a
   marker interface.
 - Endpoints track if they are free and track the last timestamp
   when a message has been completed.
 - The Environment now has a ServiceConfig that is used to configure
   the services instead of a simple "num endpoints"

* Main changes
 - the PoolService implements the pool functionality and the
   PoolServiceTest file has all the relevant tests for it
 - The old pools have not been removed since the config and DCP
   service still use it, a ticket has been filed to remove that
   in subsequent changesets.
 - The selection strategy has been modified to work with a list
   and not an array

Result
------
The SDK now by default has a flexible pool structure for N1QL, Views
and Fulltext.

Change-Id: Ibdeeb975949a8e2262d41c9d70dd369c14170961
Reviewed-on: http://review.couchbase.org/72365
Reviewed-by: Marques Ellis xxx@xxx.xxx
Tested-by: Natasha Sullivan xxx@xxx.xxx


COMMENTS

author: Natasha Sullivan
date: 2017-02-05 19:43:01.996000000

Change has been successfully cherry-picked as 40a93d713872740a3e2a0807c61f961ba3ee7805 by Natasha Sullivan

-------------------------------------
