Load-balanced discover command

The discoverloadbalanced command allows a discovery to be run in load-balancing mode, which means discovery is continuous.

About load-balanced discovery

Load-balanced, continuous discovery is achieved by using a pool of discovery servers. This maximizes the utilization of servers and guards against failover, thereby preventing discoveries in progress from being interrupted.

Discovered environments can be separated into areas, with each drawing on its own pool of servers. Furthermore, each discovery can be performed by entering sets of IP addresses to define the scope of the discovery.

The PrimaryStorageServer (PSS) acts as a load-balancing discovery controller, allocating scope and resources to each discovery server (DS). Should the PSS fail, all running discoveries will be completed, but no new ones allocated.

The PSS controls discovery by placing work into a queue, but does not push tasks to a DS; instead, each participating DS actively requests work from the queue when it is ready, and then signals 'in progress' as the discovery progresses. If no 'in progress' signal is received, the PSS reallocates the work.

When running in continuous (or load-balanced) mode, the discovery server starts another discovery as soon as DiscoveryWorker threads are free to be used, even before the previous discovery is finished. The continuous discovery command is available only to the primary storage server (in enterprise mode) and domain server (in single domain mode).

Tip: Sensors storing data are displayed on the UI as 'in progress'. This is not the same as the number of threads running a discovery.

Load-balancing discovery scenarios

Load-balanced HA discovery based against scope group: A user runs a discovery against a defined scope group via command line interface (CLI). The pool of servers is used to optimize the discovery workload, based on the scopes (from single to multiple scope) being part of the scope group.
Dynamic generation of the scope group: A user runs a load-balanced discovery using a list of IP addresses provided as a file. This creates a scope group with automatically generated child scopes, with IP addresses divided between child scopes.

Command syntax

As for a standard discovery using the discover command, the continuous load-balanced discovery functionality (discoverloadbalanced) is controlled using api.sh.

Parameters

api.sh -u <user> -p <password> discoverloadbalanced start --poolName <poolName> --scopeGroup <scopeGroup> [--profile <profileName>]: Starts continuous, load-balanced discovery against each scope in <scopeGroup>, and on each server belonging to the discovery pool specified in <poolName>
api.sh -u <user> -p <password> discoverloadbalanced startFromFiles --files <filePath1,filePath2,...> --maxScopeSize <size> [--profile <profileName>]: Starts continuous, load-balanced discovery against each scope passed via --files argument. Each file is parsed, and a new group named from file name (without extension) is created. In each group scopes are added with the maximum size specified in maxScopeSize. poolName is assumed to be the same as groupName
api.sh -u <user> -p <password> discoverloadbalanced startFromDirectory --dir <directory path> --maxScopeSize <size> [--profile <profileName>]: As in the --files example, but starts discovery against each file in the directory.
api.sh -u <user> -p <password> discoverloadbalanced status: Prints the current status of the load-balanced discovery.
api.sh -u <user> -p <password> discoverloadbalanced abort <poolName>: Aborts the discovery for the poolName specified.
api.sh -u <user> -p <password> discoverloadbalanced abort --poolName <poolName> --scopeSet <scopeSetName>: Aborts the discovery for a specified scopeSet running or scheduled within the specified poolName.
api.sh -u <user> -p <password> discoverloadbalanced pause <poolName>: Stops discovery for the specified poolName, and all scopes in the discovery run are moved back to the 'forTake' state. Current discoveries are aborted.
api.sh -u <user> -p <password> discoverloadbalanced resume <poolName>: Resumes paused discovery for the specified poolName, and all scopes in the 'forTake' state are now available for processing.

Properties for the primary storage server

Set the following properties for the primary storage server (or domain):

com.ibm.cdb.internalscheduling.discoverypool.scopePrefix: Specifies prefix for automatically created scopes (for startFromFiles and startFromDirectory options); Default: com.ibm.cdb.internalscheduling.discoverypool.scopePrefix=_auto_
com.ibm.cdb.internalscheduling.discoverypool.timeToNonAlive: Specifies time in seconds, that is, how long discovery is assumed to be running without any information about status received from the discovery server.; Once time has passed, the scope is moved back to the 'forTake' state, so it can be redone by the other server which is still responding.; Default: com.ibm.cdb.internalscheduling.discoverypool.timeToNonAlive=180

Properties for the discovery server

Set the following properties for the discovery server (or domain):

com.ibm.cdb.internalscheduling.discoverypool.enabled: Enables the discovery server to be part of the discovery pool.; Default: com.ibm.cdb.internalscheduling.discoverypool.enabled=false
com.ibm.cdb.internalscheduling.discoverypool.name: Defines poolName for a discovery server; Default: com.ibm.cdb.internalscheduling.discoverypool.name=DEFAULT
com.ibm.cdb.internalscheduling.discoverypool.checkinginterval: Specifies the time in seconds that the discovery server checks for new jobs to take (if ready), also how often to inform PSS about a job in progress.; The interval specified needs to be smaller than com.ibm.cdb.internalscheduling.discoverypool.timeToNonAlive on a primary storage server, otherwise scopes might be returned to be re-executed.; Default: com.ibm.cdb.internalscheduling.discoverypool.checkinginterval=20

Note: Logs from the continuous discovery are stored in the ApiServer.log on the domain, or the primary storage server.