Deploying a multi-tenant Synthetic PoP

With a multi-tenant point of presence (PoP), you can monitor multiple customer environments through a single deployment of the Instana Synthetic Monitoring service. Each environment is isolated from others but shares resources such as hardware and network infrastructure.

Multi-tenant PoP is supported on Synthetic PoP 1.1.8 and later.

Architecture

The architecture for the multi-tenant PoP in a Kubernetes cluster is shown in the following diagram. arch

The playback engine pods and Redis pod are deployed in a common namespace. They are shared by multiple tenant units. You can deploy multiple playback engines depending on the workload of each test type, such as X HTTP playback engines, Y JavaScript playback engines, Z BrowserScript playback engines.

For each tenant unit, a single PoP controller pod must be deployed in the tenant unit namespace to connect to its Instana backend server. Different tenant unit's PoP controller pod can connect to the same or different Instana backend server. All tenant tenants' PoP controller pods communicate with the shared playback engines through the shared Redis pod.

For each tenant unit, a single PoP controller pod must be deployed in the tenant unit namespace to connect to its Instana backend server. The PoP controller pod of another tenant unit can connect to the same Instana backend or another Instana backend. All tenant PoP controller pods communicate with the shared playback engines through the shared Redis pod.

Multi-tenant PoP has the following benefits compared to single-tenant PoP:

  • Fast deployment for a tenant unit: A single-tenant PoP deployment takes about 5 minutes for a tenant unit whereas you can deploy a PoP controller in multi-tenant PoP in one minute.
  • Cost-effectiveness: A multi-tenant PoP solution for the same workload costs 50% less.
  • Easy to scale: You can scale the playback engine pods in the common namespace for all tenant units.
  • Centralized management capabilities: You can manage all customer environments from a single interface.

Installing a multi-tenant Synthetic PoP

To install a multi-tenant PoP, complete the following steps:

  1. Install shared Redis and playback engines for all tenant units in a common namespace.
  2. Install a PoP controller in the tenant unit's namespace for each tenant unit.

Installing shared components

Install Redis and playback engines in a common namespace as shown in the following example:

helm install synthetic-pop \
    --repo https://agents.instana.io/helm  \
    --namespace "<common-namespace>" \
    --create-namespace \
    --set downloadKey="yourdownloadkey" \
    ## disable controller
    --set controller.enabled=false \
    ## redis server: need to have a large resource allocation
    --set redis.tls.enabled=false \
    --set redis.password="a1fc5d01bcbb" \
    --set redis.resources.requests.cpu="200m" \
    --set redis.resources.limits.cpu="1000m" \
    --set redis.resources.requests.memory="300Mi" \   
    --set redis.resources.limits.memory="1000Mi" \
    ## http playback engine, no need to change resource request/limit
    --set http.replicas=2 \
    ## javascript playback engine is using single-thread 
    ## In this example, 3 javascript playback engine pods and each of them is running test in single-thread
    ## so totally 3 API scripts can be run in parallel
    --set javascript.replicas=3 \
    --set javascript.maxConcurrentTests=1 \
    ## browserscript playback engine is using single-thread with reduced resource request/limit
    ## In this example, 3 browserscript playback engine pods and each of them is running test in single-thread
    ## so totally 4 browser scripts can be run in parallel
    --set browserscript.replicas=4 \
    --set browserscript.maxConcurrentTests=1 \
    --set browserscript.resources.requests.cpu="2000m" \
    --set browserscript.resources.limits.cpu="3500m" \
    --set browserscript.resources.requests.memory="1000Mi" \
    --set browserscript.resources.limits.memory="2000Mi" \
     ## ism playback engine, no need to change resource request/limit
    --set ism.replicas=2 \
 synthetic-pop

Notes:

  • PoP controller pod is disabled.
  • Redis pod's CPU/Mem resource limit is increased.
  • Single-thread is applied in the JavaScript and BrowserScript playback engine to run API scripts and browser script tests to ensure isolation when scripts are run for different tenants in the same container.
  • BrowserScript playback engines' CPU/Mem resource requests/limits are reduced because single-thread is applied to running browser script tests,
  • Evaluate the total workload and set the replica number for each playback engine.
  • Redis password is used in the example. You can also use Redis TLS to encrypt the communication through Redis, the parameters are the same as in the single-tenant PoP deployment.

Installing PoP controller

For each tenant unit, install the PoP controller pod in a tenant unit-specific namespace as shown in the following example:

helm install synthetic-pop \
    --repo https://agents.instana.io/helm \
    --namespace "<tenant-unit-namespace>" \
    --set downloadKey="yourdownloadkey" \
    --set controller.location="MyPoP;My PoP;MyCounty;MyCity;0;0;This is a testing Synthetic Point of Presence" \
    --set controller.instanaKey="instanaAgentkey" \
    --set controller.instanaSyntheticEndpoint="https://synthetics-green-saas.instana.io" \
    # specify the common namespace where to deploy redis and playback engines
    --set controller.redisNamespace="<common-namespace>" \
    # use redis password for communication through redis
    --set redis.tls.enabled=false \
    --set redis.password="a1fc5d01bcbb" \
    # disable redis and playback engine pods
    --set redis.enabled=false  \
    --set http.enabled=false  \
    --set javascript.enabled=false  \
    --set browserscript.enabled=false  \
    --set ism.enabled=false  \
synthetic-pop

Notes:

  • Install only pop controller pod and disable all other pods.
  • Make sure that the correct common namespace parameter is passed through --set controller.redisNamespace.
  • Redis password is used in the example. You can also use Redis TLS to encrypt the communication through Redis, the parameters are the same as in the single-tenant PoP deployment.

Upgrading a multi-tenant PoP

To upgrade a multi-tenant PoP, complete the following steps:

  1. Upgrade the Redis and playback engine pods with the "helm upgrade" command in the common namespace.
  2. Upgrade the PoP controller pod per tenant unit with "helm upgrade" command in the tenant unit namespace.

Scaling playback engines

The number of tests that are supported with a single playback engine pod for each test type is listed in the following table:

Test type Frequency Duration Complexity Number of tests
API Simple test 1 minute ~ 200 ms 2000
API Script test 5 minutes ~ 800 ms issuing 5 HTTP calls 80
Browser Script test 15 minutes ~ 20 seconds opening 2 Web pages 12
ISM test 1 minute ~ 240 ms 2000

Depending on the total workload of each test type, change the number of replica playback engines to support horizontal scalability as shown in the following example:

helm upgrade synthetic-pop \
    --repo https://agents.instana.io/helm  \
    --namespace "<common-namespace>" \
    --set http.replicas=2 \
    --set javascript.replicas=4 \
    --set browserscript.replicas=5 \
    --set ism.replicas=2 \
    synthetic-pop

Monitoring a multi-tenant PoP

To monitor the multi-tenant PoP, install the Instana host agent on the Kubernetes cluster where your PoP is installed. Synthetic PoP sensor is automatically installed to monitor your PoP after the Instana host agent is installed.

The health data in each PoP shown in the monitoring dashboard is collected from PoP controller in the tenant unit namespace. The workload data represents the playback statistics for this tenant unit.

The "Queued Tasks" metric in the workload chart is showing the total queued tasks in the Redis for all tenant units instead of this tenant unit.

Uninstalling a multi-tenant PoP

To uninstall a multi-tenant PoP, complete the following steps:

  1. Uninstall the Redis and playback engine pods, run the following command in the common namespace:
    helm uninstall synthetic-pop -n "<common-namespace>"
  2. Uninstall the PoP controller pod from every tenant unit, run the following command in the tenant unit namespace:
    helm uninstall synthetic-pop -n "<tenant-unit-namespace>"

Migrating from single-tenant PoP to multi-tenant PoP

To migrate single-tenant PoPs to a multi-tenant PoP, complete the following steps:

  1. Install shared Redis and playback engine pods first in the common namespace with the helm install command and set the number of playback engine replicas based on the total workload from all single-tenant PoPs.
  2. For each tenant Unit, upgrade its single-tenant PoP to a single PoP controller of the multi-tenant PoP with the helm upgrade command. Make sure to pass appropriate parameters.