Using scalability mechanism

Edit online

OMEGAMON® AI Insights V 2.1 brings scalability mechanism. Predict and train containers can have multiple instances to distribute the load of multiple contexts.

Configuration can be added before starting the containers through docker-compose file.

To add a new instance of container, configuration should be duplicated or modified for predict or train action.

The following snippet is the extract of the docker-compose.yml file. Check the fields marked in bold.
  • A set of parameters should be added per container instance.
  • Name, REST_PORT and GRPC_PORT should be different for each container instance.
  • Test URL should be adapted for container healthcheck.
  • The ml-orchestrator service must wait for the complete start of all the ml-services defined above. The full list of ml-services should be provided.
  ml-service-training:      <- name of the first training container 
    image: dockerrepository/kmua/kmua-service-training:2.1.0 
… 
      test: curl -s http://localhost:8002/health | grep -q '"status":[ ]*"UP"' 
…   
    REST_PORT: 8002 
    GRPC_PORT: 50052 
  … 
  ml-service-training-1:      <- New second training container 
    image: dockerrepository/kmua/kmua-service-training:2.1.0 
… 
      test: curl -s http://localhost:8012/health | grep -q '"status":[ ]*"UP"' <- check port 
…   
    REST_PORT: 8012  <- REST port for second training container 
    GRPC_PORT: 50062 <- GRPC port for second training container 
    EUREKA_CLIENT: ml-service-training-1
  … 
  ml-service-forecast: <- name of the first forecast container 
    image: dockerrepository/kmua/kmua-service-forecast:2.1.0 
… 
      test: curl -s http://localhost:8001/health | grep -q '"status":[ ]*"UP"' 
… 
      REST_PORT: 8001 
      GRPC_PORT: 50051 
      EUREKA_CLIENT: ml-service-forecast
… 
  ml-service-forecast-1:  <- name of the second forecast container 
    image: dockerrepository/kmua/kmua-service-forecast:2.1.0 
… 
      test: curl -s http://localhost:8011/health | grep -q '"status":[ ]*"UP"' <- check port 
… 
      REST_PORT: 8011 <- REST port for second forecast container 
      GRPC_PORT: 50061 <- GRPC port for second forecast container 
      EUREKA_CLIENT: ml-service-forecast-1
… 
 ml-orchestrator:
    image: wal-artifactory.rocketsoftware.com:6579/kmua/kmua-service-orchestrator:2.1.0
    healthcheck:
      start_period: 45s
      interval: 10s
      retries: 3
      test: curl -s -f http://localhost:8070/actuator/health | grep -q '"status":[ ]*"UP"'
    depends_on:
      ml-service-training:
        condition: service_healthy
      ml-service-training-1:
        condition: service_healthy        
      ml-service-forecast:
        condition: service_healthy
      ml-service-forecast-1:
        condition: service_healthy