Installing the Management subsystem into a Kubernetes environment

Use the Install Assist utility program to install the Management subsystem into your Kubernetes runtime environment.

Before you begin

Before installing the Management subsystem, complete the steps described in First steps for installing API Connect: Upload files to registry. For instructions on using the APICUP installer, see Tips and tricks for using APICUP.

Procedure

The commands for installing the Management subsystem are described in the following table. Sample code is also provided. These are the basic commands for installing an API Connect Management service in a Kubernetes cluster. Namespaces must be configured in Kubernetes before starting the installation process. We do not recommend that you use the default namespace. DNS domain names must be configured for the endpoints. Resource allocation sizes are recommended minimums and will depend upon your environment. It is highly recommended that you include the backup schedule for the management database.

Note:

If the create-crd command is set to false, then the Kubernetes administrator is responsible for creating CRD's before installing the management subsystem. See Manually creating a CustomResourceDefinition in a Kubernetes environment.
Host names and DNS entries may not be changed on a cluster after the initial installation.
Domain names that are used for endpoints cannot contain the underscore "_" character. See Pre-installation requirements.

A few lines in the following commands set multiple values with a single set command. For instructions on configuring multiple settings per command and other tips for using apicup, see Tips and tricks for using APICUP

Command Values/Definition

apicup init Initializes the apicup installation utility and creates the apiconnect-up.yml configuration file

apicup subsys create mgmt management --k8s

Describes a management service in the Kubernetes cluster . The identifier mgmt is the name you have assigned to your management service. You can assign it any name, as long as the identifier consists of lower case alphanumeric characters or '-', with no spaces, starts with an alphabetic character, and ends with an alphanumeric character.

The command management indicates that you are creating a management subsystem. The subsystem will be deployed in dev mode by default, unless standard mode is explicitly specified.

apicup subsys set mgmt extra-values-file <path/file name> - Path to the extra values file. One extra values file is permitted. The extra values file requires a .yaml format. It contains the ingress annotations and other settings for the subsystem. For an example of an extra values file for each subsystem, see Creating an extra values file in a Kubernetes environment.

apicup subsys set mgmt ingress-type Must be explicitly set to route for an OpenShift environment. For a standard K8s environment, ingress-type defaults to ingress and there is no need to explicitly enter this parameter.

apicup subsys set mgmt mode

Determines whether the installation is for development, testing, and demo purposes or for a production environment. If not explicitly set, the mode will default to dev (development and testing) for versions 2018.4.1.4 and later. For versions 2018.4.1.3 and earlier, the default for mode is standard.

dev - (Default) Use dev mode for development, testing, and demo purposes. It allows one instance of each subsystem to be created.
standard - Use standard mode for production environments. It allows three containers each containing one instance of the subsystem to be created, for a total of three instances.

apicup subsys set mgmt namespace <namespaceName> - Defines the namespace where the management service resides; every subsystem can be in a separate namespace. Valid namespaces are configured in Kubernetes prior to starting the API Connect installation.

apicup subsys set mgmt registry <registry-url> - The location where you are running the image registry (for example, Artifactory)

apicup subsys set mgmt registry-secret For example, <registrySecret> - Kubernetes secret with Docker credentials to authenticate with the image registry. The value for registry-secret must match the name of the secret created using the kubectl create secret command. It contains the Docker credentials for accessing the registry.

apicup subsys set mgmt storage-class

<storageClass> - Refers to a valid StorageClass object in your Kubernetes cluster that supports dynamic PersistentVolume provisioning. Static or manual PersistentVolume provisioning can be used, but it requires additional steps not covered in this documentation. Block storage is required. Select a block storage format of your choosing in order for API Connect components to be supported. GlusterFS is not recommended as a storage option due to severe performance degradation. For AWS, the gp2 and io1 types of Elastic Block Storage (EBS) are supported. Note that NFS is not supported.

apicup subsys set mgmt platform-api

api.<hostname>.com - The endpoint that is exposed by Kubernetes to access the admin and provider Platform REST APIs. The endpoints have to be resolvable by DNS. Requires a FQDN, in lowercase. Domain names that are used for endpoints cannot contain the underscore "_" character. See Pre-installation requirements.

Refer to the deployment overview diagram for information about endpoints and certificates: Deployment overview for endpoints and certificates.

apicup subsys set mgmt api-manager-ui apim.<hostname>.com - The endpoint that is exposed by Kubernetes to access the API Manager user interface. Requires a FQDN, in lowercase. Domain names that are used for endpoints cannot contain the underscore "_" character. See Pre-installation requirements.

apicup subsys set mgmt cloud-admin-ui

cm.<hostname>.com - The endpoint that is exposed by Kubernetes to access the Cloud Manager user interface. Requires a FQDN, in lowercase. Domain names that are used for endpoints cannot contain the underscore "_" character. See Pre-installation requirements.

apicup subsys set mgmt consumer-api consumer.<hostname>.com - The endpoint that is exposed by Kubernetes to access the consumer REST APIs. Requires a FQDN, in lowercase.Domain names that are used for endpoints cannot contain the underscore "_" character. See Pre-installation requirements.

cassandra-backup-auth-user (Optional, not required to complete the installation, but highly recommended to configure backups.) The username for the server specified in cassandra-backup-host. If using object store, this would be the S3 Secret Key ID.

cassandra-backup-auth-pass

(Optional, not required to complete the installation, but highly recommended to configure backups.) The password for the server specified in cassandra-backup-host. If using object store, this would be the S3 Secret Access Key parameter. The password will be stored in Base64 encoded format.

For example:

apicup subsys set mgmt cassandra-backup-auth-pass '<password>'

Note that you cannot use the `=` sign to assign the password to cassandra-backup-auth-pass.

cassandra-backup-host (Optional, not required to complete the installation, but highly recommended to configure backups.) The fully qualified domain name of the backup server, in lowercase only. Ensure that the Kubernetes nodes can access this host. If using object store, enter Endpoint/Region. (The "/" character between the endpoint and region are required for this setting.

cassandra-backup-port (Optional, not required to complete the installation, but highly recommended to configure backups.) The port for the protocol to connect to the cassandra-backup-host. The backup port is not required for object storage.

cassandra-backup-protocol

(Optional, not required to complete the installation, but highly recommended to configure backups.) The backup protocol. Specify one of the following values:

sftp - for secure file transfer protocol
objstore - for S3 compatible object storage

cassandra-backup-path (Optional, not required to complete the installation, but highly recommended to configure backups.) The full path to the directory where the backup files will be stored. For object storage (objstore) , the path can be set to the bucket value or the bucket/subfolder value.

cassandra-backup-schedule

(Optional, not required to complete the installation, but highly recommended to configure backups.) Cron like schedule for performing automatic backups. The format for the schedule is:

* * * * *
- - - - -
| | | | |
| | | | +----- day of week (0 - 6) (Sunday=0)
| | | +------- month (1 - 12)
| | +--------- day of month (1 - 31)
| +----------- hour (0 - 23)
+------------- min (0 - 59)

For example: 30 22 * * 1 will perform backups at 10:30 pm on Mondays. The timezone for backups is that of the node on which the Cassandra pod is scheduled.

The backup schedule defaults to 0 0 * * *. This means a backup is run every day at midnight and minute zero UTC.

When you configure a host, if you do not specify a value for cassandra-backup-schedule, the default backup schedule is automatically set. Note that the default backup schedule is not set, and scheduled backups not enabled, until host configuration is completed.

apicup subsys set mgmt cassandra-max-memory-gb

This command sets the total memory allocated to the Cassandra server. This setting is used to calculate the MAX_HEAP_SIZE value of the server which is always 47% of the total memory allocated to Cassandra server. The default and minimum allowed value for cassandra-max-memory-gb is 9GB.

The following table compares values for memory allocation for the management database:

Memory allocated to Kubernetes node where Cassandra is deployed	Recommended cassandra-max-memory-gb	MAX_HEAP_SIZE (47% of cassandra-max-memory-gb value)
16GB	9GB (default)	4331m
32GB	16GB	7700m
64GB	24GB	11550m

apicup subsys set mgmt cassandra-cluster-size 3 is a recommended minimum; actual value will depend upon your environment.

apicup subsys set mgmt cassandra-volume-size-gb The default value is 50GB, which is the minimum recommended setting for a non-production environment; the minimum recommended setting for a production environment is 250GB. The actual value will depend upon your environment. The cassandra-volume-size-gb parameter is a PER NODE value, so the total amount of storage space for the management database will be the cassandra-volume-size-gb multiplied by the number of nodes. IMPORTANT: The storage size for the management database on a node cannot be increased once the pod is deployed to the node.

apicup subsys set mgmt create-crd

Options are true or false. The default value is true. When set to true, the custom resources required by the management subsystem are automatically created at the cluster level. The management database requires the following two CustomResourceDefinitions (CRDs):

cassandraclusters.apic.ibm.com - stores the custom resource that contains the specs for the management database cluster
cassandraclusterbackups.apic.ibm.com - stores the custom resource that contains information for each completed backup

When set to false, the Kubernetes administrator must manually create the CRDs before installing the management subsystem. Note that CustomResourceDefinitions require ClusterRole privileges. For instructions on how to manually create the CRDs, see Manually creating a CustomResourceDefinition in a Kubernetes environment.

apicup subsys set mgmt external-cassandra-host <host name> - Hostname where a Cassandra database cluster can be reached, if not using the Cassandra Operator bundled with API Connect.

apicup certs set mgmt CERT_NAME

[CERT_FILE KEY_FILE CA_FILE][flags] - Set the required certificates for the subsystem. Custom certificates may be set, otherwise, default certificates are automatically configured. For information on what certificates are available and how to set them see: Working with certificates.
[KEY_FILE][flags] - Set the encryption secret for the management database.
Important: The encryption secret can only be set once and only during initial installation. See Setting the encryption-secret for the management database.

apicup subsys set mgmt license-version
<license_type>

<license_type> - Specify the type of license, either Production or Nonproduction. If not specified, the default value is Nonproduction.

Note that the license type does not impact which Management server image to install. The API Connect Kubernetes Pod annotations productID and productName reflect the selected license.

apicup subsys install mgmt - Starts the installation of the management service.

Note: API Connect v2018 cannot be deployed on NFS.


apicup subsys create mgmt management --k8s
apicup subsys set mgmt extra-values-file=<path/file name>
apicup subsys set mgmt ingress-type=<route - for OpenShift environments only> <ingress - default>
apicup subsys set mgmt mode=<dev>
apicup subsys set mgmt namespace=<namespaceName>
apicup subsys set mgmt registry=<registry-url>
apicup subsys set mgmt registry-secret=<registrySecret>
apicup subsys set mgmt storage-class=<storageClass>

apicup subsys set mgmt platform-api=my.platform.com api-manager-ui=my.apim.com cloud-admin-ui=my.cloud.com consumer-api=my.consumer.com

apicup subsys set mgmt cassandra-backup-auth-user=MyUsername cassandra-backup-auth-pass 'MyPassword'
apicup subsys set mgmt cassandra-backup-host=<hostname>
apicup subsys set mgmt cassandra-backup-path=</backups>
apicup subsys set mgmt cassandra-backup-port=<22>
apicup subsys set mgmt cassandra-backup-protocol=<sftp, objstore>
apicup subsys set mgmt cassandra-backup-schedule=<"0 0 * * *"> 
apicup subsys set mgmt cassandra-max-memory-gb=9 cassandra-cluster-size=3                
apicup subsys set mgmt cassandra-volume-size-gb=<50>
apicup subsys set mgmt create-crd=<true>
apicup subsys set mgmt external-cassandra-host=<hostname>

//Set the certificates 
apicup certs set mgmt CERT_NAME=[CERT_FILE KEY_FILE CA_FILE] [flags] <sets certificates>
apicup certs set mgmt CERT_NAME=[KEY_FILE] [flags] <sets the encryption secret>

// OPTIONAL: Write the plan to an output directory to inspect myProject/install-plan prior to installation
apicup subsys install mgmt --out=mgmt-install-plan 
// Start the installation from the output directory. Enter the full path to the output directory.
apicup subsys install mgmt --plan-dir=./myProject/mgmt-install-plan
//If install-plan is not used, enter the following command to start the installation
apicup subsys install mgmt

Important: The management database requires mmap counts higher than the operating system defaults. To change the mmap counts on the live system, run sudo sysctl -w vm.max_map_count=1048575 on each Kubernetes node. To persist this change when node restarts occur, add the following to the /etc/sysctl.conf file: vm.max_map_count = 1048575.

You can use the get command to print out the settings for the subsystem prior to installing it. For example, apicup subsys get mgmt, where mgmt is the name of the subsystem. The get command displays any default settings for the subsystem and the settings from the apiconnect-up.yml file. The Errors column indicates if a parameter is invalid. Following is an example of output received from apicup subsys get mgmt:

Kubernetes settings
===================

Name				Value				Errors
----				-----				------
extra-values-file
ingress-type			ingress
mode				 standard
namespace			   default
registry
registry-secret
storage-class


Subsystem settings
==================

Name				Value				Errors
----				-----				------
cassandra-backup-auth-pass
cassandra-backup-auth-user
cassandra-backup-host
cassandra-backup-path	    /backups
cassandra-backup-port	    22
cassandra-backup-protocol	sftp
cassandra-backup-schedule	0 0 * * *
cassandra-cluster_size	   1
cassandra-max-memory-gb	  9
cassandra-volume-size-gb	100
create-crd			true
external-cassandra-host


Endpoints
==================

Name				Value			     Errors
----				-----			     ------
api-manager-ui							api-manager-ui is not a valid FQDN
cloud-admin-ui							cloud-admin-ui is not a valid FQDN
consumer-api						         consumer-api is not a valid FQDN
platform-api						         platform-api is not a valid FQDN

Correct any errors indicated by the output from the get command and then run apicup subsys install <SUBSYS> to install the subsystem.

After running the apicup subsys install <SUBSYS> command, run the apicup subsys get <SUBSYS> --validate to validate the installation. You must correct any errors indicate by --validate prior to continuing.

Kubernetes settings                                                                                           
===================                                                                                           
                                                                                                              
Name                               Value                                                                       
----                               -----                                                                      
extra-values-file                                                                                              ✔ 
ingress-type                       ingress                                                                     ✔ 
mode                               standard                                                                    ✔ 
namespace                          prod-test                                                                   ✔ 
registry                           docker-local.artifactory.swg-dev.com/apicup-imgs/2018.4.1-550               ✔ 
registry-secret                    apiconnect-image-pull-secret                                                ✔ 
storage-class                      rook-block                                                                  ✔ 
                                                                                                              
                                                                                                              
Subsystem settings                                                                                            
==================                                                                                            
                                                                                                              
Name                               Value                                                                       
----                               -----                                                                      
az-name                            default-az                                                                  ✔ 
cassandra-backup-auth-pass                                                                                     ✔ 
cassandra-backup-auth-user                                                                                     ✔ 
cassandra-backup-host                                                                                          ✔ 
cassandra-backup-path              /backups                                                                    ✔ 
cassandra-backup-port              22                                                                          ✔ 
cassandra-backup-protocol          sftp                                                                        ✔ 
cassandra-backup-schedule          0 0 * * *                                                                   ✔ 
cassandra-cluster-size             3                                                                           ✔ 
cassandra-max-memory-gb            9                                                                           ✔ 
cassandra-volume-size-gb           50                                                                          ✔ 
create-crd                         true                                                                        ✔ 
external-cassandra-host                                                                                        ✔ 
                                                                                                              
                                                                                                              
Endpoints                                                                                                     
=========                                                                                                     
                                                                                                              
Name                               Value                                                                       
----                               -----                                                                      
api-manager-ui                     apim.test.company.com                                                  ✔ 
cloud-admin-ui                     cm.test.company.com                                                    ✔ 
consumer-api                       consumer.test.company.com                                              ✔ 
platform-api                       api.test.company.com                                                   ✔

What to do next

Continue your API Connect installation in a Kubernetes runtime environment by installing the other subsystems.