Enhanced Ingress Domain Functionality for Kubernetes Service, OpenShift and Satellite Clusters

You can use the new ibmcloud ks ingress domain commands to manage the domains and associated resources for your cluster. The command is grouped under the ingress namespace to enable better discoverability and to co-locate it alongside sibling commands in the Ingress feature family:

➜  ~ibmcloud ks ingress domain -h 
NAME: 
       ibmcloud ks ingress domain - [Beta] Manage a cluster's Ingress domains. 
USAGE: 
       ibmcloud ks ingress domain command [arguments...] [command options] 

COMMANDS: 
   create          [Beta] Create an Ingress domain for a cluster. 
   credential   [Beta] Manage a cluster's external domain provider credentials. 
   default        [Beta] Manage a cluster's default Ingress domain. 
   get              [Beta] View the details of an Ingress domain. 
   ls                [Beta] List all Ingress domains for a cluster. 
   rm                [Beta] Remove an Ingress domain from a cluster. 
   secret          [Beta] Manage the secrets for an Ingress domain. 
   update         [Beta] Update an Ingress domain for a cluster. The records passed in will fully replace the current records associated with the domain. Passing in no records will unregister the current records from a domain.

ibmcloud ks ingress domain create -h  
NAME:  
       create - [Beta] Create an Ingress domain for a cluster.  
 
USAGE:  
       ibmcloud ks ingress domain create --cluster CLUSTER [--crn CRN] [--domain DOMAIN] [--domain-provider PROVIDER] [--domain-zone ZONE] [--hostname HOSTNAME] [--ip IP] [--is-default] [--output OUTPUT] [-q] [--secret-namespace NAMESPACE]  
     
PARAMETERS:  
   --cluster value, -c value   Specify the cluster name or ID.  
   --domain value              The Ingress domain. To see existing domains, run 'ibmcloud ks ingress domain ls'.  
   --domain-provider           The external DNS provider type. The default is 'akamai'. Available options: akamai, akamai-ext, cis-ext, cloudflare-ext  
   --ip value                  The IP addresses to register for the domain.  
   --is-default                Include this option to set the relevant domain as the default domain for cluster.  
   --crn value                 The CRN for the IBM CIS instance.  
   --domain-zone value         The ZoneID for CIS.  
   --hostname value            For VPC clusters. The hostname to register for the domain.  
   --secret-namespace value  The namespace that the TLS secret is created in.  
   --output                    Prints the command output in the provided format. Available options: json  
   -q                          Do not show the message of the day or update reminders.

We have standardized the command operations on a CRUD model and created a cluster-infrastructure-agnostic command structure in order to provide a more consistent and understandable user experience.

The CLI command ibmcloud ks ingress domain create now supports custom domains, IBM Cloud Internet Services domains and third-party provider domains from Akamai and Cloudflare. If you do not specify a provider on the create domain command, the domain will be managed by IBM using the default domain provider:

➜  ~ibmcloud ks ingress domain create -h  
NAME:  
       create - [Beta] Create an Ingress domain for a cluster.  
 
USAGE:  
       ibmcloud ks ingress domain create --cluster CLUSTER [--crn CRN] [--domain DOMAIN] [--domain-provider PROVIDER] [--domain-zone ZONE] [--hostname HOSTNAME] [--ip IP] [--is-default] [--output OUTPUT] [-q] [--secret-namespace NAMESPACE]  
     
PARAMETERS:  
   --cluster value, -c value   Specify the cluster name or ID.  
   --domain value              The Ingress domain. To see existing domains, run 'ibmcloud ks ingress domain ls'.  
   --domain-provider           The external DNS provider type. The default is 'akamai'. Available options: akamai, akamai-ext, cis-ext, cloudflare-ext  
   --ip value                  The IP addresses to register for the domain.  
   --is-default                Include this option to set the relevant domain as the default domain for cluster.  
   --crn value                 The CRN for the IBM CIS instance.  
   --domain-zone value         The ZoneID for CIS.  
   --hostname value            For VPC clusters. The hostname to register for the domain.  
   --secret-namespace value  The namespace that the TLS secret is created in.  
   --output                    Prints the command output in the provided format. Available options: json  
   -q                          Do not show the message of the day or update reminders.

The ibmcloud ks ingress domain get and ibmcloud ks ingress domain ls CLI commands have been updated to display more relevant data in the table output and condense the content to improve domain detail visibility:

➜  ~ibmcloud ks ingress domain get -h 
NAME: 
       get - [Beta] View the details of an Ingress domain. 

USAGE: 
       ibmcloud ks ingress domain get --cluster CLUSTER --domain DOMAIN [--output OUTPUT] [-q] 
    
PARAMETERS: 
   --cluster value, -c value  Specify the cluster name or ID. 
   --domain value              The Ingress domain. To see existing domains, run 'ibmcloud ks ingress domain ls'. 
   --output                    Prints the command output in the provided format. Available options: json

➜  ~ibmcloud ks ingress domain ls -h 
NAME: 
       ls - [Beta] List all Ingress domains for a cluster. 

USAGE: 
       ibmcloud ks ingress domain ls --cluster CLUSTER [--output OUTPUT] [-q] 
    
PARAMETERS: 
   --cluster value, -c value  Specify the cluster name or ID. 
   --output                    Prints the command output in the provided format. Available options: json

The CLI Command ibmcloud ks ingress domain update  follows a PUT model to align more closely with the backend operations and reduce ambiguity in record updates:

➜  ~ ibmcloud ks ingress domain update -h 
NAME: 
       update - [Beta] Update an Ingress domain for a cluster. The records passed in will fully replace the current records associated with the domain. Passing in no records will unregister the current records from a domain. 

USAGE: 
       ibmcloud ks ingress domain update --cluster CLUSTER --domain DOMAIN [--hostname HOSTNAME] [--ip IP] [-q] 
    
PARAMETERS: 
   --cluster value, -c value  Specify the cluster name or ID. 
   --domain value                   The Ingress domain. To see existing domains, run 'ibmcloud ks ingress domain ls'. 
   --ip value                         The IP addresses to register for the domain. 
   --hostname value                For VPC clusters. The hostname to register for the domain. 
   -q                                     Do not show the message of the day or update reminders.

The CLI command ibmcloud ks ingress domain rm  supports deleting a domain and all associated resources from your cluster:

➜  ~ ibmcloud ks ingress domain rm -h 
NAME: 
       rm - [Beta] Remove an Ingress domain from a cluster. 

USAGE: 
       ibmcloud ks ingress domain rm --cluster CLUSTER --domain DOMAIN [-f] [-q] 
    
PARAMETERS: 
   --cluster value, -c value  Specify the cluster name or ID. 
   --domain value                  The Ingress domain. To see existing domains, run 'ibmcloud ks ingress domain ls'. 
   -f                          Force the command to run without user prompts. 
   -q                          Do not show the message of the day or update reminders.

You can use the ibmcloud ks ingress domain create functionality to create and register a custom domain, IBM Cloud Internet Services domain or a third-party DNS provider domain with any load balancer service in your cluster. We will fully manage the DNS registration and certificate lifecycle of this new domain on your behalf in the same way the existing domains are currently managed.

Previously, all domains managed by IBM Cloud Kubernetes Service were created with the format 

<cluster_name>-<account_hash>-<counter>.<region>.<dns_zone> 

(e.g.

long-classic-k8s-1-23-1-1e7743ca80a399c9cff4eaf617434c72-0000.us-east.containers.appdomain.cloud 

),including the default domain for the cluster. The new Ingress domain functionality supports creating a managed domain with a custom subdomain.

To create a custom domain, specify the desired subdomain using the --domain flag on the create command. Note that the DNS zone for the custom domains are still managed by IBM, so a provided custom subdomain of test-custom-domain  will result in a full domain

test-custom-domain.us-south.containers.appdomain.cloud

.The custom domains are validated for uniqueness to ensure there are no noisy neighbor conflicts:

➜  ~ibmcloud ks ingress domain create -c cgl90um10k5cc5n2msfg --domain test-custom-domain --ip 169.60.40.106

Creating test-custom-domain.us-south.containers.appdomain.cloud

The enhanced Ingress domain functionality supports the ability to create a domain for your cluster from an IBM Cloud Internet Services domain. This allows you to enable Web Application Firewalls, DDOS protection and global load balancing for your applications.

To create a domain from an existing IBM Cloud Internet Services domain, ensure that you have the appropriate service-to-service authorization policy in place. More details on creating this policy can be found here.

Once the service-to-service authorization is in place, you can use the ibmcloud ks ingress domain create command with the --domain-zone and --crn flags to create a domain from an IBM Cloud Internet Services domain. More details on the benefits of using IBM Cloud Internet Services and how to create an instance can be found here:

➜  ~ibmcloud ks ingress domain create -c test-cluster --domain jh-app1.com --domain-provider cis-ext --ip 169.60.40.106 --domain-zone 88ea2a737fbd5b149aa62c03d0adf343 --crn crn:v1:staging:public:internet-svcs:global:a/e3f386b3b6d14874a5437701b88371ca:f96ddbe5-6512-42ce-864e-d4dcabcc7057::
OK
ibmcloud ks ingress domain ls -c test-cluster
OK
Domain Target(s) Default Provider Secret Status Status
jh-app1.com 169.60.40.106 no cis-ext pending pending

You can now integrate an existing third-party Akamai or Cloudflare domain with your cluster for global load balancing support. To create a domain from a third-party provider, set the appropriate credentials for your cluster and use the domain create command with the --domain-provider flag. Note that you can only choose one active third-party provider for a cluster.

To begin, ensure that you have created credentials with the required permissions:

• Akamai: Read-write permissions for the /config-dns  endpoint
• Cloudflare: dns:read-write , zone:read-write , api-tokens:read

To set the credentials for your cluster, use the ibmcloud ks domain credential set  command for the appropriate third-party provider:

➜  ~ibmcloud ks ingress domain credential set -h 
NAME: 
       ibmcloud ks ingress domain credential set - [Beta] Add an external domain provider credential for the cluster. 
USAGE: 
       ibmcloud ks ingress domain credential set command [arguments...] [command options] 

COMMANDS: 
   akamai        [Beta] Set credentials for Akamai. 
   cloudflare   [Beta] Set credentials for Cloudflare.

➜  ~ ibmcloud ks ingress domain credential set akamai -h 
NAME: 
       akamai - [Beta] Set credentials for Akamai. 

USAGE: 
       ibmcloud ks ingress domain credential set akamai --cluster CLUSTER [--access-token TOKEN] [--client-secret SECRET] [--client-token TOKEN] [--domain-zone ZONE] [-f] [--host HOST] [-q] 
    
PARAMETERS: 
   --cluster value, -c value  Specify the cluster name or ID. 
   --host value                The host for the Akamai API Client Credentials. 
   --client-token value        The client_token for the Akamai API Client Credentials. 
   --client-secret value       The client_secret for the Akamai API Client Credentials. 
   --access-token value      The access_token for the Akamai API Client Credentials. 
   --domain-zone value      The zone to operate in. 
   -f                          Force the command to run without user prompts.

➜  ~ ibmcloud ks ingress domain credential set cloudflare -h 
NAME: 
       cloudflare - [Beta] Set credentials for Cloudflare. 

USAGE: 
       ibmcloud ks ingress domain credential set cloudflare --cluster CLUSTER [--domain-zone ZONE] [-f] [-q] [--token TOKEN] 
    
PARAMETERS: 
   --cluster value, -c value  Specify the cluster name or ID. 
   --token value               The API token. 
   --domain-zone value      The zone to operate in. 
   -f                          Force the command to run without user prompts.

You can use the additional ibmcloud ks ingress domain credential  commands to manage the lifecycle of your credential. You can remove the credential from your cluster at any point by using the ibmcloud ks ingress domain credential rm  command. If there are active domains for the provider still associated with your cluster, those domains will no longer receive record updates and will be marked with an error code in the Ingress status report. You can rotate the credential by re-running the ibmcloud ks ingress domain credential set command and specifying a new credential.

The ibmcloud ks ingress domain credential get command will supply credential metadata to help you keep track of which credential is in use for your cluster. Please note that once the credential is set, there is no way to view the actual credential:

➜  ~iks ingress domain credential get -c cgmog4k10hlptpsevhk0
OK
Credential:     12345
Provider:       akamai-ext
Expires At:     2024-04-26T17:03:58.000Z
Last Updated:   11 hours ago

Once you have set the third-party provider credential for your cluster you can use the --domain-provider flag on the ibmcloud ks ingress domain create command to create a domain for that provider. You can choose to create a brand-new domain based on the existing DNS zone or use a pre-existing domain for global load balancing (GLB).

To create a new custom domain based on an existing DNS zone in your third-party domain provider, supply the fully qualified domain with the --domain  flag on the create command. For example, if you have a DNS zone in your provider testing-custom.com and you want to create a new domain for your cluster new-custom.testing-custom.com , you would include --domain new-custom.testing-custom.com on the create command.

To use an existing third-party domain with your cluster, create a cluster-associated domain with the ibmcloud ks ingress domain create command and provide the existing domain. The IPs will be appended to the existing registration, which allows multiple clusters to use the same domain:

➜  ~ibmcloud ks ingress domain create -c test-cluster --domain new-prefix.fvtcustom.com --domain-provider cloudflare-ext --ip 169.60.40.106
OK
ibmcloud ks ingress domain ls -c test-cluster
OK
Domain Target(s) Default Provider Secret Status Status
new-prefix.fvtcustom.com 169.60.40.106 no cloudflare-ext pending OK

A cluster’s default domain is the domain reserved for registering the ALBs or OpenShift Ingress Controllers that come by default with your cluster. In Red Hat OpenShift on IBM Cloud clusters, this domain is the domain that exposes the OpenShift console (as well as the other default routes in the cluster).

The current default domain can be found in the Ingress Subdomain section of your cluster details or by listing the domains for your cluster using the ibmcloud ks ingress domain ls  command:

➜  ~ibmcloud ks cluster get -c cgmrhv620eqpknudf6rg
Retrieving cluster cgmrhv620eqpknudf6rg...
OK
Name: pvg-vpc-gen2-atpclujpb41ex83adi
ID: cgmrhv620eqpknudf6rg
State: normal
Status: All Workers Normal
Created: 2023-04-05 14:07:54 -0400 (7 hours ago)
Resource Group ID: 164fc63e5b694d4ca62ae09a8cae87de
Resource Group Name: Default
Pod Subnet: 172.23.64.0/18
Service Subnet: 172.21.0.0/16
Workers: 2
Worker Zones: us-south-3
Ingress Subdomain: cf-ext-replace563976.fvtcustom.com

➜  ~ibmcloud ks ingress domain ls -c cgmrhv620eqpknudf6rg
OK
Domain Target(s) Default Provider Secret Status Status
ak-custom211905.us-south.stg.containers.appdomain.cloud - no akamai created OK
cf-ext-replace563976.fvtcustom.com 58207625-us-south.lb.test.appdomain.cloud yes cloudflare-ext created OK

You can update the default domain for you cluster by using the ibmcloud ks ingress domain default replace command or by specifying the --is-default flag on the ibmcloud ks ingress domain create command. To set a custom domain as the default domain for your cluster during cluster creation, use the ibmcloud ks ingress domain create command immediately following the cluster create command with the new cluster ID.

For more information, check out our official documentation.

Learn more about IBM Cloud Kubernetes Service and Red Hat OpenShift on IBM Cloud.

If you have questions, engage our team via Slack by registering here and join the discussion in the #general channel on our public IBM Cloud Kubernetes Service Slack.