How do I Secure API Data Store Communication using HTTPS with ReadonlyREST Plugin?

Edit online

You can use the ReadonlyREST plugin to secure API Data Store (a simple Elasticsearch instance), one of the components in an API Management setup, to communicate securely over HTTPS.

Before you begin

Ensure that you have:

  • Downloaded the ReadonlyREST plugin. You can download the ReadonlyREST plugin compatible with Elasticsearch 8.2.3 from https://readonlyrest.com/download and store it in your file system.
  • Installed an updated version of Java in your system and the path of the environment variable is set.

To secure API Data Store communication using HTTPS

  1. Install and initialize the ReadonlyREST plugin.
    1. Shutdown API Gateway.
    2. Open the command prompt from the location SAG_Install_Dir/InternalDataStore/bin
    3. Run the following command: 
      elasticsearch-plugin.bat install file:///ReadonlyREST_plugin_zip_file_location_in_the_file_system
    4. Type y when the installation procedure prompts for additional required permissions.
      Installed readonlyrest message appears on successful installation.
    5. Navigate to the <SAGInstallDir>\InternalDataStore\config folder and create an empty file with the name, readonlyrest.yml (in the same folder where elasticsearch.yml is available).
    6. Copy the folder sagconfig from <SAGInstallDir>\IntegrationServer\instances\Instance_name\packages\WmAPIGateway\config\resources\elasticsearch to <SAGInstallDir>\InternalDataStore.
    7. Copy the certificates node-0-keystore.jks and truststore.jks from <SAGInstallDir>\InternalDataStore\sagconfig folder to <SAGInstallDir>\InternalDataStore\config folder. Because, the keystore should be stored in the same directory where elasticsearch.yml and readonlyrest.yml is available.
    8. Patch the Readonly REST plugin by running the command -
      java -jar <SAGInstallDir>/InternalDataStore/plugins/readonlyrest/ror-tools.jar patch --es-path <SAGInstallDir>/InternalDataStore
    9. Once the patching is successful, you can verify it by running the command - 
      java -jar <SAGInstallDir>/InternalDataStore/plugins/readonlyrest/ror-tools.jar verify --es-path <SAGInstallDir>/InternalDataStore
  2. Protect API DataStore with two-way SSL and basic authorization.

    The ReadonlyREST plugin supports plain-text credentials and hashed credentials for basic authorization. You can choose to add plain-text credentials or hashed credentials in the readonlyrest.yml file.

    Plain-text credentials

    Perform the following steps to include plain-text credentials for basic authorization in the readonlyrest.yml file.

    1. Open the readonlyrest.yml file from the SAG_Install_Dir\InternalDataStore\config folder and add the following text: 
      
readonlyrest:    
    access_control_rules:
 
    - name: "Require HTTP Basic Auth"
      type: allow
      auth_key: <plain_text_credentials> 
     (#For example: auth_key:Administrator:manage)
           
    ssl:
      enable: true
      keystore_file: "node-0-keystore.jks"
      keystore_pass: a362fbcce236eb098973
      key_pass: a362fbcce236eb098973
      client_authentication: true
      truststore_file: "truststore.jks"
      truststore_pass: 2c0820e69e7dd5356576

      where access control rule name: Require HTTP Basic Auth is basic auth authentication, auth_key is the credentials (username and password) in the plain text.

      Note: Ensure that the content is indented properly as shown above, so that the YAML parser can parse them correctly.

      Hashed credentials

      The ReadonlyREST plugin supports obfuscating the credentials by hashing the credentials using SHA256 algorithm. Use hashed credentials to keep the application secure.

      Perform the following steps to include hashed credentials for basic authorization in the readonlyrest.yml file.

      To generate the hash code for your password:
      1. Use the tool https://xorbin.com/tools/sha256-hash-calculator to hash your credentials. This generates the hash code.
      2. Replace the auth_key property in the readonlyrest.yml file with auth_key_sha256 property.
      3. Add the hashed credentials to the property auth_key_sha256 in readonlyrest.yml file located at SAG_Install_Dir\InternalDataStore\config. 
        
readonlyrest:    
    access_control_rules:
 
    - name: "Require HTTP Basic Auth"
      type: allow
      auth_key_sha256: 927d5619ff87227be6ca8a2cc9ee68c11dd7a08d64d1e20bdc8d86254850b418
           
    ssl:
      enable: true
      keystore_file: "node-0-keystore.jks"
      keystore_pass: a362fbcce236eb098973
      key_pass: a362fbcce236eb098973
      client_authentication: true 
      truststore_file: "truststore.jks"
      truststore_pass: 2c0820e69e7dd5356576

        where access control rule name: Require HTTP Basic Auth is basic auth authentication, auth_key_sha256 is the credentials (username and password) in the hashed format and client_authentication is the two way SSL authentication. By default, this property is disabled. You can remove this property or set the value to false if you do not want the server or API Gateway Data Store to validate the client authentication.

      4. Save the configuration file readonlyrest.yml.
      Note: The keystore file and its passwords keystore_pass & key_pass are shipped out with API Gateway product by default. This may not be safe for production environment. For production setup, you can generate your own certificates (keystore and trustore) and configure in the readonlyrest.yml file. To generate your own certificates, see ReadonlyREST.
    2. After adding plain-text credentials or hashed credentials in readonlyrest.yml file, open the elasticsearch.yml file from the SAG_Install_Dir\InternalDataStore\config folder and add the following configuration to the existing content and save the file. 
      
http.type: ssl_netty4
xpack.security.enabled: false

      This enables HTTPS connection for API Data Store.

  3. Secure the inter-node communication.

    In a clustered setup, to establish a secure communication between the API Gateway Data Store instances, perform the following steps:

    1. Open the readonlyrest.yml file from the SAG_Install_Dir\InternalDataStore\config folder and add the following configurations towards the end of the existing content and save the file. This configuration must be added to all the nodes which requires encrypted communication within cluster. 
      
  ssl_internode:
  keystore_file: "node-0-keystore.jks"
  keystore_pass: a362fbcce236eb098973
  key_pass: a362fbcce236eb098973
  client_authentication: true
      The consolidated content of the readonlyrest.yml is as follows: 
      
    readonlyrest:    
    access_control_rules:
 
    - name: "Require HTTP Basic Auth"
      type: allow
      auth_key_sha256: 927d5619ff87227be6ca8a2cc9ee68c11dd7a08d64d1e20bdc8d86254850b418
           
    ssl:
      enable: true
      keystore_file: "node-0-keystore.jks"
      keystore_pass: a362fbcce236eb098973
      key_pass: a362fbcce236eb098973
      client_authentication: true
      truststore_file: "truststore.jks"
      truststore_pass: 2c0820e69e7dd5356576

 
    ssl_internode:
      keystore_file: "node-0-keystore.jks"
      keystore_pass: a362fbcce236eb098973
      key_pass: a362fbcce236eb098973
      client_authentication: true
    2. Open the elasticsearch.yml file from the SAG_Install_Dir\InternalDataStore\config folder and add the following configuration to the existing content and save the file. 
      transport.type: ror_ssl_internode
      This configuration must be added to all the nodes which requires encrypted communication within cluster.
    3. Shutdown and restart the API Data Store node and verify if it is protected by accessing the node in HTTPS URL with the given username and password. 
      https://<host>:<port>

      API Data Store now runs on a secure channel on the HTTPS port and requests the basic authentication details.

  4. Change the Kibana configuration to connect to API Data Store.
    1. Open the kibana.yml file from the SAG_Install_Dir\profiles\IS_instance_name\apigateway\dashboard\config folder.
    2. Remove the # symbol from the following properties and provide the corresponding values for the properties and save the file.
      • elasticsearch.customHeaders.Authorization: "Basic QWRtaW5pc3RyYXRvcjptYW5hZ2U="
      • elasticsearch.ssl.verificationMode: certificate
      • elasticsearch.ssl.certificateAuthorities: <file path of your root-ca.pem certificate>
      • elasticsearch.ssl.keystore.path: "<SAGInstallDir>/InternalDataStore/config/node-0-keystore.p12"
      • elasticsearch.ssl.keystore.password: "password"
      • elasticsearch.ssl.truststore.path: "<SAGInstallDir>/InternalDataStore/config/truststore.p12"
      • elasticsearch.ssl.truststore.password: "password"
      • elasticsearch.ssl.alwaysPresentCertificate: true
      • elasticsearch.hosts: [https://<host>:<port>]

        Here, QWRtaW5pc3RyYXRvcjptYW5hZ2U= is Base64 encoded format of username: password - Administrator:manage. See https://forum.readonlyrest.com/t/kibana-error-401/1878 for more information.

        Sample kibana.yml file is as follows: 
        server.port: 9405
server.host: "0.0.0.0"
server.basePath: "/apigatewayui/dashboardproxy"
elasticsearch.hosts: ["http://localhost:9240"]
elasticsearch.customHeaders.Authorization: "Basic QWRtaW5pcRyYXRvcjptYW5hZ2U="
elasticsearch.ssl.verificationMode: certificate
elasticsearch.ssl.certificateAuthorities: <SAG_Root>/InternalDataStore/sagconfig/root-ca.pem
pid.file: kibana.pid
console.enabled: false
elasticsearch.ssl.keystore.path: "<SAGInstallDir>/InternalDataStore/config/node-0-keystore.p12"
elasticsearch.ssl.keystore.password: "password"
elasticsearch.ssl.truststore.path: "<SAGInstallRoot>/InternalDataStore/config/truststore.p12"
elasticsearch.ssl.truststore.password:"password"
elasticsearch.ssl.alwaysPresentCertificate:true
    3. Open the uiconfiguration.properties file from the SAG_Install_Dir\profiles\IS_instance_name\apigateway\config folder and set the property apigw.kibana.autostart to false.
    4. Genarate a PKCS12 file from jks files for both node-0-keystore.jks and truststore.jks files. To generate the PKCS12 files run the following command - 
      keytool -importkeystore -srckeystore [MY_KEYSTORE.jks] -destkeystore [MY_FILE.p12] -srcstoretype JKS -deststoretype PKCS12 -deststorepass [PASSWORD_PKCS12]

Example : 
keytool -importkeystore -srckeystore <SAG_Root>/InternalDataStore/config/node-0-keystore.jks -destkeystore node-0-keystore.p12 -srcstoretype JKS -deststoretype PKCS12 -deststorepass password

keytool -importkeystore -srckeystore <SAG_Root>/InternalDataStore/config/truststore.jks -destkeystore truststore.p12 -srcstoretype JKS -deststoretype PKCS12 -deststorepass password

    Once the files are generated, copy the files to <SAGInstallDir>/InternalDataStore/config.

  5. Change the API Gateway configurations to connect to API Data Store.
    1. Open the config.properties file from the SAG_Install_Dir\IntegrationServer\instances\instance_name\packages\WmAPIGateway\config\resources\elasticsearch folder.
    2. Remove the # symbol from the following properties and provide the corresponding values for the properties and save the file.
      • pg.gateway.elasticsearch.http.username=Administrator
      • pg.gateway.elasticsearch.http.password=manage
      • pg.gateway.elasticsearch.https.truststore.filepath=<SAGInstallDir>​/InternalDataStore/sagconfig/truststore.jks
      • pg.gateway.elasticsearch.https.truststore.password=2c0820e69e7dd5356576
      • pg.gateway.elasticsearch.https.enabled=true
      • pg.gateway.elasticsearch.https.keystore.filepath=<SAG_Root>/InternalDataStore/sagconfig/demouser-keystore.jks
      • pg.gateway.elasticsearch.https.keystore.password=6572b9b06156a0ff778c
    3. Start API Data Store.
    4. When API Data Store is up and running, start the Kibana server by running the kibana.bat file located at SAG_Install_Dir\profiles\IS_default\apigateway\dashboard\bin.
    5. Start API Gateway.

You can now log on, create APIs, and access the Analytics page without any challenge window for user credentials.

Configuring ReadonlyREST plugin with Self-generated certificates

As an API Provider, if you want to generate your own certificates to use with the ReadonlyREST plugin instead of the default certificates that are shipped with API Gateway, you can configure ReadonlyREST with user generated certificates. You have to perform this procedure if your organization does not have policies and procedures in place regarding the generation and use of digital certificates and certificate chains, including the use of certificates signed by a CA and you want to generate a self-signed certificate and import them into the keystore and truststore. Use Java Keytool to generate the required certificates for running ReadonlyREST in a production environment.

  1. Shut down API Gateway, API DataStore, and Kibana.
  2. Generate keystore.jks file.

    1. Run the following command from <Java_Install_Directory>/bin to generate keystore certificate in the corresponding file location using Java Keytool: 

      keytool -genkey -alias alias_name -keyalg RSA -keystore file_location\keystore.jks -storetype JKS
      The keystore certificate is generated in the file location that you specify.

      Example:

    2. Replace the generated keystore.jks file in the SAG_Install_Dir\InternalDataStore\config folder.
    3. Update the properties keystore_file, keystore_pass and key_pass in the readonlyrest.yml file located at SAG_Install_Dir\InternalDataStore\config.
      Sample readonlyrest.yml file is as follows: 
      
readonlyrest:    
    access_control_rules:
 
    - name: "Require HTTP Basic Auth"
      type: allow
      auth_key: Administrator:manage
           
    ssl:
      enable: true
      keystore_file: "keystore.jks"
      keystore_pass: <Keystore pass>
      key_pass: <Key pass>
      client_authentication: true 
      truststore_file: <<Truststore pass>>
      truststore_pass: <<Truststore pass>>
  3. Export certificate from the keystore to configure in Kibana.yml file:
    1. Run the following command to export the certificate from the keystore and place it in the required location. : 
      keytool -export -alias alias_name -keystore keystore_file_location -rfc -file <filename>.cert
      The exported certificate is saved in the specified location.
    2. After exporting, update the exported certificate name and the file location of the exported certificate in elasticsearch.ssl.certificateAuthorities property in kibana.yml file located at SAGInstallDir\profiles\IS_default\apigateway\dashboard\config.

      Example: 

      elasticsearch.ssl.certificateAuthorities: C:\work\apigateway\test.cert
  4. Generate truststore and import the generated certificate.
    1. Run the following command to generate the truststore file and import the generated certificate into the truststore file. 
      keytool -import -alias alias_name -file Certificate_file_location
 -storetype JKS -keystore file_location_and_file name_for_trustore.jks

      The trust store certificate is stored in the specified location. For example: C:\work\apigateway\trustore.jks.

    2. Update the details of the generated truststore file in the following properties
      • pg.gateway.elasticsearch.https.truststore.filepath
      • pg.gateway.elasticsearch.https.truststore.password in the config.properties file located at SAG_Install_Dir\IntegrationServer\instances\default\packages\WmAPIGateway\config\resources\elasticsearch.

      Example: 

      
pg.gateway.elasticsearch.https.truststore.filepath=C:/work/apigateway/trustore.jks
pg.gateway.elasticsearch.https.truststore.password=your_trustore_password
    3. Start API Data Store.
    4. When API Data Store is up and running, start the Kibana server by running the kibana.bat file located at SAG_Install_Dir\profiles\IS_default\apigateway\dashboard\bin.
    5. Start API Gateway.
You can now log on, create APIs, and access the Analytics page without any challenge window for user credentials.