Architecture for Open Data for Industries
IBM® Open Data for Industries is installed on the IBM Cloud Pak for Data control plane on a Red Hat® OpenShift® Container Platform cluster.
Entitled software for Open Data for Industries
- IBM Cloud Pak for Data control plane (the
lite
assembly) - IBM Cloud Pak® foundational services (such as the License Service and the IAM Service.)
IBM Open Data for Industries also includes entitlement to several additional, separately licensed Red Hat products, including, but not limited to:
- Red Hat OpenShift Container Platform
- Red Hat OpenShift Container Storage
- Red Hat Enterprise Linux®
- Red Hat AMQ Broker
Secure by design
Because IBM Open Data for Industries is fully integrated with Cloud Pak for Data, it benefits from a set of security mechanisms that are implemented at the platform level. For more information, see platform's Security considerations.
Solution architecture for Open Data for Industries
The following diagram shows the various components that form the Open Data for Industries solution:
- IBM Cloud
- Amazon Web Services
- Microsoft Azure
- Google Cloud
- OpenStack
- VMware
Information about supported versions of Red Hat OpenShift are listed in System requirements for Open Data for Industries.
Required software
Your cluster must include the following software:
-
Red Hat OpenShift Container Platform
- Purpose
- A hybrid cloud, enterprise-grade Kubernetes platform for containerized applications.
- Provided by
- Red Hat
- Details
- For installation information, see the Red Hat OpenShift Container Platform
product documentation.
For information on supported versions, see the System requirements for Open Data for Industries.
- Licensing
- For license terms, see the IBM Open Data for Industries Software License Agreement and the Red Hat Enterprise Agreements site.
-
Container registry
- Purpose
- Used to store container images.
- Provided by
- Customer-provided
You can use either the internal registry in your Red Hat OpenShift Container Platform cluster or your existing container registry, provided that it is in close proximity to the cluster.
- Details
- See Setting up your registry server.
-
Red Hat
OpenShift Container Storage
- Purpose
- Shared persistent storage for Open Data for Industries.
- Provided by
- Red Hat
- Details
- For installation information, see the Red Hat
OpenShift Container Storage product documentation.
For information on supported versions of Red Hat OpenShift Container Storage, see the Storage requirements.
- Licensing
- For license terms, see the IBM Open Data for Industries Software License Agreement and the Red Hat Enterprise Agreements site.
-
Red Hat AMQ Broker
- Purpose
- A message broker that allows application components to exchange data.
- Provided by
- Red Hat
- Details
- When you run the script to install the software
utilities, the AMQ Broker operator is installed and an instance of
Red Hat AMQ Broker is deployed to the
osdu-amq
project. The script installs the version of Red Hat AMQ Broker specified in System requirements for Open Data for Industries. - Licensing
- For license terms, see the IBM Open Data for Industries Software License Agreement and the Red Hat Enterprise Agreements site.
-
Apache Airflow
- Purpose
- A workflow management platform that is used to manage data ingestion.
- Provided by
- Apache Airflow is open source.
- Details
- When you run the script to install the software
utilities, an instance of Apache Airflow is
deployed to the
osdu-airflow
project. The script installs the version of Apache Airflow specified in System requirements for Open Data for Industries. - Licensing
- Terms are governed by an Apache License.
-
Apache CouchDB
- Purpose
- A document-oriented, NoSQL database used to store metadata generated by Open Data for Industries.
- Provided by
- Apache CouchDB is open source, but the operator is provided by IBM.
- Details
- When you run the script to install the software
utilities, the Operator for Apache CouchDB operator is installed and
an instance of Apache CouchDB is deployed to the
osdu-couchdb
project. The script installs the version of Apache CouchDB specified in System requirements for Open Data for Industries. - Licensing
- Terms are governed by an Apache License.
-
Elasticsearch
- Purpose
- A search engine used to search the index.
- Provided by
- Elasticsearch is open source, but the operator is provided by Red Hat.
- Details
- When you run the script to install the software
utilities, the Elasticsearch Operator is installed and an instance of
Elasticsearch is deployed to the
osdu-elastic
project. The script installs the version of Elasticsearch specified in System requirements for Open Data for Industries. - Licensing
- Terms are governed by the Elastic License.
-
Keycloak
- Purpose
- Software for SSO with Identity and Access Management for user and group management.
- Provided by
- Keycloak is open source, but the operator is provided by Red Hat.
- Details
- When you run the script to install the software
utilities, the Keycloak Operator is installed and an instance of
Keycloak is deployed to the
osdu-keycloak
project. The script installs the version of Keycloak specified in System requirements for Open Data for Industries. - Licensing
- Terms are governed by an Apache License.
-
MinIO
- Purpose
- S3 compatible storage where raw data from Open Data for Industries is stored.
- Provided by
- MinIO is open source.
- Details
- When you run the script to install the software
utilities, an instance of MinIO is
deployed to the
osdu-minio
project. The script installs the version of MinIO specified in System requirements for Open Data for Industries. - Licensing
- Terms are governed by an Apache License.
-
Redis
- Purpose
- An in-memory storage used as database for fast access for authorization in Open Data for Industries.
- Provided by
- Redis is open source.
- Details
- When you run the script to install the software utilities, an instance of Redis is deployed to
the
osdu-redis
project. The script installs the version of Redis specified in System requirements for Open Data for Industries. - Licensing
- Redis is released under the three clause BSD license.
-
OpenShift
Service Mesh
- Purpose
- Used to provide observability, security and communication between Open Data for Industries APIs.
- Provided by
- Red Hat OpenShift
- Details
- Service Mesh is installed and configured in the post-installation phase.
- Licensing
- OpenShift Service Mesh is under Red Hat license agreements.
-
IBM Cloud Pak for Data control plane
- Purpose
- Provides a common integration layer for IBM Cloud Pak for
Data services.
The control plane provides an administration interfaceand route for accessing the platform.
For an overview of the control plane, see Architecture for IBM Cloud Pak for Data.
- Provided by
- IBM
- Details
- For installation information, see the Installing section of Getting started with Open Data for Industries.
- Licensing
- For license terms, see the IBM Open Data for Industries Software License Agreement.
-
IBM Open Data for Industries
- Purpose
- Provides the core API components that you can use to govern your Oil & Gas data. For a description of the components, see Open Data for Industries APIs.
- Provided by
- IBM
- Details
- For installation information, see the Installing section of Getting started with Open Data for Industries.
- Licensing
- For license terms, see the IBM Open Data for Industries Software License Agreement.
After you install the Open Data for Industries service, you can extend the service with custom extensions or domain data management services.
Optional software
You can optionally use the following software on your cluster:
- Cluster management software
-
-
Operator Lifecycle Manager
- Purpose
- Used to help install, update, and manage the lifecycle of operators and their associated services on the cluster.
- Provided by
- Red Hat
- Details
- Operator Lifecycle Manager is available by default when you install Red Hat OpenShift Container Platform. However, you are not required to use it. For details, see Operator Lifecycle Manager concepts in the Red Hat OpenShift Container Platform product documentation.
- Licensing
- Terms are governed by the Red Hat Enterprise Agreements
-
Operator Lifecycle Manager
- Monitoring and logging software
-
-
Prometheus
- Purpose
- Used to monitor the cluster.
- Provided by
- Red Hat
- Details
- Red Hat OpenShift Container Platform monitoring stack is based on the Prometheus open source project. By default, the stack monitors the core platform components. However, you can optionally monitor user-defined projects (namespaces). For details, see Understanding the monitoring stack in the Red Hat OpenShift Container Platform product documentation.
- Licensing
- Terms are governed by the Red Hat Enterprise Agreements
-
Grafana
- Purpose
- Used to analyze and visualize metrics.
- Provided by
- Red Hat
- Details
- A read-only instance of Grafana is included in the Red Hat OpenShift Container Platform monitoring stack. For details, see Default monitoring components in the Red Hat OpenShift Container Platform product documentation.
- Licensing
- Terms are governed by the Red Hat Enterprise Agreements
-
EFK
- Purpose
-
Used to collect, index, search, and visualize log data.
- Provided by
- EFK, which stands for Elasticsearch, Fluentd, and Kibana, is an open source logging stack for Red Hat OpenShift Container Platform.
- Details
- For details, see Installing cluster logging in the Red Hat OpenShift Container Platform product documentation.
- Licensing
- Terms are governed by the Red Hat Enterprise Agreements
-
Prometheus
- IBM Cloud Pak foundational services
-
-
IAM Service
- Purpose
- Used by the Cloud Pak for Data control plane. This service does not integrate with Open Data for Industries.
- Provided by
- IBM
- Details
- For details on the IAM Service, see Shared cluster components.
- Licensing
- For license terms, see IBM Open Data for Industries Software License Agreement.
-
License Service
- Purpose
- Used to measure your VCP use.
- Provided by
- IBM
- Details
- For details on the License Service, see Shared cluster components.
- Licensing
- For license terms, see IBM Open Data for Industries Software License Agreement.
-
IAM Service