From 4.0.3-fp1 to 4.0.3-fp2
Summary
Optimization Server 4.0.3-fp2 introduces major change in
MongoDB version
PostgreSQL version
Development of applications
| Library | Backward compatibility | Deprecations |
|---|---|---|
| Master API client | Backward compatible | |
| Worker (Java) | Backward compatible | |
| Worker (Python) | Backward compatible |
Deployment of Optimization Server
| Chart | Backward compatibility | Deprecations |
|---|---|---|
| dbos-volume | Backward compatible | |
| dbos-secrets | Backward compatible | |
| dbos-infra | Break changes | |
| dbos | Backward compatible | |
| cplex | Backward compatible | |
| wod | Backward compatible |
Helm charts
Before upgrading your environment, please review the databases migration procedure.
dbos-infra chart
See this page for more details.
Below is the list of changes to configure the Postgres container:
the environment variable
POSTGRESQL_ADMIN_PASSWORDhas been replaced withPOSTGRES_PASSWORDthe volume for data has been updated to
/var/lib/postgresql/datathe execution command should now be specified on the container to take into account custom configuration:
command: [ "-c", "config_file=/var/lib/postgresql/postgresql.conf"]if a custom config file is provided, make sure
listen_addresses = '*'is in the file
The Keycloak URLs have been simplified, the trailing slash and /auth have been removed.
Below are the corresponding changes in the Helm files:
...
master:
...
environment:
- - KEYCLOAK_PUBLICAUTHSERVERURL=${KEYCLOAK_AUTH_SERVER_URL}/auth
+ - KEYCLOAK_PUBLICAUTHSERVERURL=${KEYCLOAK_AUTH_SERVER_URL}
...
web-console:
...
environment:
- - KEYCLOAK_URL=${KEYCLOAK_AUTH_SERVER_URL}/
+ - KEYCLOAK_URL=${KEYCLOAK_AUTH_SERVER_URL}Databases migration
MongoDB version has changed from 4.4.17 to 6.0.5. The easiest way to migrate existing data is to:
start a MongoDB 5.0 with the existing data and migrate using a query
start a MongoDB 6.0 with the existing data and migrate using a query
PostgreSQL version has changed from 12 to 15.2. The easier way to migrate existing data is to:
dump data with Postgres 12
use a fresh volume with Postgres 15.2
restore dump with Postgres 15.2
These migrations can be done in parallel.
Dump data from Postgres 12
In the Postgres 12 container the following command should be run to dump the databases.
# dump keycloak database
kubectl exec -t svc/postgres -- su - postgres -c "pg_dump -Fc keycloak" >./keycloak.pgdumpIf needed other databases can be dumped as well.
Then the generated files must be saved for later use.
Deploy the new chart
Undeploy the dbos and worker charts to avoid issues during the migration of the databases. Only the dbos-secret and dbos-infra charts must be deployed before starting the migration. You must use a new empty volume for Postgres and keep the existing one for MongoDB.
Then you can deploy the new dbos-infra chart (after migration when necessary) using these values in addition to your usual values:
rabbitmq:
enabled: false
keycloak:
enabled: false
mongodb:
image:
imageId: mongo:4.4Check there are only two pods running: postgres (with image tag 15.2) and mongo (with image tag 4.4).
Restore data into Postgres 15.2
Once Postgres 15.2 is up and running with a fresh volume the following commands will restore the database:
# restore keycloak database
kubectl exec -i svc/postgres -- su - postgres -c "pg_restore -d postgres --create --clean" <./keycloak.pgdumpIf needed other databases can be restored as well.
Migrate MongoDB data
To keep existing data, an upgrade procedure must be done. See this page for detailed information. Note that the current MongoDB version can be either in version 4.2 or 4.4 before this migration.
Validate the current database version
In the mongo container, the following command must be run:
mongo -u $MONGO_INITDB_ROOT_USERNAME -p $MONGO_INITDB_ROOT_PASSWORD
> db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )This command should print something like:
{ featureCompatibilityVersion: { version: 'X.Y' }, ok: 1 }Where X.Y is the current database version.
At this stage if the version is lower than 4.4, the following commands should be run in the terminal in the mongo container:
mongo -u $MONGO_INITDB_ROOT_USERNAME -p $MONGO_INITDB_ROOT_PASSWORD
> db.adminCommand( { setFeatureCompatibilityVersion: "4.4" } )Migrate to 5.0
Once the database is in version 4.4, the migration to 5.0 can be done. You must first, redeploy the dbos-infra chart with these values:
rabbitmq:
enabled: false
keycloak:
enabled: false
mongodb:
image:
imageId: mongo:5.0And then to migrate the database, the following commands should be run in the terminal in the mongo container:
mongo -u $MONGO_INITDB_ROOT_USERNAME -p $MONGO_INITDB_ROOT_PASSWORD
> db.adminCommand( { setFeatureCompatibilityVersion: "5.0" } )Migrate to 6.0
Finally, the operation must be reproduced to migrate in version '6.0'. You must first, redeploy the dbos-infra chart with these values:
rabbitmq:
enabled: false
keycloak:
enabled: false
# No values for mongodbThe provided MongoDB in version 6.0.5 should be started with the current data (in version 5.0), the following commands should be run in the terminal in the mongo container:
mongosh -u adminUser -p adminPassord
> db.adminCommand( { setFeatureCompatibilityVersion: "6.0" } )Final deployment
You can now deploy all charts with your usual values.