MongoDB to PostgreSQL database migration fails because of the data issues in MongoDB

The connectivity issues with MongoDB, mongo document parsing issues, and data corruption issues can cause data migration from MongoDB to the PostgreSQL database to fail.

Symptoms

If the authentication.operator.ibm.com/retain-migration-artifacts annotation is set to false in the authentication custom resource, the migration from MongoDB to PostgreSQL database is failed. You can check the Identity Management (IM) operator logs to find the reason for migration failure.

Note: If the migration fails, the IM operator does not create platform-auth-service, platform-identity-management, and platform-idenity-provider operand pods.

Resolving the problem

To resolve the data migration issue, complete the following steps:

1. Get the IM operator logs. You can review the logs to find the document with the migration issues in MongoDB and fix it.

   oc logs -f deploy/ibm-iam-operator | grep 'migration_worker'
   

   The following is the sample IM operator log:

   {"level":"error","ts":"2024-04-02T05:50:18Z","logger":"migration_worker","msg":"Failed to INSERT into table","MongoDB.DB":"platform-db","MongoDB.Collection":"Groups","table":"platformdb.users_groups","error":"ERROR: null value in column \"user_uid\" of relation \"users_groups\" violates not-null constraint (SQLSTATE 23502)","stacktrace":"github.com/IBM/ibm-iam-operator/migration.insertGroupsAndMemberRefs\n\t/home/prow/go/src/github.com/IBM/ibm-iam-operator/migration/migration.go:1657\ngithub.com/IBM/ibm-iam-operator/migration.MongoToV1\n\t/home/prow/go/src/github.com/IBM/ibm-iam-operator/migration/migration.go:329\ngithub.com/IBM/ibm-iam-operator/migration.(*Migration).Run\n\t/home/prow/go/src/github.com/IBM/ibm-iam-operator/migration/migration.go:194\ngithub.com/IBM/ibm-iam-operator/migration.Migrate\n\t/home/prow/go/src/github.com/IBM/ibm-iam-operator/migration/migration.go:212"}
   
   {"level":"error","ts":"2024-04-02T05:50:18Z","logger":"migration_worker","msg":"1 of 1 members could not be migrated for group cn=afaas-hyc-cloud-private-read,ou=memberlist,ou=ibmgroups,o=ibm.com","MongoDB.DB":"platform-db","MongoDB.Collection":"Groups","error":"ERROR: null value in column \"user_uid\" of relation \"users_groups\" violates not-null constraint (SQLSTATE 23502)","stacktrace":"github.com/IBM/ibm-iam-operator/migration.insertGroupsAndMemberRefs\n\t/home/prow/go/src/github.com/IBM/ibm-iam-operator/migration/migration.go:1664\ngithub.com/IBM/ibm-iam-operator/migration.MongoToV1\n\t/home/prow/go/src/github.com/IBM/ibm-iam-operator/migration/migration.go:329\ngithub.com/IBM/ibm-iam-operator/migration.(*Migration).Run\n\t/home/prow/go/src/github.com/IBM/ibm-iam-operator/migration/migration.go:194\ngithub.com/IBM/ibm-iam-operator/migration.Migrate\n\t/home/prow/go/src/github.com/IBM/ibm-iam-operator/migration/migration.go:212"}
   

2. Connect to the MongoDB instance and fix the data format of the specific failed document. If the issue is still not resolved, you can manually set "migrated" to true for the specific failed document to complete the data migration. You can add a row manually in the EDB Postgresql database to migrate the failed document.

   oc exec -it icp-mongodb-0 -c icp-mongodb -- bash
   

   mongo --host rs0/mongodb:27017 --username $ADMIN_USER --password $ADMIN_PASSWORD --authenticationDatabase admin --ssl --sslCAFile /data/configdb/tls.crt --sslPEMKeyFile /work-dir/mongo.pem
   

   use platform-db
   db.Users.updateOne({"_id": "52ead134-3915-43d9-b823-331aac3e141c "}, {$set: {"migrated": true}})
   

Note: If the migration fails, the IM operator does not create platform-auth-service, platform-identity-management, and platform-idenity-provider operand pods. To troubleshoot the pods issue, see

1. If the failed document does not cause functional issues during data migration, you can set authentication.operator.ibm.com/migration-complete to true in the authentication custom resource manually to create the platform-auth-service, platform-identity-management, and platform-idenity-provider operand pods.

   oc edit authentication.operator.ibm.com example-authentication -n `<your-foundational-services-namespace>`
   

   Replace <your-foundational-services-namespace> with the namespace where you deployed the foundational services.