Bulkload

Edit online

You can use the bulkload utility to load data directly into the DB2® database from an LDIF file. The bulkload utility is a faster and better method to load large amount of data.

The bulkload utility parses the LDIF file, creates DB2 load files, and then loads the data directly into the DB2 database, bypassing IBM® Verify Directory.

The advantage to using the bulkload utility as compared to the ldif2db or ldapadd commands is the performance. The bulkload utility is many times faster than any other method of loading users. The bulkload utility might not be the right choice for every situation, such as to load small number of entries.

Consider the following points to determine whether to use the bulkload utility:

  • When you load data by using the bulkload utility, the directory server might not be available for other LDAP-related operations. In a replication environment, ensure that one or more replicas or peer servers are available when data is loaded to a server with the bulkload utility.
  • Updates that are made to a server by using the bulkload utility do not get automatically replicated in a replication environment. Each server in the replication environment must be updated by using the bulkload utility to synchronize all servers with the same data.
  • In a replication environment, you can back up a server to other servers before you load data to the server by using the bulkload utility. For the backup and restore, you must re-create the replication agreements on the restored servers. After the restore, all servers contain the same database.

You can run the bulkload utility with the appropriate parameters based on the data to load into a directory server. You must not specify certain parameters as they are deprecated. The defaults for the following parameters are set for optimal performance and must be considered before you specify them.

-A yes|no
Specifies whether to process the ACL information that is contained in the LDIF file. The default value is yes. If you specify no with the parameter, it loads the default ACLs.
-c | -C yes|no
Specifies to skip index recreation. You can skip index recreation between the loads and postpone the index creation until the last idsbulkload operation. Run the last idsbulkload with the -c yes parameter.
-e yes|no
Drops indexes before the load operation.

Effects of using the -k parameter

You can use the -k parameter with idsbulkload to load data in smaller chunks. You can use this parameter on systems where memory is limited. With the -k parameter, idsbulkload parses and loads data in smaller increments.

Note: If you specify small chunk size to parse and load, it might take considerable time to load data.
Figure 1. The relationship between data size and memory used

The relationship between data size and memory that is used when the -k parameter is specified.

The graph illustrates the effects on memory usage. As the chunk size increases, the memory utilization increases.

Figure 2. The relationship between data size and time to load data

The relationship between data size and time to load data when the -k parameter is specified.

The graph illustrates that as the chunk size increases, the load time decreases. The recommendation is to use chunk sizes of at least 1 million entries.