Time information
You must determine the time that is required to load the data in a directory server from an LDIF file. You can use the load time to decide whether to load the file at one time or in smaller chunks.
The following graphs and tables show results of the bulkload utility when LDIF
files with 100,000 entries and 1 million entries (for small and large entries) are loaded.
The bulkload utility was run in two
passes, first the parse phase and then the load phase. No tuning was done on the database settings
before the bulkload utility was run. For the smaller entry, an index for the
seeAlso attribute was added.
- Time to bulkload LDIF files with 100,000 entries
Table 1. Time to bulkload 100,000 entries in 2 stages Time in seconds Small entries Large entries Parse time 35 127 Load time 47 128 Total time 82 255 - Time to bulkload LDIF files with 1,000,000 entries
Table 2. Time to bulkload 1,000,000 entries in 2 stages Time in minutes Small entries Large entries Parse time 5 25 Load time 8 112 Total time 13 137
By comparing the time to bulkload the same LDIF file on two different systems, the time variations, and difference in hardware type can be observed. The following graph and table show the results for two bulkload runs with an LDIF file of the 1,000,000 small entries. The two systems used were of the following specification:
- System 1
- AIX® 6.1 with 2 1499-MHz POWER 5™ processors and 7967 MB of memory (for one logical partition (LPAR)
- System 2
- AIX 7.1 with 2 3,612 MHz POWER 5 processors and 16,268 MB of memory
- Time to bulkload LDIF file with 1,000,000 small entries
Table 3. Time to bulkload in 2 stages for 1,000,000 small entries on 2 different systems Time (minutes) System 1 System 2 Parse time 5 5.12 Load time 8 5.41 Total time 13 10.53
On the AIX 6.1 system, the bulkload parse operation is run in same time as on AIX 7.1 system. On AIX 6.1 time to load the same amount of data took more time. The bulkload utility is a single threaded process, so improvements in time come from the processor speed, hard disk speed of the system, and the amount of memory used.
In general, the parse time increases linearly as the number of entries in the LDIF file increases. For example, the parse time for 100,000 small entries was 35 seconds and the parse time for 1,000,000 small entries was approximately 10 times. Load time does not increase linearly to the data set size.
During the parse phase, intermediate files are generated and these files are used in the load phase. If the input LDIF file is large, more data is written to the intermediate files. The increase in time is because of disk I/O and you can verify it based on the results that are obtained for the different LDIF files. The average entry parse time per second is greater for the larger entry size LDIF. For an LDIF File with 100,000 entries of small size, an average of 2800 entries were parsed per second. For the larger sized entries, only 780 entries were parsed per second.