This blog promotes knowledge sharing through experience and collaboration. For more product information, visit our WebSphere Commerce CSE page. For easier navigation, utilize the Categories to find posts that match your interest.
Validating The Search Index
At times it is worthwhile to go back and verify that the index is built with the most up to date information. It's really quite simple once you understand a few of the solr query commands available to you. I will walk you through steps on how you can validate the index and why it can be important.
I would not completely rely on the success message displayed within the console when running the di-buildindex.sh script as it tends to display false positives. You can directly query the solr index to ensure that it was built successfully. This can be done for each core using the status command:
You should see a success response like so:
You should see a failure response like so:
From the above responses take note of the 'Indexing Completed' message. The failure should be reflected through the status message (as shown in the second example). Another parameter to take note of is the 'Total Documents Processed' which for a full-build index should equal the total number documents within the solr index. You can gauge that something went wrong with the build if that number is oddly low.
Furthermore, it is also worthwhile to check the 'Committed' field even if the status message shows that the build completed. Many a times users find themselves working with a stale index which is the reason why their data cannot be found within the solr index. This field logs the date and time stamp of the last update to the index which can help ensure that you're working with a recently built index.
Another quick check that can be used to determine if you have a working index is to simply execute the following solr query:
This will return *all* the documents within the solr index. So you should see a response like so:
If you see '0' documents returned, then you can be sure that the index is empty. This maybe due to a buildindex failure, or because the TI_* temporary tables ( populated by the pre-process utility) are empty.
Re-building the index:
If you find that the index was not recently updated or failed, you can re-run the full-import or delta-import command via the commerce utility or directly against the search server using the following commands:
Note: Ensure that your preprocess related temporary tables have the correct information before running build-index. The temporary tables is where buildindex pulls its data from. For delta-indexing, be sure to specifically check the TI_DELTA_* tables. All the temporary tables are defined by the Knowledge Center: Temporary Table Schema Definition
The above URLs should build the index for the specified core. The commerce buildindex script essentially uses the SRCHCONF/SRCHCONFEXT tables to form the buildindex URL for each core and then executes them against the search server to build the index. There are times where the utility errors out on the Commerce side without having executed the buildindex command against the server. This usually means that there were issues creating the buildindex commands, but not with building the index itself. There are also times where the utility is able to build all solr cores except one, but due to the failure of one core it does not proceed to build the rest. This can impede progress.
When the utility fails, you can build the index manually using the URL above for each core. This will help to quarantine the problem to a specific core within build-index. If all cores build correctly and you have a working up to date index, then you can assume the issue is on the Commerce side where its unable to correctly form the full-build queries.
You can troubleshoot Commerce side buildindex issues further by enabling utility tracing as shown by the following Knowledge Center article: di-buildindex Utility
You can troubleshoot search side buildindex issues by enabling search side tracing. The following mustgather lists the trace strings: Mustgather: di-buildindex Utility
For some generic information regarding build index, check out the following FAQ: Commerce Search di-buildIndex Utility