This blog promotes knowledge sharing through experience and collaboration. For more product information, visit our WebSphere Commerce CSE page. For easier navigation, utilize the Categories to find posts that match your interest.
Troubleshooting the Data Flow to the Storefront: Checking The Index
Recently I blogged an overview on how to begin troubleshooting issues related to the search data flow. As discussed in my earlier post the troubleshooting process consists of checking the index, the database and the runtime flow. In this post I will briefly discuss the first step of checking the index to help you ensure that the expected data exists within the solr index. It’s a simple step which requires querying the index and analyzing its response.
Querying the Index:
Knowing that commerce at runtime displays catalog information stored within the solr index, the first logical step would be to perform an initial check against the solr index to ensure it returns the data you are expecting. You can do this by executing a simple solr query directly against the solr index which would look like:
You can create more complex solr queries to query missing data specific to your scenario. Refer to the following article which describes in detail how a solr query is put together for further information: Breaking Down A Solr Query
In the case of a missing product, you would simply use the query as pasted above which will search the index for the relevant catentry_id. The query will depend on the missing data that you're trying to retrieve. If the issue is with a specific product, then querying the catentry_id or a partNumber will be ideal. However, if the issue for example is related to how many products are returned for a category, then it would make more sense to query the parentCatgroup_id_facet field (contains the associated category_ids for the catentry) for a specific category_id, and the catenttype_id_ntk_cs (type of catentry) with a value of 'productBean'. This will return a list of solr documents each representing a product associated with the problematic category.
Analyzing the Response:
The response returned from the sample solr query is the following:
Using this response you can verify that the data in question correctly exists as part of the index. In the case of a missing product, ensure that a solr document exists (which in the above case it does), the product is published (published search field) and that it is associated with the correct category (parentCatgroup_id_facet search field), catalog (catalog_id search field ), and store (storeent_id search field).
If the solr response does not return the correct response from the index, then we can be sure there is a data issue. This means that data is not part of the index because it was not built/indexed at the pre-process or build-index level of the data flow. I will discuss how to debug data related issues later in the series. At this point in time it would also be worthwhile to validate that the index was built successfully, ensuring you're not working with a stale index. Many a times I have seen issues where the client is working with an older index which is why their catalog changes are not reflected on the storefront. I would suggest reading through the following blog post: Validating The Search Index
On the other hand, if the solr response seems correct in that it returns the expected data, then you can conclude that the data does exist as part of the index. The issue could be that though the data exists, the storefront forms a final solr query which filters out the relevant solr documents. At this point it would make sense to start troubleshooting the scenario at runtime.
Alternatively, you can also use REST:
In FEP7+ Commerce uses REST requests to communicate with the Search Server which returns a response in JSON format. For debugging simple scenarios such as a missing shortDescription, it may be more convenient to use REST requests to analyze the data within the index as it is nicer to read. The following Knowledge Center link provides more information on the different Search REST API's available: WebSphere Commerce search interface
An example of a search REST request:
The JSON response is a processed form of what you find within the Solr index. Keep in mind that the search server refines the Solr response and adds quite a bit of information to the original xml returned. Also note that it is possible that the REST request may not return the most up to date information from the index as REST requests can also be cached. In that case, irrespective of the scenario you will need to directly query the index.
In my upcoming posts, I will talk about debugging the scenario at runtime and at the database level.