This blog promotes knowledge sharing through experience and collaboration. For more product information, visit our WebSphere Commerce CSE page. For easier navigation, utilize the Categories to find posts that match your interest.
Search 'n' Rescue: Search Term Associations
On review of search statistics you are capturing, whether it is through the out-of-box Search Statistics functionality or a third-party tool, you may notice a significant number of search misses happening. When none of the expected search results are returned, the user's ability to find the products they're looking for is significantly impacted, reducing your conversion ratio and negatively impacting the user experience. Search term associations can be used to reduce the scenarios in which no results are returned by either replacing bad search terms with better search terms, or adding additional search terms to find related products. To understand how to troubleshoot issues with search term associations, you should first understand the different types of search term associations available.
Types of Search Term Associations
Different types of search term associations should be used, depending on the behavior you are trying to achieve:
Now that you understand the different types of search term associations available, and have confirmed that you are using the right search term association for your scenario, you can collect the Search Runtime MustGather when testing our search scenario to better understand why your search term associations are not working as expected.
What do I look for in the captured tracing?
Once you have captured tracing of the search that should be using your search term associations, the first thing you should look for is Final Solr query to confirm that the expected search term associations are not being considered in the search query. For example, let's consider the example that you've created the synonym (coffee, tea) and see this Final Solr query in the tracing after searching for coffee:
[1/18/17 18:50:07:019 EST] 0000002b SolrRESTSearc 1 com.ibm.commerce.foundation.server.services.rest.search.processor.solr.SolrRESTSearchExpressionProcessor performSearch(SelectionCriteria) Final Solr query expression:...&q="tea" "coffee"...
We know in this case that the synonym is working as expected since we have tea being searched for as well. Similarly, you can also check the Final Solr query to verify if the replacement was performed. If you don't see this happening in the Final Solr query tracing, then you can work backwards from here to see how the search term associations are being processed. You can now look for SolrRESTSearchTermAssociationExpressionProvider invoke(SelectionCriteria) in the tracing captured to confirm that the expected search term is being used:
[1/18/17 18:50:06:889 EST] 0000002b SolrRESTSearc 1 com.ibm.commerce.foundation.server.services.rest.search.expression.solr.SolrRESTSearchTermAssociationExpressionProvider invoke(SelectionCriteria) Search term: coffee
If the wrong search term is being used here, then you need to confirm if your storefront is doing any processing of the search phrase being executing the search REST request as well as confirm if there are any query expression providers being used before SolrRESTSearchTermAssociationExpressionProvider that would be modifying the search terms before they are checked for possible search term associations. Next, you can then confirm if any search term associations for the specified term are found by looking for SolrRESTSearchTermAssociationExpressionProvider getSynonyms(String, String, Integer, String, String) in the tracing:
[1/18/17 18:50:06:893 EST] 0000002b SolrRESTSearc 1 com.ibm.commerce.foundation.server.services.rest.search.expression.solr.SolrRESTSearchTermAssociationExpressionProvider getSynonyms(String, String, Integer, String, String) Encoded synonyms of search term 'coffee' are [tea<SY@SP>coffee]
If you don't see the expected search term associations returned, then you will need to look into the search term associations created to make sure that one exists for specific search terms being looked up. You can see the search term associations in the SRCHTERM table, with the relationship (synonym, AlsoSearchFor replacement, InsteadSearchFor replacement, and landing page) defined in the SRCHTERMASSOC table.
You may follow this process, confirm that the expected data exists, but the search term association still isn't being applied as expected. The most common cause in this scenario is due to the search type being used.
Interaction Between Search Type and Search Term Associations
You should not see any problems with being able to apply your search term associations when the ANY (ORing the individual terms) search type is used. However, the ALL search type (ANDing the individual search terms) is problematic because of how the individual terms are split up in the query. If the ALL search type is used and you search for black roast coffee, you will get the following query generated:
[1/18/17 18:52:07:019 EST] 0000002b SolrRESTSearc 1 com.ibm.commerce.foundation.server.services.rest.search.processor.solr.SolrRESTSearchExpressionProcessor performSearch(SelectionCriteria) Final Solr query expression:...&q=(+("black") +("roast") +("coffee") (black roast coffee)) OR ("black roast coffee")...
The pluses are added to each individual term to make sure that there are matches on all the search terms. The problem with this, is that it causes problems for multi-term synonyms or replacements. For example, if you want to replace black roast with dark roast, it will not perform the replacement properly as black roast cannot match on the individually separated terms (black != black roast and roast != black roast).
If you want to make search more restrictive, but also want to use search term associations, then I would suggest that you use the ANY search type with minimum match. With minimum match, you can require that either a certain number of terms in the search phrase or percentage of the search phrase are matched on. For example, if you set minimum match to be 50%, and the user searches for dark roast coffee beans, then only results that match on at least 50% of the phrase will be returned (dark roast, coffee beans, dark coffee, dark beans, etc). This will allow you to make your search more restrictive while not being extremely restrictive like ALL search, which requires matches on every single search term. More information on using the ANY search type (with minimum match) for synonym expansion can be found on the Combining minimum match with search term associations Knowledge Center doc.
Thorough investigation into search misses and typical user searches, and then applying the appropriate search term association to resolve such issues will improve search result relevancy as well as improving your conversion ratio. If you have any questions or suggestions for troubleshooting search term association issues, feel free to post a comment below!