The shoppers have loved the search bar on the shopping sites and come to expect them to be their first friend to take them close to the product they intend to shop on the site. This is akin to the quintessential ‘Shopping Assistants’ in the marts, supermarkets, our our big or small fashion outlets. The shop may be earmarked and categorized, however if we are not familiar that particular store, we tend to get awed by the array of aisles and have an instant liking towards the store assistant who can either guide you the aisle/corner of the store for your desired product or even better escort you to the product that you are seeking. You would agree that some of these assistants have converted how many bags you carried back home from your shopping expedition.
The current state of eCommerce has come a long way and with most search engines offering a set of standard technical capabilities, one would expect to find our shopping search experiences almost similar. However, we are actually at length from this experience. While I was looking for studies on the subject, I stumbled upon a 2014 report on smashingmagazine.com, where they benchmarked the search experience of the 50 top-grossing US e-commerce websites, rating each website across a set of 60 search usability parameters, which you would agree is rather very fine grained. Offcourse a study from 2014 would require a revisit in 2016 at the pace one would expect our eCommerce sites to launch capabilities in an agile way, to stay competitive.
What does this mean to you for as eCommerce Implementation Project Manager?
I have observed that even though search is an important functional feature on eCommerce sites, the search relevance testing is continued to be missed out of the plan. The traditional functional testing, includes search and browse feature test, where the test team is responsible for a set of use-cases for checking that the search functionality works as expected. However, this approach largely misses out on the desired shopper experience for the search results or/and the desired search results from the business point of view. The User Acceptance Testing (UAT) phase would typically have the business users look at search relevancy, however it remains more of an organic exercise, amongst the overall umbrella of UAT, not getting the due attention, time, effort and most importantly understanding of the approach, input and outcome.
What does this mean to you as the eCommerce Product Leadership for your business?
This is definite a very continual exercise for you, just as it is to deliver on the business requirements via your eCommerce and mCommerce channels. You will be doing analysis at the ‘search hits’ and ‘search misses’, hopefully on a weekly basis, and working towards tuning your results based on the data your shoppers are feeding into you. The strong point is that because you have invested into WebSphere Commerce, you will find that the Business Tools allow you with a lot of flexibility and ease to achieve your results.
Let us explore an approach towards search relevance, look at a proposal for the test strategy and guidelines for your WebSphere Commerce Search powered commerce site.
Outlining the search relevance testing proposal and approach
The Project Management of eCommerce sites must include a sprint (or more depending upon the catalog size) which focuses on the search relevance. I have had discussions with several test managers on the topic and found them wanting for guidance around the ‘search relevance testing’. This prompted me to create an approach document that can be used as starting guideline for your eCommerce site.
- Your list of search terms: Work with the business to identify a list of ‘n’ search terms which you would like to focus in your initial iterations. Where n could be 50, 100, 200 so on based on the width and depth of the catalog, product items.
- Variety in search terms: The search terms should include a variety, where you have single words, multi-words, words with units like weight (think grocery), phrases with attributes like color (think garments), variance in how shoppers search for certain products, brands which your catalog does not support, misspelled products …
- Get into the shopper’s shoes, sandals: Leverage analytics data from different sources to identify the changing trends and shoppers behavior. Go out there to your favorite, and not so favorite retailers with similar catalogs, and learn what you like and what you would wish did better.
- Product title/description recommendations: The quality of the search result highly depends upon the way the catalog data is created and maintained. The natural search results from the search engine is based on the query relevance as defined in the configuration. One of the important recommendation which would also come out of the relevance activity is on enriching the product descriptions and other searchable attributes.
- Tester training: Your army of testers may not be trained to think on the topic of search relevance. Plan for a training and get them to think and experience shopping around the domain. This may be simpler for certain industries like grocery and fashion, however this may slightly trickier incase your site is selling spare parts, or heavy industry tools.
- Search Developers: The search developer needs to be part of the iteration to support the relevance tuning. This will require to look at the relevancy scores, and providing analysis on the result. The developer will also be responsible for making any changes which come through search configuration changes.
- Inputs to the search relevance activity: At the start of the search activity, you will have the list of search terms, expected results, synonyms, and search replacement terms.
- Outputs of the search relevance activity: At the end of the search relevance activity, you will list of search terms, actual result-set, if it meets expected results and no what is the delta. You will also have a more expanded/modified list of synonyms, search replacement terms and search rules.
- Iterative: This exercise should be iterative based on how is your development life-cycle. If the catalog upload, products, brands are staggered towards different releases, plan the search relevance accordingly.
- Input to the performance cycle: The outcome from this exercise should also be fed into the performance system as the search performance should factor in the recommendations which will be applied on the production.
The search relevance is a very subjective criterion which varies by industry, business, catalog managers, merchandizers, the current promotions, and most of all your product catalog.
The search relevance activity needs to get under the skin of the shopper to be able to identify the patterns and results. The business needs to be engaged very closely in the process.
I would hope that the approach outlined in this article helps you to work towards the planning exercise and also gives a starting point. Do share your experience on search relevance, as I look forward to the different ways in which we are out to achieve the business outcome from our commerce sites – that is convert the searches into baskets and orders.
I am attaching a basic version of search template. If you think this is useful, do let me know so that I can share other versions on the same.
Previous related blog: Customizing search for shoppers and retaining them as their attention time dwindles
3 - Search Rules