For many Americans no weekend is complete without a trip to the home improvement store. The home improvement market is a substantial one with spending reaching $275B in 2011, which is 1.8% of the US GDP. As the US economy has recovered from the Great Recession those numbers have continued to tick upward. For home improvement stores peak selling season spans the late spring and early summer months when Americans are ready to tackle big DIY projects in their homes and gardens. That said, every quarter’s performance is important to publicly traded companies and sales during the Black Friday – Cyber Monday bonanza are arguably the most closely scrutinized for all retailers.
Home Improvement Company (HIC) operates more than 1,500 stores in the US. Like other retailers they’ve invested heavily in creating an online marketplace and IBM’s commerce solution and network backbone help deliver what customers experience. The Accelerated Value Program (AVP) has helped HIC manage their system for several years, but due to HIC’s restrictive travel policies almost all the work had to be done remotely. While IBM is a valued partner, this limitation frustrated both teams. Nevertheless it was business as usual in preparing for Black Friday 2013 and the HIC team felt prepared.
From a customer buying perspective it was a smashing weekend. Shoppers perused the deals, made their selections, and keyed in their credit card numbers. While watchdog sites reported HIC’s transaction speeds were slow, from a public perspective the site was never down and orders kept streaming in. Success!?
Behind the scenes, things were not so rosy for HIC. While the public facing online marketplace never appeared to be down, a cascading series of application failures created an overselling situation. For more than 12 hours happy customers were placing orders for items HIC didn’t have in inventory at many stores, nor were these items committed via their supply chain. For IT, it was all hands on deck running cycles around the clock to triage the failure. AVP was engaged and escalated to many senior IBM resources to triage and repair the system problems. For HIC, this was followed by 4 weeks of manually back tracking on all open orders to ensure customers received their merchandise and/or received goodwill compensation.
Emerson pegs the average cost of a critical retail application failure at $673K/hr. making HIC’s raw cost of lost productivity $8.07M for this 12 hour span. IBM Digital Analytics benchmarks the performance of the US retail market and publishes an annual Black Friday report. Their analysis shows that peak online buying hours are from about 7:30AM-7:30PM PST. Using publicly available information from HIC’s website we estimate that HIC’s online revenue accrual rate was >$47M/hr. tallying sales of $4.2B for this important 4 day period. At these rates, even a single hour’s compromised revenue and productivity seriously hurts the bottom line. HIC was in real pain.
Committed to helping our client, IBM made some significant changes to the way we partner with HIC in 2014, assigning local AVP team members with WebSphere MessageQueuing (MQ) expertise. This enabled IBM to go onsite frequently and work with HIC’s staff and systems. More face time with their local IBM team helped strengthen the relationship so that AVP truly became a trusted advisor. AVP also engaged a distinguished engineer (DE) as part of the extended team to provide guidance on HIC’s architecture. With this IBM investment we were able to help HIC create high availability clusters to optimize system performance. IBM also executed end to end testing to identify and optimize communications between all system nodes.
In the run up to Black Friday 2014, the WebSphere Application Server (WAS) AVP team got involved to help HIC’s portal team to troubleshoot and resolve frequent native memory issues being encountered in the performance testing environment. AVP collaborated with IBM Level 2 support as they worked to expedite resolution of related Portal PMRs. AVP's involvement helped to quickly resolve the native memory issues that were blocking performance testing and provide an opportunity for skills transfer on debugging and understanding native memory issues.
For the big sales event AVP sent 2 resources onsite and had WAS and MQ experts on standby throughout the long weekend. The proactive work paid off and the system performed beautifully without any outages or application failures. That set the stage for HIC to have a very rewarding and profitable online holiday shopping season.
AVP covers a broad set of technologies at HIC. For more information about this customer and the work IBM Middleware does there, please contact firstname.lastname@example.org.
 The US Housing Stock: Ready for Renewal. Improving America’s Housing 2013; Joint Center for Housing Studies of Harvard
 Rackspace. Based on the 2013 Emerson survey of 67 U.S. Data Centers