The CHRO's guide to AI in HR: Myths, mayhem and measurable impact

In HR at IBM, we’ve been on a digital transformation journey for nearly 10 years. We’ve learned a lot along the way and can tell you one thing unequivocally: The most impactful lessons weren’t about implementing the technology—although we can certainly tell you all about that.

It may sound counterintuitive, but we’ve learned even more critical lessons about behavior, culture and leadership. 

In 2016, we gave IBMers access to a product called Watson Assistant®, which allowed any IBMer to develop basic question-and-answer chatbots without a coding background. This was fantastic—except that by early 2017, we had an army of chatbots running around HR. We joke that it was like “a bad Sci-Fi movie.” We had more than 30 bots in HR alone. This experience left employees feeling confused and frustrated, never knowing which bot to go to for which HR process, and we knew we had to make a change.

To address that change, we consolidated all the bots into one easy-to-use interface called AskHR. Employees didn’t need to worry about which bot to go to anymore; everything was in one interface. AskHR became our digital front door into HR. The only problem? IBMers weren’t using it. Why would they, when they could still call a 1-800 number, send an email, or even walk right down the hall and ask an HR Partner for help?

Real behavior change sometimes requires bold action. In 2018, we took a significant step to change IBMers’ behavior. Overnight, we shut down the 1-800 number and email address for all 350,000 IBMers and removed HR Partner support for first-line managers globally. We knew intuitively this was what it would take to force our team to use the new digital front-end into HR. Ultimately, we were right, but the journey was a rocky one.

Before we went “big bang” and shut off the email, phone number and first-line manager HR Partner support, our Net Promoter Score (NPS) for HR was +19. Shortly thereafter it was down to -35. That’s a 54-point decline.

You can have the best technology, workflows and processes, but at some point, you might have to decide: Do we ride out the -35? Do we believe we’re doing the right thing for the company and for our employees? Or do we make a different decision? Not every company has the culture to ride out a storm like that.

For us, the drop was painful, but not unexpected. We were confident that despite it, we were on the right path. Eighteen months later, our NPS was beginning to recover. Today, it’s +74. 

Sometimes we hear pushback when we tell our HR transformation story because people are fearful of AI hallucinations or bias. Our answer to that is: “Humans are always in control.” You’re leaders. You decide when, where and how to deploy AI. It’s a tool, and it’s not going to run wild. However, you do need to be practical in its application.  

Don’t let FOMO—the fear of missing out—drive your AI decisions. What does that mean? Don’t make decisions based on what the company down the street is doing because it sounds exciting, or because it’s getting a lot of press or social media attention. If it’s not aligned to your company’s values or the way your business makes money, don’t do it.

While that sounds simple, there’s a lot more to it. You need to be asking questions up front such as, “What are you comfortable with AI handling?” or “What aligns with the values of your company?”

At IBM, any use of AI will always include a human in the loop. Humans are the final decision makers—that’s one of our core values. For example, we use AI to recommend salary increases based on an employee’s skills, market scarcity and number of years in the job role. But how much to give is always the manager’s final decision. We’re comfortable with using AI for compensation recommendations to managers. Your company may not be.

However, we don’t use AI to screen candidates. As a skills-first company, we care about a candidate’s technical skills—not where they got them. We worry an algorithm might reject candidates from non-traditional backgrounds. Your company may want to do the complete opposite with AI, and that’s OK. It’s all about what’s right for your business.

It’s important that no matter where or how you use AI that you have a tool that’s watching, with governance built in. When the AI starts to deviate from the answers you’d expect to see, you must be alerted immediately. 

You need to be aware of how your AI is performing for your end users. Feedback mechanisms are critical. Starting out, a simple thumbs up or down may suffice, but users want to provide feedback via write-in comments. Ongoing feedback also allows you to spot trends in the data and update programs and policies faster.

With AskHR, for example, which has evolved into our AI agent, we receive thousands of comments a year on how we can enhance and improve the tool. When employees know that you’re hearing their pain points and addressing them, they buy in quickly. 

We’re often asked, “How do you decide what AI and automation projects to implement?” It’s difficult with so many choices, but we start out by asking ourselves some fundamental questions.

1. What problem are we trying to solve?
2. Are we really solving a problem?
3. Is it a problem worth solving?
4. What’s the return we’re getting?

Why these questions? There will never be enough budget, time or technical skills to implement every potential AI project. We must prioritize. Think about the use cases you have and consider those that will solve real problems, address employee pain points, and be big enough for scale and leverage in your organization.

And don’t be afraid to pilot. When we were considering a move from traditional to generative AI (gen AI) for our benefits content, some voiced concerns. There is no gray area in benefits—the answer is either right or wrong—so we ran a pilot.

The benefits team wanted the enhanced employee experience that gen AI could provide since it’s multi-turn and remembers the conversation; traditional AI has no memory. We ran a 90-day pilot within HR and realized that some questions are only appropriate for traditional AI, while gen AI excels at others. Variables like these make pilots crucial. Before you scale, ensure you’ve sufficiently tested with a big enough group aligned to your risk. 

If you haven’t begun your AI journey, there’s no time to waste. Find a small employee pain point, automate it and get started. You can scale from there... but begin today.

For those already underway, know that AI can unlock yet untold levels of employee experience, productivity and transformation. Be bold.