The future of the fan experience at the US Open

The 2019 US Open, the fourth and final Grand Slam of the year, is set to begin August 26 for 14 days of championship tennis, featuring some 256 of the best players on the planet and a record $57 million in prize money to be played for. While a major tennis tournament is not your everyday event, to produce the best product possible, the United States Tennis Association (USTA) faces the same types of challenges as any company in just about any industry: How to deliver unique customer experiences, drive insights from multiple data sources, refine the efficiency of their operations, and much more – all in the digital age.

So with the help of IBM, the US Open is transforming its technology operations to create the future of championship sporting events today. They’re scaling their use of AI to deliver next-generation fan experiences, migrating mission-critical applications to an open, hybrid cloud, and protecting their brand with world-class security. The result is a US Open fueled by data, guided by insight, and built for change.

Redefining the fan experience with AI

Central to any sporting event is the fan experience. At the US Open it’s digital and it’s driven by data – an experience created from the scores and sights and sounds that are the raw materials of how fans follow the action, both in person and across the world. The Open, like any big sporting event, produces massive amounts of that data, and it comes from many sources. Some are more obvious, like the chair umpire, who uses a device called a chum to record the scores, or the courtside statisticians who record match events like unforced errors, aces, and winners, or the radar guns that send speed readings to the US Open control room.

But in the age of AI, the matches are also producing unstructured data – the visual and audio action of the US Open captured through the livestreams and devices. Some 370 hours of video footage will be produced over the course of the tournament – featuring over 31,000 data points, based on the 2018 scoring data. So the question is, what do you do with all this data? How do you make sense of it? How do you transform it into a meaningful experience for fans? The answer is, with AI.

IBM Watson, our enterprise-grade AI, is helping the USTA redefine how fans consume the action and experience every detail of the tournament. Powered by Watson’s conversational AI capabilities, the new “Ask the US Open” chat interface in the US Open app serves as the tournament’s virtual concierge, guiding patrons through the entirety of the fan experience. Users can interact with the chatbot in natural language, and it provides “know before you go” tips, stadium points of interest with a map integration functionality, live scores and results, match schedules, weather alerts, and more.

For the millions of viewers catching the action around the world, the 2019 US Open will feature an even more robust AI Highlights feature, which captures the most exciting moments from the tournament to compile a highlight reel in near-real time. Watson was trained to digest highlight clips and rank the excitement level or highlight-worthiness of each clip. To start, we put IBM Watson Media to work to automatically identify the start and end of each point, based on the camera angle and the sound of the tennis ball being struck. Then Watson’s visual recognition capabilities identify the celebratory gestures of a player – a fist pump or an arm raised overhead in celebration. Watson can even detect emotion in a player’s face, and it assigns each reaction a score. Next, with the latest in machine learning breakthroughs, Watson analyzes the crowd roar – again, giving each clip a crowd reaction score. Finally, the context of the match – whether it’s a break point or set point and so on – is factored into the overall excitement level.

In the AI Highlights Dashboard, also new for the 2019 US Open, Watson automatically curates highlights of all men’s and women’s singles matches on the key courts, based on the excitement level of each point. The IBM team enriches the video content with meta-data associated with the players, the score, and other factors, and offers an extended playlists of shots to be included in the highlight package for review by the USTA editorial team. All of this aggregating typically takes less than five minutes to produce, rather than the hours required to create manually.

With AI, the US Open can scale the management of its editorial content exponentially – but with that scale comes challenges with ensuring trust and transparency in the AI systems. To manage that, the IBM team used IBM Watson OpenScale, which opens up the black box of AI and allows users to manage production AI, with trust and confidence in outcomes and bias mitigation in build and runtime. In the context of the US Open, that means ensuring the AI system treats all highlights the same, regardless crowd size or court location. You can imagine how important this kind of validation would be in countless other contexts – from a bank using AI to help qualify loan applicants, to an insurance company using AI to help make policy determinations.

In this year’s US Open, even coaches and players can benefit from AI. Generally, when it comes to analyzing a player’s physiology, mechanics, or fatigue, coaches have relied heavily on experience and instinct. Now IBM is applying analytics to measure the physical exertion of a player over the duration of a match. The AI solution recognizes every instance of player acceleration or deceleration, and gauges the intensity of every movement. It then accurately charts the amount of energy expended, the physiological toll on the body, and its impact on the player’s performance in a match. Match video is then indexed against these measurements, allowing the coach to quickly see the cumulative physical demands, changes in mechanical intensity, and moments of fatigue. The coach can use this information to make changes to game strategy, training, or conditioning, and the system can even alert a coach or trainer to the potential for injury.

Creating the future of the sports experience

As unique an event as is the US Open, the problems they’re solving for and the ways they’re putting the latest technologies to work is not all that different from what’s facing virtually every industry. With AI, analytics, cloud, and world-class industry experience, IBM is helping the US Open advance on its journey to digital transformation. Together, we’re ensuring that journey is built upon an infrastructure that is cloud-first and AI ready, so that it’s highly scalable and highly flexible. We’re helping build trusted, transparent, AI systems that change the way fans, players, and coaches use data. And we’re securing the entire operation to protect their brand and continue to build value.

Enjoy the tournament!

Explore more stories of Watson at work in the world