IBM at the Masters in 2017

Every April, the Masters golf tournament treats fans to a fierce competition set against a stunning backdrop of natural beauty. Walking the pine straw pathways at Augusta National Golf Club, you can’t help but feel the decades of history and tradition; an enduring game held at a timeless venue.

But there is a modernity to this tournament that few understand, and even fewer get to see. Hidden amidst the magnolias, dogwoods, and azaleas of Augusta National is a technological support system that fuels one of the most innovative tournaments in golf. In fact, the Masters lays claim to many of the technology firsts that have come to define the modern experience of golf. From the first television broadcast in 1956 to the first iPhone app in 2009, The Masters has steadily pushed the boundaries of golf technology in service of fans everywhere.

Cognitive Highlights Proof of Concept using Watson Technology

The Challenge

Capturing all the action at the Masters and sharing it with fans in a timely fashion is a huge video production effort. With 90 golfers playing multiple rounds over four days, video from every tee, every hole and multiple camera angles can quickly add up to thousands of hours of footage.

The Solution

IBM Research and IBM iX developed a Cognitive Highlights POC to auto-curate individual shot highlights from live video streams to help simplify and accelerate the video production process for highlight packages.

This first-of-a-kind system extracts exciting moments from live video streams of the Masters tournament based on multimodal cognitive computing techniques. Computer vision algorithms understand the content of the video, (detection of a player celebrating, the start of a golf swing). Audio analysis, speech to text and tone analysis detect crowd cheering and commentator excitement.

These markers are then fused to determine the start and end of highlight segments with a confidence score corresponding to the excitement level of the segment. By leveraging TV graphics and optical character recognition (OCR), the system automatically gathers information about the player name and hole number. This metadata is then associated with the detected highlight segments and posted to a dashboard. In the future this will enabling searches like “show me all highlights of player X during the tournament” or personalized highlights based on my favorite players.

The ability to quickly review and retrieve segments by player or hole can speed the process for creating and sharing highlights with fans eager to see the latest action. Such a system could also increase the scale and scope of videos produced to include highlights for every player or players grouped by country for greater fan personalization.

The learning from the POC here at the Masters will be applied not only to the 2018 Masters, but we are also evaluating automatically generating highlights for tennis matches where the early rounds can have up to 18 matches in progress simultaneously, generating thousands of hours of video footage.

Looking ahead, this solution can boost productivity for media and entertainment applications and the components of this solution can be customized to create efficient productivity tools for organizations across industries.

IBM powers the entire Masters digital experience

  • Video Everywhere:
    Live video that's resizable and moveable follows a user everywhere on the Masters site; offers closed captions support and channel change function; allows fans to watch video while reading an article

  • Leader Board Everywhere:
    Allows fans to access and follow the Leader Board at all times and stay up-to-date on scores will navigating the Masters site/app.

  • Track 3.0:
    Ability to view all players on a hole at any time from anywhere

  • iOS enhancements:
    Offers gestures for navigating back and forth between index and detail views; pinch and zoom on photo gallery images and on course map

  • Apple TV and 4K streaming

  • Watson Speech to Text for Video-on-Demand captioning and transcripts:
    Watson will assist Augusta with transcribing interviews and captioning VOD content, helping to deliver Masters content to fans more quickly.