Increasing Engagement Through Captions

Share this post:

Have you ever wondered what it would be like if only a tiny fraction of videos on the Internet had sound? How useful would they be, especially as video will account for 79 percent of all Internet traffic by 2018?

When you turn that scenario around and realize the infinitesimally small number of videos that are captioned, you start to understand what deaf and hard of hearing users have to endure on a daily basis.

This was the primary motivation behind my doctoral research – to understand the resource commitment and investigate Automatic Speech Recognition (ASR) and speech visualization techniques as possible low-cost, lower-resource solutions to the problem. The end goal is to vastly improve accessibility on the Internet for deaf users.

The lack of captioned videos blocks deaf users from vast knowledge and puts them in a disadvantageous position. The scarcity of captioned videos can, in part, be explained by the limited implementation and enforcement of legislative regulations coupled with the resources required for any captioning project.

A person holds an iPad and watches a captioned video of Brent Shiver with captions that say, "Hello, I am Brent Shiver, a software engineer with IBM Accessibility."

The overview and results of my research, “Evaluating Alternatives for Better Deaf Accessibility to Selected Web-Based Multimedia,” were presented last year at the International SIGACCESS Conference on Computers and Accessibility in Lisbon, Portugal.

The following research questions were addressed:

  • What aspects of using the Internet prove most difficult for deaf and hard of hearing users? For multimedia content, would transcripts be an acceptable alternative to captions? What is the current user experience with automatic captioning?
  • If hiring a skilled captionist or transcriptionist weren’t possible, would lower-quality text, generated by ASR, be better than nothing? Would a visualization of the text’s quality be useful?

The findings confirmed that captioning makes multi-media more accessible to people who are deaf or hard of hearing. The study revealed that even automatic captions on online videos may be beneficial to people who are deaf or hard of hearing, especially with online news. The research also concluded that additional studies are necessary in order to determine whether color-coding and other visualization techniques could provide additional advantage.

Captioning videos will not only benefit the deaf viewers, but there are additional advantages, including:

  • Lending support to viewers who are learning English as a second language; and, can also be translated to other foreign languages through automatic translation services extending audience reach.
  • Allowing the growing aging population with progressive hearing losses to enjoy videos that are captioned.
  • Offering help to people who are “situationally” disabled, such as those in a noisy cafeteria or airport.
  • Enhancing the understanding of content by employees challenged with cognitive disabilities and gradual hearing loss due to aging.
  • Exporting text to transcripts so employees who do not have time to watch the entire video can read them.
  • Enhancing Search Engine Optimization (SEO) rankings and improving discovery and marketability of certain videos.

Although there are multiple avenues to explore in researching how to make captioning widely available, one thing is certain – if you listen closely, you will hear a wide audience clamoring for captions.

Follow Brent on Twitter @BrentShiver

Add Comment
No Comments

Leave a Reply

Your email address will not be published.Required fields are marked *

More User Experience Stories
By Masashi Oikawa on July 19, 2017

IBM Japan’s “Access Blue” Program Trains Students with Disabilities for New Collar Jobs

In my country of Japan, there is a legal requirement (Act for Employment Promotion of Persons with Disabilities that states that at least 2 percent of employees across all business organizations (with workforces greater than 50) must be persons with disabilities. This requirement will increase to 2.2 percent in 2018 and 2.3 percent in 2020. […]

Continue reading

By Tim Powers on June 19, 2017

Creating Inclusive Workplaces is a Business Imperative

by Lou Orslene, Mobile Accommodation Solution (MAS) Project Manager and Co-Director of the Job Accommodation Network (JAN), Center for Disability Inclusion The World Health Organization reports globally there are more than 1 billion people with disabilities. Increasingly, chronic health conditions and disabilities are becoming more commonplace in the workplace. This has major implications for employers […]

Continue reading

By Tim Powers on April 25, 2017

Guiding the Way: On the Streets and in Life

by David Wei, Development Manager, Watson Career Coach With ear-splitting applause and screams, I cross the fabled finish line of 121th Boston Marathon, step by step with Erich Manser. I had the opportunity of running the Marathon by being a running guide for Erich, a blind Marathoner, an Ironman, an IBM colleague, and my friend. I […]

Continue reading