First response made faster with video analytics

Share this post:

Gal Ashour

Editor’s note: This article was authored by Gal Ashour, manager of video and GIS analytics at IBM Research – Haifa.

Drones have been all over the news lately, with futuristic promises to do everything from chasing criminals to delivering packages. The reality is that drones, also known as unmanned aerial vehicles,are already commonly used in complex systems. This includes systems that rely on obtaining large amounts of video information in real time, such as those used for managing large-scale emergencies, as well as ones used for improving farming techniques and traffic control. 
My team at IBM Research – Haifa developed a framework for command and control (C2) applications that use mobile video feeds. We call it CARMEL, short for IBM Context-Aware Rich-Media Extensible middLeware.
These drones carry mobile sensors that can film large areas, providing good video coverage at a low cost. They come in all shapes and sizes, and have even made their way to the consumer market in the form of quad-copters. In disasters such as floods or fires, when road travel may not be an option, video from drones may be the only view we can get. Our aim is to help the people who need to interpret these types of videos.
Reducing information overload
As is often the case with new technology, these video sensors can create as many problems as they are attempting to solve. They provide such huge sets of data so quickly that sorting through them in time for practical use is difficult. And because the sensors are in motion as they photograph, figuring out when or where events took place can be nearly impossible.
CARMEL works by pairing video segments with the location information provided by metadata from the sensors’ geographic information system (GIS), annotating the videos with date and time information. Users can watch videos streamed in real time, understand what they are seeing, and then make decisions based on accurate, up-to-date information.  

The software uses IBM middleware such as Websphere, MQ, and Informix. It is not integrated directly with drones. But once the feeds from the drone are received at the ground-station network in industry standard format, CARMEL can detect these feeds (scanning the relevant IP/Port ranges) and ingest the data along with geographical metadata into the system. 

Collaboration capabilities
Today, emergency officials can get video feeds of the aftermath of a flood. Watching from a command center, away from the disaster, the videos come in faster than they can watch them, let alone analyze what they are looking at. Imagine that the officials see video of people stranded on their rooftops, yet they are not sure exactly where the houses are, or where rescue crew members are, or even if the roads between them are passable.
Using CARMEL, rescue teams in this situation can search for people who need help while simultaneously watching aerial video feeds and projecting their coordinates to their managers. The command center can then locate those in need and view their immediate surroundings to quickly and safely direct their rescue forces there.
CARMEL can also be configured to focus on a specific area, reducing information overload and potential distractions. The technology also provides instant-replay and collaboration capabilities while video continues to be captured. This allows CARMEL users to collaborate by sharing annotations on the viewed video in real-time In the case of disaster recovery, emergency managers can divide a region among rescue workers. These features work together to help str

eamline planning, evaluating, and acting — crucial in situations in which every minute counts.

More stories

A new supercomputing-powered weather model may ready us for Exascale

In the U.S. alone, extreme weather caused some 297 deaths and $53.5 billion in economic damage in 2016. Globally, natural disasters caused $175 billion in damage. It’s essential for governments, business and people to receive advance warning of wild weather in order to minimize its impact, yet today the information we get is limited. Current […]

Continue reading

DREAM Challenge results: Can machine learning help improve accuracy in breast cancer screening?

        Breast Cancer is the most common cancer in women. It is estimated that one out of eight women will be diagnosed with breast cancer in their lifetime. The good news is that 99 percent of women whose breast cancer was detected early (stage 1 or 0) survive beyond five years after […]

Continue reading

Computational Neuroscience

New Issue of the IBM Journal of Research and Development   Understanding the brain’s dynamics is of central importance to neuroscience. Our ability to observe, model, and infer from neuroscientific data the principles and mechanisms of brain dynamics determines our ability to understand the brain’s unusual cognitive and behavioral capabilities. Our guest editors, James Kozloski, […]

Continue reading