On November 30, the Partnership for Public
Service (www.ourpublicservice.org) released
their new study, “From Data to Decisions: The Power of Analytics.” [i]
Keynoting the event was Shelley Metzenbaum, Associate Director for
Performance and Personnel Management, OMB.
She told the audience that Performance Management is a core pillar of
the Obama Administration and that Measurement and Analysis was the key tenet to
PM. She encouraged the audience to
identify analytics practices that work and spread the word to others. She exhorted the audience to not just collect
data but to use the data to pinpoint problems – “Ask Why, Why, Why” with
respect to performance problems.
The report studied 7 programs[ii] in 8 federal
agencies to understand how they use analytics and how it helped them achieve
better program results. The study
provides clear examples of how data is being used to understand problems and
improve mission performance. It
documents how CMS is using data to answer the question why isn’t health care
quality better and how can we direct scare resources to improve it? In a similar fashion, VA and HUD are using
data to figure out how to reduce homelessness of Veterans including identifying
bottlenecks that are keeping their voucher program from being more
successful. David Zlowe, Performance
Improvement Officer at VA, emphasized in the study that the power of VA’s
analytics approach isn’t in the numbers but in the discussion that are sparked….having
leadership engage in an appreciative conversation guided by hard data.” The 4th program that the
Partnership reported on in some detail is the FAA’s Safety Management
System. This program helps to identify
risks and to understand what contributes to all levels of hazards.
The Partnership event included a panel discussion with Michelle Snyder,
Deputy COO, CMS; Estelle Richman, COO
and Acting Deputy Secretary, HUD; and David Zlowe, PIO, VA. Ms.
Snyder’s advice to the audience was, “Take data, analyze it, tell the story to
the people so it relates and influences the decision makers.” Ms. Richman’s recommendation was to remember
that the analytics are but a method to accomplish the goal of creating an
outcome that can improve people’s lives.
And Mr. Zlowe summarized by saying, “We don’t lack data, we lack
We’d like to hear your experiences driving decisions based on data
in the government. If you'd like a copy of the report, write to me at: ASCdc@us.ibm.com
[i] The Study was a
collaboration between IBM’s Center for the Business of Government and the
Partnership for Public Service
[ii] HUD and VA Veternans
Affairs Supportive Housing (HUD-VASH) program; Safety Management System (SMS)
in the FAA; HHS CMS nursing homes and transplant programs; Coast Guards’
Business Intelligence system (CGBI); NHTSA “Click it or Ticket” campaign;
Navy’s Naval Aviation Enterprise; SSA’s use of mission analytics n customer
Modified on by fstein
Guest blogger: Zoya Yeprem, Senior Technical Solution Specialist Intern
IBM Summit Program, IBM Federal
Computer vision is a technology that acquires, processes and analyzes images, and can automate what human visual analysis can perform. And with recent advances in this field, we are witnessing its use in our personal and professional lives more than ever. Such systems are heavily dependent on highly complex deep learning models called Deep Neural Networks (DNN). These models are capable of successfully performing complex tasks like object detection, image classification, segmentation, etc. without being explicitly programmed.
Numerous applications and systems that we will be using in the future will be using deep learning models behind the scenes to perform high-level cognitive tasks e.g. self-driving cars. In addition to the day-to-day use of such systems, computer vision can be extremely useful for government agencies. It can help agencies categorize and analyze images and gain high level cognitive insights automatically in real time. For instance, the surveillance system in airports can be integrated with computer vision systems that can help detect any abnormal activity that may be considered a threat. Also, when the threat is detected, it can help track down the person responsible by identifying other sightings of similar looking individuals. Another use case is in geospatial technologies. Geospatial imagery encompasses a wide range of graphical products that convey information about natural phenomena and human activities occurring on Earth's surface. This technology uses computer vision to provide complex insights in real time, providing crucial information to humanitarian and disaster relief agencies where accuracy and timeliness are top priorities.
While deep learning models used in computer vision systems are normally very accurate, they are vulnerable to special attacks that use adversarial examples. Adversarial examples are input images that have a carefully crafted noise added to them. While these images appear identical to the originals, they are completely misclassified by the DNN. A simple example can be found below which demonstrates how a stop sign image is misclassified as an “Ahead Only” sign when certain noise is added to it.
Adversarial attacks pose a real threat to the deployment of AI systems in security critical applications. Virtually undetectable manipulations of images, video, speech, and other data have been crafted to confuse these systems. Such manipulations can be crafted even if the attacker doesn’t have exact knowledge of the architecture of the deep learning model or access to its parameters (Black-Box attacks). Even more worrisome, adversarial attacks can be launched in the physical world: researchers have proven that instead of manipulating the pixels of a digital image, adversaries could defeat visual recognition systems in autonomous vehicles by sticking patches to traffic signs, or they can fool facial recognition systems by wearing specially designed glasses. Therefore, it is crucial to protect our deep learning models against such attacks.
IBM Research in Ireland has released the Adversarial Robustness Toolbox (ART) that provides protection against adversarial attacks on DNNs. ART is an open source library written in python that supports most popular deep learning frameworks such as: TensorFlow, Keras, PyTorch, etc. ART can provide protection to a DNN in three stages:
First, we check to see if the DNN model is vulnerable against adversarial attacks as not all DNNs are vulnerable. ART has implementations of state-of-the-art attacks, which can be used to craft an adversarial image and feed it to the DNN. Then, by recording the loss of accuracy on adversarially altered inputs, you can detect how vulnerable your model is to that specific attack. Other approaches measure how much the internal representations and the output of a DNN vary when small changes are applied to its inputs.
Second, after confirming vulnerability of a certain type of attack, a given DNN can be “hardened” to make it more robust against adversarial inputs. Common approaches are to preprocess the inputs of a DNN, to augment the training data with adversarial examples, or to change the DNN architecture to prevent adversarial signals from propagating through the internal representation layers.
Finally, runtime detection methods can be applied to flag any inputs that an adversary might have tempered with. During this stage, ART can somewhat act like an antivirus application where it checks the inputs and flags the one that are adversaries to protect the DNN.
In conclusion, any new technology comes with strengths and weaknesses. Take E-mail technology for instance; it provided fast and convenient way of communication and reduced the need of hard copied documents dramatically. However, in the beginning, users were extremely vulnerable to different worms and viruses spread across mailboxes. But through several years of using them, we learned how to mitigate those vulnerabilities while enhancing its positive capabilities. Same goes with visual recognition technology. no one can deny all the goods that these systems have brought to us but to embrace it, it’s crucial to first: find possible vulnerabilities and second: have tools to protect our system against adversaries, and this is exactly where Adversarial Robustness Toolbox can help.
ART in Action
Open source demo can be found here: ART Demo.
This implementation contains attack and defense against a model trained on GermanTrafficSign dataset. Full documentation on each step of the implementation is included in the notebook file.
ART open source library
To install ART and start using it, check out the open-source release under Adversarial Robustness Toolbox .The release includes extensive documentation and tutorials to help researchers and developers get started.
 Sharif et al. 2016, “Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition”
 Eykholt, et al. 2018, “Robust Physical-World Attacks on Deep Learning Visual Classification” arXiv:1707.08945v5 [cs.CR]
You can contact the author at email@example.com
Modified on by fstein
AI & Quantum – Tie the Knot
Guest Post By IBM Summit Trainees Sophie Nguyen, Justin Miller, Kavita Dhallan, & Megan Clifford.
“Quantum computing holds various promises.” – Bob Sutor
Quantum computing. From the outside looking in, the subject may come off intimidating especially for a team of new Summit hires from various backgrounds in Marketing, Finance, Law, and Biomolecular Science. The event itinerary lists presenters like Steve Margolis, PMP, CISSP (not to mention an IBM Q Ambassador), Aaron Potler (who is a Distinguished Engineer focusing on High Performance Computing, and also an IBM Q Ambassador), Bob Sutor, Ph.D. (IBM’s VP for IBM Q Strategy & Ecosystem), and Kenneth Wood (the Global Business Development Lead for IBM Q Network). They are all IBM Q subject matter experts. Are we going to understand what Quantum Computing is? Where it’s going? How, as future IBM sellers, are we going to be able to do them justice speaking on quantum computing? Soon into the presentation, however, we quickly realized there was as much for us to gain from the event as the IBM clients that were invited to the A3 Center that day in Washington, D.C.
Bob started the presentation asking us to think of the words “open mind” differently. Yes, we’re constantly asked to think outside-the-box in the realm of technology, but he means it literally. “You’re thinking too classically,” as he explains in the world of computers, “Don’t do it today.” That’s exactly what he meant when he proposed that quantum computing holds various promises. To see those promises, we need to stop thinking classically.
Thinking Non-Classically. Steve Margolis dives into this after Bob. Today’s classical computer works on classical bits. For those of you who don’t know what a bit vs. a qubit is (like the people writing this blog), a bit is short for “binary digit”, the basic unit of logic that sits as 0 or 1. A qubit or quantum bit is the fundamental unit for quantum information just like bits are for classical computing. Qubits are able to exist in a combination of two states, a 0 or a 1, based on the principals of entanglement and superposition. Just remember… quantum computers work on qubits. This video was a great resource for understanding qubits.
Quantum Computing Promises and Possibilities. What are these “various promises” the presenters speak of? We’re talking about a computer that can compute mathematical data in numbers and volumes that are more than the number of atoms that exist on our planet Earth. We’re talking about applying that sort of ability from quantum computing into new application areas such as: Chemistry (material design, oil & gas, drug discovery), AI (classification, machine learning), and even Financial Services (asset pricing, risk analysis, rare event simulation). The takeaway was that unlike the classical computer (for example, today’s servers), quantum computing has room for exponential growth that can do more to help these application areas.
Current State of Quantum Computing. Currently, IBM has multiple quantum computers and a growing network of users (IBM Q Network). Yes, they admit that naming them after global cities ended up being confusing for instance, the IBM Q Tokyo and IBM Q Melbourne are actually located in Yorktown, NY. One name that you probably won’t get confused on is the latest IBM Q System One. It is a beauty! Not only is it going to pave the way for quantum computing possibilities, it is astonishingly beautiful. We’re talking the whole 9 yards, or should we say feet (it’s 9x9 ft). IBM built this with a dream team of engineers, mathematicians, and industrial designers to work on all its nuances such as sound sensitivity while thinking about it visually.
Future State of Quantum Computing. We still have research and development to do. We still have kinks to sort out, including error rates, as our goal is to create a universal fault-tolerant quantum computer. Right now we are in the “Quantum Ready” stage. That is, we are beyond the early stage of Quantum Science, making qubits work reliably and making them “last” longer, which is coherence time. In the Quantum Ready stage IBM and members of the IBM Q Network are developing algorithms to work with this new type of computing and making the infrastructure to run quantum computers in commercial data centers.
How do we continue to progress along this path to the goal of demonstrating a true advantage over classical systems for commercial and scientific applications? That’s where our next speaker, Aaron Potler comes in. He explains the available networks and quantum computing platforms available to continue growth and collaboration needs in order to gain more insight. Anyone can register to use the public, cloud-based 5- and 16-qubit IBM Q Experience systems. More than 110,000 people have run more than 7 million experiments, and published more than 130 research papers using the IBM Q Experience. The IBM Q Experience, as well as the commercial IBM Q systems use Qiskit, an open-source quantum computing framework for programming today’s quantum processors for research, education, and business. There have been articles and educational pieces published that lead us to ideas and theories about where quantum computing is going. Research articles include, “Quantum Risk Analysis” by Stefan Woerner, “Scientists Prove a Quantum Computing Advantage over Classical,” by Sergey Bravyi, and an article from MIT Technology Review, “Machine Learning, Meet Quantum Computing.” You can find these papers by following the link at the end of this article.
“First movers can accrue substantial value,” Kenneth Wood states as he enters next into the presentation. The world, including the US, is investing heavily in quantum research. Recently, Congress agreed on unanimously passing the National Quantum Initiative Act in November 2018. The initiative is to provide $1.275 billion in research funding from 2019 to 2023. Moreover, according to Gartner, “Within five years, analysts estimate that 20 percent of organizations will be budgeting for quantum computing projects and, within a decade, quantum computing may be a USD15 billion industry.” At CES 2019, Ginni Rometty comments “Quantum does not replace every kind of computer, it’s for a certain kind of problem. And it’s the kind of problem where the world doesn't realize how many things are approximated out there.” That is why we are ahead of the game in the number of quantum computers available to the public and why we have built the IBM Q Network. The missions of the IBM Q Network are: accelerating research, commercial application, and educate and prepare. The offerings of the IBM Q Network are: technology, enablement, collaboration, and business framework.
By the end of the presentation, we had ourselves thinking about the very questions Bob asked when he started, “How can you do more? How can you learn more?”
If interested, check out more about IBM Q and the articles mentioned here.
Find out more about the IBM Center for Analytics, Automation, and AI solutions at ibm.com/a3center
As government leaders do you believe the world is getting
more complex? More volatile? If so, you’re not alone - - Sixty percent of
the CEOs surveyed by IBM in our 2010 CEO Study thought the world was getting
more complex, and even more, 69%, felt the world was getting more
For the first time, we also posed a similar set of questions
to college students. These future
leaders viewed the world as even more complex than the CEOs we surveyed. But
they saw less volatility, and significantly less uncertainty than the CEOs (65%
of the CEOs, but only 48% of the students).
Could it be that the students are more acclimated to economic boom/bust
cycles and feel more comfortable with the uncertainty of today’s world?
Or could it be that in the instrumented, interconnected,
collaborative world that they are used to (most of the students never knew a
world without web browsing and many don’t remember the pre-Facebook era), they
feel more comfortable dealing with this complex world? As a student in France put it, “We will have more
information, so it [the world] should be more predictable.”
We found that students who had the greatest sense of
complexity put much more emphasis on the analytics and predictive capabilities
of information. They were 50% more
likely to expect significant impact from increased information than peers who
did not have the same sense of complexity.
And they were 22% more likely to believe that organizations should focus
on insight and intelligence to enable their strategies. Also,
interestingly, students in China
were significantly more likely to prefer a fact- and research-based style of
decision making than their peers around the world. Does that indicate that the Chinese students
have been trained to feel more comfortable dealing with data than their
With the baby boom heading towards retirement in the coming
years, does this mean the government workers who replace them will be more
comfortable using information and analytical techniques to handle the world’s
problems? Or could it be that complexity
will always rise to be just beyond our ability to manage it with our current
level of technology?
Click here to see the IBM Report: “Inheriting
a complex world”
Click here to see the IBM Report: “2010 Global CEO
More on Analytics for Government here: www.ibm.com/ASCdc
Do you think our future leaders are inheriting a more
complex world? And do you feel they are
more prepared to manage it?
Comment on this blog or write to me at ASCdc@us.ibm.com
Frank Stein, Director of IBM’s Analytics Solution
Does your government agency monitor the social media for information relevant to your mission? Should it?
IBM's Analytics Solution Center recently held a seminar to explore
how agencies and companies can obtain value and insight using social
Pat Fiorenza discussed how agencies can develop an ROI Model - Return
on Influence Model - for social media. Agencies use social media
analytics to help inform their decision making by gathering
information/research, and learn what other agencies and citizens are
saying. Interesting examples from CDC and Govloop were provided.
Learn more here.
Ed Burek, IBM, talked about how savvy companies are now taping into
customer generated content, how government agencies could do the same to
learn how tax payers feel about government actions and messaging. He
gave examples of how regulatory agencies could received the unvarnished
comments from those impacted by regulations, as well as how they could
stay on top of "negative chatter." IBM has created a framework to
derive business insight from the vast amounts of social media that is
now being transmitted. Called Cognos Consumer Insight it provides real
time information on trends and sentiment.
Rick Lawrence, IBM Manager for Machine Learning at Watson Research
Center next talked about the leading edge of social media analytics. He
provided examples from the research portfolio on discovering Who are
the Key Influencers? , Identifying emerging topics of discussion, and
Mapping the billions of tweet to concepts that we really care about.
All of the presentations are available on the ASC website under Past Events (May 10, 2012)
Does your agency care about what its constituents are saying about it
on social media? Does your agency need to have real time intelligence
on events within its mission space? With 340 million Tweets per Day, 2
million blog posts, and 500 million facebook updates, how can you find
the important information? Social Media Analytics may be an idea
whose time has come.
Analytics Solution Center
P.S. The Center for the Business of Government issued a new report on Tweeting in Government. Pat provided a good overview here.
Derechos, Droughts, Hottest July on Record, Shattered
High Temp Records, Greenland Ice Sheet Melts. Just what is going on with the weather these
days? Is this weather really abnormal or
does it just seem to be that way? Is this part of a trend? Does global climate change mean we’ll have
more of these extreme weather events? Being
a data and analytics person, I started looking to see what data analysis had
been done on this subject.
The US Climate Extremes Index[i] provides
a measure to track the occurrence of extreme data (although it doesn’t take
into account Derechos and other severe wind events). The trend of the index (smoothed) has been on
the rise since 1970 and now is at an all time high, as shown below. The Index
was at a record high 46% during the January-July period, over twice the average
value, and surpassing the previous record large CEI of 42% percent which
occurred in 1934. Extremes in warm
daytime temperatures (83 percent) and warm nighttime temperatures (74 percent)
both covered record large areas of the nation, contributing to the record high
year-to-date USCEI value.
This index is
compiled by combining measurements throughout the country (1,218-station US Historical Climatology Network)
that show the percentage of the country impacted by extreme weather in terms of
maximum temperatures much above or below normal, minimum temperatures
above/below normal, percentage of country in severe drought/severe moisture
surplus, percentage of the country with a much greater than normal proportion
of precipitation derived from extreme 1 day events, and the percentage of the
country with a much greater than normal number of days with
The U.S. Global
Change Research Program in 2009 published a study which documented the changing
climate and its impact on the United
study uses 3 standard forms of data analysis: 1) reports on observations, 2)
predictions based on the observed trends, and 3) modeling to better predict future
climate changes based on various assumptions about the amount of heat-trapping
gases in the atmosphere. While the first
two types are based on large quantities of collected data, they use only U.S.
observations. The modeling, however,
must be done on a global basis which substantially increases the amount of data
that must be crunched.
Here are some of the findings as they relate to extreme
Overall Warming of the Climate
Temperatures, on average, in the1993-2008 period are 1-2ºF
higher than in the 1961-79 baseline. By
the end of the century, the average U.S. temperature is projected to
increase by approximately 7-11ºF under a high emissions model and by
approximately 4-6.5ºF under a lower emissions scenario. The temperature observations show that there
has been an increase in warmer and more frequent warm days and warm nights, and
warmer and less frequent cold days and cold nights in most areas.
More intense, more frequent, and longer-lasting heat waves
In the past several decades, there has been an increasing
trend in high-humidity heat waves, characterized by extremely high nighttime
temperatures. Parts of the South that
currently have about 60 days per year with temperatures over 90ºF are projected
to experience 150 or more days a year above 90ºF under a higher emissions
scenario. In addition to occurring more
frequently, at the end of this century these very hot days are projected to be
about 10ºF hotter than they are today.
Increased extremes of summer dryness and winter wetness with a generally
greater risk of droughts and floods.
Trends in drought have strong regional variations. Over the past 50 years, with increasing
temperatures, the frequency of drought in many parts of the West and Southeast
has increased significantly. Models show
that the Southwest, in particular, is expected to experience increasing drought
as the dry zone just outside of the tropics expands northward with global
Precipitation coming in heavier downpours, with longer dry periods in
While average precipitation over
the nation as a whole increased by about 7% over the past century, the amount
of precipitation falling in the heaviest 1% of rain events increased nearly
20%. One of the outputs of the climate
modeling is to project the probability of certain events. For example, heavy downpours that are now a “1
in 20 year occurrence” are projected to occur about “once every 4-15 years” by
the end of the century. These heavy downpours are expected to be
10-25% heavier by the end of the century than they are now. This will likely cause more flooding events
(flooding depends both upon the weather and the susceptibility of the area to
More intense but fewer severe storms
Reports of severe weather such as
tornadoes and severe thunderstorms have increased during the past 50 years.
However the climate study indicates that much of this may be due to better
monitoring technologies, changes in population areas, and increasing public
awareness. Climate models do project an increase in the frequency of
environmental conditions favorable to severe thunderstorms. But the report notes, “the inability to
adequately model the small-scale conditions involved in thunderstorm
development remains a limiting factor in projecting the future character of
severe thunderstorms and other small-scale weather phenomena.[iii]” Advances in modeling and big data analytics,
as well as improved monitoring networks are likely to reduce this limitation in
The June Derecho that hit the Washington metropolitan
area shows an example of the current state of the art in forecasting a severe
storm. The Storm Prediction Center of
NOAA was able to provide approximately 4 hours advance warning of the
storm. Longer term predictions would
require additional data about the atmospheric instability that propelled the
Derecho from Iowa to the Washington
Metro area, as well as better real time modeling.
Shift of storm tracks towards the poles
Cold season storm tracks are
shifting northward over the last 50 years, with a decrease in the frequency of
storms in mid-latitude areas. The
northward shift is projected to continue, and strong cold season storms are
likely to become stronger and more frequent, with greater wind speeds and more
extreme wave heights.
The climate changes will have an
interesting effect on the so called “lake-effect”. Over the past 50 years, there is a record of
increased lake-effect snowfall near the Great Lakes. As the climate has warmed there is less ice
on the Great Lakes which has allowed greater
evaporation from the surface resulting in heavier snowstorms. Eventually, the temperatures are expected to
rise sufficiently that much of the precipitation will end up falling as rain,
reducing the snow totals.
While trending of individual elements such as temperatures
is useful, accurate predictions require consideration of the interaction
between the climate elements. For
example, there is mutual enhancement effect between droughts and heat
waves. Heat waves enhance soil drying,
and drier soil heats the air above more since no energy goes into evaporating
the soil moisture. Big data modeling can
show the results of this escalating cycle of warming on the future climate.
The New Normal
So it seems that all this abnormal weather we are seeing
will become the new normal. Forewarned
Analytics Solution Center, Washington, DC
[ii] Global Climate Change
Impacts in the United States,
Thomas R. Karl, Jerry M. Melillo, and Thomas C. Peterson, (eds.) Cambridge University Press, 2009
Contestants on “Jeopardy!” have to understand the clues in
light of the Category (context) and then quickly sift through their accumulated
knowledge over their lifetime to come up with potential answers, decide if they
have confidence in their answer, and then quickly respond by hitting the
buzzer. Except for the buzzer, does
that sound familiar and relevant to your job? Would a computer that could do all that plus
cite the evidence backing up its answer be helpful to your job? If so, then read on.
At last week’s Analytics
Solution Center June seminar, David Ferrucci from IBM Research described
the IBM Research Grand Challenge of making a computer that can win at the game
of “Jeopardy!”(For more on “Watson,” the computer behind the challenge, check
New York Times Magazine cover article from last week)
Les Drieling, the keynote speaker and former US Government Intelligence
Agency senior scientist (now an IBM executive in the Global Business Services NISC
business unit), highlighted the needs in the intelligence community to sift
through very large volumes of structured and unstructured data with the goal of
making very important (some life and death) decisions under extreme time
pressure. It might seem obvious that the
DeepQA technology behind the “Jeopardy!”
project can be used to help intelligence analysts to filter and retrieve information using natural
But don’t many agencies have the need to retrieve
information quickly and precisely in response to questions? For example, the Coast Guard may have wanted
to know what dispersants are best for using in the open ocean to fight the Gulf Coast
oil spill, what evidence exists as to their efficacy and safety, and a
confidence value on the proposed answer.
The open government movement is also spurring use of this
sort of technology to help its citizens.
For example, a citizen could use it ask about the status of a new law or
who to contact with regard to a particular problem.
Let’s say the citizen’s question was “How to get my car
emissions tested when I’m away at college out of state and this state doesn’t
do emissions testing?” A question &
answer system would have to be able to decompose the query, search its database
for possible answers, and then select the best answer to return to the citizen.
If all the questions are known ahead of time, then government personnel can
develop a simple look-up system.
However, when there are so many questions that they can not all be
itemized ahead of time, then a more sophisticated approach is required. This is where the IBM DeepQA technology comes
in to play.
Text Analytics underlies DeepQA as well as the other presentations
at the June Analytics seminar. Another presentation showed how the National
Highway Safety Administration (NHSTA) Defects and Recalls database could use a
text analytics tool to alert on the possible connection between Toyotas and
rapid acceleration much earlier than this pairing came to the NHTSA, or the
public’s, attention. The tool used for
this demonstration was the IBM
Content Analyzer and it allows enables you to search, discover, and perform
the same analytics on your textual data that is done with structured data. In the demonstration it identified the
unusual relationship between “Toyota”
and “acceleration” – automatically.
Another example showed how an Intelligence Agency was using
text analysis to analyze the performance of its counter terrorism efforts. In this example, report text was analyzed and
scored to determine whether their objectives were being met and which intelligence
methods were most helpful in meeting their objectives.
Finally, IBM showed how text analytics could be used to
discover major themes that occurred when USAID (US Agency for International
Development) ran a “Jam” that asked participants from around the globe to
propose “pragmatic ideas and solutions to some very real issues and problems
facing our communities and our world today” (this is from the Global Pulse 2010
Website). The “Jam” tools quickly identified and classified the participant’s
key ideas in real time so that later participants could join the conversation
on their topics of interest.
The Category is “Government.” For $100, the clue is “Text Analytics.” <Buzz> “What can help the Government make better and
faster decisions from our mounds of unstructured data?”
To see the charts and listen to the replay from the seminar
go to www.ibm.com/ASCdc and look under
Give me your comments on how this technology can help you and
your agency or write to me at firstname.lastname@example.org.
- Frank Stein, Director of IBM's Analytics Solution Center, Washington, D.C.
On July 4th, CERN scientists announced that they
observed a particle that strongly resembles the Higgs boson, a critical element
of the standard model of particle physics.
This particle is thought to be responsible for the characteristic of
mass, which gives objects weight when combined with gravity.
Detection of the Higgs Boson would not have been possible
without the last decade’s advances in processing big data. Joe Incandela, CMS Spokesman at CERN,
explained that if every collision that they scanned was a sand grain, these
sand grains would have filled up an Olympic sized pool over the last 2
years. They had to find the several
dozen or so grains of sand that exhibited characteristics consistent with the
In addition to developing the Large Hadron Collider, the
CERN teams also developed a data strategy to deal with the data from the
hundreds of millions of particle collisions occurring each second. The sensors record the raw data on billions
of events occurring in the proton collider. These readings are then reconstructed
to show the energy and directions of many particle traces. The data goes through 2 stages of filtering
to reduce the data on 40 million collisions/sec down to 10 million interesting
ones per second, and then to 100 or 200 collisions that are studied in
According to Rolf-Dieter Heuer, director general at CERN, “The
computing power and network is a very important part of the research.” Over
15 Petabytes (1 million Gigabytes) are stored each year. This is distributed through the Worldwide
Large Hadron Collider Computing Grid (WLCG) to each of 11 major Tier 1 centers
around the world, and from there to research centers and individual
scientists. In the U.S., the Open
Science Grid, supported by NSF and DOE, provides much of the compute and
storage power for this work. The
scientists use Monte Carlo simulations for
generating and propagating the physics interactions of the elementary particles
passing through the collider to determine which ones correspond to the
hypothesized behavior of the Higgs Boson.
What they found was a never seen before elementary particle
that seems to fit the behavior of the Higgs Boson and is very heavy –
approximately 133 proton masses. Further
data analysis is now needed to ascertain its spin, decay modes, and other
Think the amount of data generated by the Large Hadron
Collider is huge? The forthcoming Square
Kilometre Array radio telescope is expected to generate 100’s of Petabytes of
data per day. More on that in a future
My work this year has taken me from Big Data and Analytics towards Cognitive Computing and what IBM is now dubbing Cognitive Businesses (or Cognitive Government in our case). Cognitive businesses leverage cognitive computing technology (think Watson) to enhance, scale, and accelerate the expertise of their personnel. Below is the summary of the first part of a symposium I co-chaired last week. I'm happy to answer any questions you may have.
The AAAI Fall Symposia on November 12-14 included tracks on 1) AI for Human-Robot Interaction, Cognitive Assistance, Deceptive and Counter-Deceptive Machines, Embedded ML, Self Confidence in Autonomous Systems, and Sequential Decision Making for Intelligent Agents. This post will provide my general impressions of the Cognitive Assistance symposium.
Jerome Pesenti, IBM VP of Watson Core Development, provided the 1st day keynote. He started with the great quote from Fred Jelinek (Cornell/IBM/JHU) that “Every time I fire a linguist, the performance of the speech recognizer goes up.” He then talked about how deep learning is allowing reco systems that approach or surpass human performance. This led to a lively discussion with the audience on the universality of learning algorithms and whether the machines were learning in the same manner that humans learn something (no). Jerome finished with some applications of Watson including the Oncology Advisor, citizen support (e.g, tax questions), and security (finding relationships between data).
The rest of the morning was filled with examples of cognitive assistance for legal tasks such as filing a protective order (Karl Branting) and human-computer co-creativity in the classroom(Ashok Goel), and a tool to help SMEs define their vocabulary to find the most relevant content on the web (Elham Khabiri).
During lunch, much of the symposium had lunch together and a lively discussion ensued on cognitive assistance. One topic that I found interesting was on ultimate chess where human-machine teams compete. While these teams in the past have beaten computer-only teams, Murray Campbell noted that the advancements in chess playing computers are decreasing the value-add of humans to the team.
The afternoon session of Day 1 started with 2 interesting talks on cognitive assistance for helping those with cognitive disabilities. Madelaine Sayko described Cog-Aid which would include a cognitive assessment, recommender system (based on the assessment) and an intelligent task status manager for starters. Then Daniel Sontag described the Kognit technology program which includes tracking dementia patient’s behavior using eye tracking and mixed reality displays to assist the patient perform activities in daily living. Kevin Burns presented a sense-making approach that could be used by an intelligence analyst to help understand and define the Prior and Posterior probability calculations as new evidence is added. This could eventually be embodied into a cognitive assistant. Next came a presentation on capturing cybersecurity operational patterns to facilitate knowledge chaining by Keith Willett.
The final session of the day was a panel discussion of workforce issues associated with cognitive assistants led by Murray Campbell. Erin Burke of Fordham University Law School talked about how legal education must transition and that she is working at the intersection of law, big data, and cognitive computing. Jim Spohrer, Director of IBM’s University Programs, provided some predictions including that by 2035 everyone will be a manager and will have at least one Cognitive Assistant working for them. A lively discussion ensued with the audience about our forthcoming relationship with Cogs including whether we could trust them, unintended consequences, whether we can build common sense into a Cog, and whether our brains will atrophy as we depend on Cogs.
I’ll cover Day 2 in the next blog post.
I gave the keynote address to George Washington University’s DATA Conference on December 2. This is what I told the students. Please reply with your thoughts and ideas to extend the conversation on how to make the world a better place through data.
Think about how you can use data and data science to make the world a better place. We are now in a unique time in history because we now have huge amounts of data being collected by all the digitized systems in the world (almost 1 ZB or 1 times 10 to the 21st power Bytes) and the Data Science techniques are becoming more powerful and easier to use. These two factors will give you the ability to do more to improve the lives of your fellow students, their professions and society at large than has ever been possible before.
Data Science innovation will be central to solving humanity's grand challenges by capitalizing on this unprecedented quantity of data now being generated on human behavior and attitudes, human health, commerce, communications, migration and more. You can help to accelerate and advance the development and democratization of Data and Data Science solutions that can address specific global challenges related to poverty, hunger, health, education, the environment, and others.
To help stimulate your imagination, I will present several examples from our work at IBM. The key is to combine your growing expertise in Data Science, with your passions. At IBM, we are encouraging students to combine Data Science studies with other disciplines, such as natural science, social sciences, healthcare, etc. - - the problem domains where the Data Science can be put to work.
For the first example of “Doing Good”, I’d like to tell you about IBM Fellow Chieko Asakawa. She became blind at the age of 14, and as a result has devoted her professional life to building solutions to allow her and other blind people to access the world and regain their independence. Chieko has developed an object recognition solution so she can “see” ordinary objects in her home and at stores, and allow her to pick out wine or know the directions on a package – all using machine learning. She has also developed an indoor navigation system that helps her to easily get from place to place at work. Both use smartphones as the user interface. See these links for more details on Chieko’s inventions: Image rec: https://www.youtube.com/watch?v=RNp4OpToAdQ (many interesting solutions, Chieko’s is featured at minute 17); Nihonbashi Tokyo NavCon: https://www.youtube.com/watch?v=mlGcutE2t2A ; TED talk: http://www.ted.com/talks/chieko_asakawa_how_new_technology_helps_blind_people_explore_the_world ).
The second example is from IBM’s Cognitive Build Competition. Two IBM employees, Karibi and Jenn proposed and prototyped a solution to help children with Autism. The solution, dubbed Pino after Karibi’s newphew, uses Watson Conversation service to help children with autism communicate more independently by providing real-time verbal prompts. It can also be used with other conditions that affect communication ability, such as stroke and Alzheimer's disease. I met Jenn a few weeks ago. She told me, “At a birthday party a couple of years ago, I saw how upset my son was when he didn't receive a cupcake because he couldn't say "yes" when offered one. He needs a therapist or caregiver to prompt him to answer basic questions. He has a communication device that can help him speak, but it requires him to know he needs to respond… When Cognitive Build started, I thought it would be great if my son's communication device could be cognitive so it could help him to be more independent when I'm not around.” Learn more at this link: https://medium.com/cognitivebusiness/addressing-autism-project-pino-3741ce13d39
The third example is about the opioid epidemic, which has become one of the worst health crises in US history. In 2015, more than 90 Americans died every day from opioid overdoses, a number comparable to deaths in car accidents and projected to have risen further in 2016 and 2017. The Centers for Disease Control and Prevention (CDC) estimate the total economic burden of prescription opioid abuse to be $78.5 billion a year, including healthcare costs, lost productivity, and criminal justice involvement.
For many addicts, the problem often begins with legitimate healthcare treatment in which opioid painkillers are first prescribed, such as for surgeries or chronic back pain. During treatment, some patients become addicted and go on to suffer the well-documented consequences of addiction, while others do not, even if they become long-term users. To combat the epidemic, it is vital to understand the exact circumstances under which medically sanctioned treatments can devolve into addiction. That’s where data science comes in to play.
This summer, we took the first steps in tackling this question in a project within our Science for Social Good program. The team, led by Bhanu Vinzamuri, focused on analyzing the relationship between factors surrounding an initial opioid prescription and a subsequent diagnosis of addiction. We found that those people that received initial prescriptions for more than 7 days has a significant correlation to Long-Term usage, as does use of Synthetic Opioid prescriptions. We also confirmed that days of supply matters much more for addiction than quantity (e.g. in milligrams of morphine equivalent) prescribed per day. Other factors that were positively correlated with long term use and which should be used by doctors when prescribing opioids were age, certain regions of the country, rural location, healthcare utilization and depression, osteoarthritis, or diabetes. See more projects at http://www.research.ibm.com/science-for-social-good/#projects
Because of the power that Data Science and data is bringing to Humans, we need to be sure it is a force for good and not for evil. IBM and XPRIZE Foundation believes Artificial Intelligence (and the data science algorithms it uses) will be central to solving humanity's grand challenges. Solutions to pressing problems related to health and wellbeing, education, energy, environment, and other domains important to humanity can potentially be found by capitalizing on the unprecedented quantities of data and recent progress in emerging AI technologies. That’s why IBM is putting up $5 million for the Watson AI XPRIZE. See https://ai.xprize.org/ for more details.
But even if you are not up for competing for the AI XPRIZE, there is lots that you can do. Find a societal problem that you are passionate about. It all starts with a problem or need, like Chieko’s blindness, or Jenn’s child with autism, or the opioid crisis. Then come up with an idea or approach. There is a lot of data now available. Our Data Science Experience is out there on the web for you to play with. It is designed to allow data scientists, business analysts, stakeholders, and programmers work together on a data project. It’s easy to use. Go out and try it. There are tutorials to guide you. It is at https://datascience.ibm.com/ . Don’t just study the problem and write a school paper, create a solution that helps people. Your university’s office of entrepreneurship can help you to build a business case for your solution. Finally, consider pitching your idea to one of the many Pitchfests that are around. One I’m familiar with that exposes your ideas to corporate sponsors such as IBM is NCET2. They are at https://ncet2.org/. Go ahead and make the world a better place!
Modified on by fstein
GUEST POST BY Mike Byerly, IBM Client Representative
Report on Thursday, September 27th: A3 Center Event: Using Artificial Intelligence to Transform Government
How has the US federal, as well as state and local, government already leveraged artificial intelligence to drive outcomes for their constituents? What have they learned? What has worked well, and what hasn’t? What lessons can other agencies learn from? These were the questions on discussion during the recently held event Using Artificial Intelligence to Transform Government, held on September 27th.
Located in the heart of Washington, DC, IBM’s A3 Center (which stands for Analytics, Automation, and AI solutions) stands in a unique position to bring together the public and private sector for thoughtful discussion. In the discussion, senior leaders from the US federal government, municipal government, the private sector, and IBM discussed some of the key successes and challenges that individual leaders have encountered when leveraging AI technologies to improve customer (and citizen) outcomes.
The conversation was wide-ranging and touched upon a number of different topics – the entire recording and pdf's can be found here. In order to keep this blog post to a reasonable length, I have highlighted a few of the topics discussed, and gone into more detail around a few select remarks.
The first panel, entitled “The Future Has Begun: Using AI to Transform Government” was moderated by Mallory Bulman (of the Partnership for Public Service) and included Camron Gorguinpour (Principal, Woden LLC), Alex Holsinger (Criminal Justice Coordinator for Johnson County, KS), and Maureen Rajaballey (IT Manager, Miami-Dade County, FL). All panelists made interesting points. Maureen Rajaballey, IT Manager from Miami-Dade County, Florida, discussed how her county had begun to leverage call center solutions to improve their bill-collecting capabilities. One of the first questions she was asked by her workers was: is this technology going to replace me? Is it going to take my job?
Maureen discussed the importance of emphasizing how technology can play a role in augmenting and improving the lives of call center works. What call center worker wants to work at 3AM, or on holidays? By focusing on the positive aspects of the call center technology, and involving those workers in the discussion about its implementation, she was able to create advocates that embraced the technology. The initiative has proven a huge success. A new analytics dashboard allows Miami-Date to monitor how many calls they receive per hour, how many are taken by AI, how many are resolved, how many gas vs. electric bills are being paid. By continually reinforcing the “wins” of the program, Mallory has been able to expand its success.
The second presentation, by Kevin Desouza of Queensland University of Technology, discussed his thoughtful report “Delivering Artificial Intelligence in Government.” Desouza frames the governmental opportunity in three key areas: technology and data, workforce, and risk management. In his view, government agency leaders are already beginning to take the first steps needed to take advantage of artificial intelligence solutions, for example, upgrading existing IT infrastructure to support AI systems, identifying data intensive applications that can benefit from AI and establishing data governance to take advantage of the benefits of AI, and enabling their workforce to use AI (through agile implementations and redesigned work processes). By being aware of these challenges and addressing them thoughtfully, government officials are more likely for a successful implementation of AI that will be embraced and yield results.
The final presentation, “Delivering AI in Government: Challenges and Opportunity” was moderated by Claude Yusti (of IBM), and included Franz Gayle (Science and Technology Advisor within the Marine Corps), Joe Greenblott (Acting Director, Analysis Division, Office of Planning, Analysis and Accountability, EPA), Jose Arrietta (Associate Deputy Assistant Secretary, Division of Acquisition, HHS), Mallesh Murugesan (Founder & CEO, Abeyon) and Armita Soroosh (of TSA).
Again, all of the panelists made interesting points. Jose Arrietta of Health and Human Servicers discussed how his team more effectively managed $24 billion dollars per year of government contract spending. By ‘building the limbic system of the enterprise’ (an indexed taxonomy of buying behavior across departments), HHS was able to push pricing info directly to their agents, show them the best prices, and enable them to negotiate better rates with suppliers, all in real-time. By doing so, Jose drove dramatic cost savings. By beginning with a small pilot project, getting consecutive buy-in as the project increased in scope, and eventually rolling out the solution to all agents, HHS was able to entirely transform its buying behavior. The cost savings were significant.
Franz Gayle, of the USMC, discussed the broad range of interest that the DoD has expressed in artificial intelligence and its potential. For example: while the Marine Corps has traditionally been characterized as a “follower” within the DoD in terms of innovation and trying new things (following the Army and Air Force), current leadership understands that today, this is no longer a viable strategy. For this reason, the USMC is currently working with AI technology firms to fund projects that “can fail”. By carefully implementing AI solutions that are similar to those that have been tried and tested in private sector, risk can be in reduced to the USMC. Creating useful military applications should therefore be possible and relatively straightforward. The DoD recognizes the importance of continuing to evolve its AI capability.
While the government is still in its early stages of leveraging the full potential of Artificial Intelligence technologies, the discussions made it clear: there are government agencies realizing benefits from AI today. Change will continue, and it will only continue to accelerate. IBM is working hand-in-hand with government customers, using best practices learned from the private sector (and other government customers), to adopt AI successfully. Public agencies are still in their relatively early days of experimenting with AI, and these efforts are bound to intensify.
To help government innovators progress in this area, the A3 Center will continue to hold events to discuss these important topics. Visit the A3 Center website to see upcoming events and register to attend.
Modified on by fstein
GUEST BLOG BY MICHELLE HUCHETTE:
My name is Michelle Huchette and I am a rising fourth year at the University of Virginia studying Computer Science and Statistics. This summer I was fortunate enough to be a part of the IBM Summit Program as a Technical Sales Intern. In this role I was able to experience what it is like to be an IBM seller by attending customer events and working on various tasks and projects over the course of 11 weeks. A few weeks ago I was challenged with creating a Proof of Technology lab that would interest customers in the field of machine learning. This is a brief overview of the creation and utilization of the model I created to diagnose breast cancer tumors.
The data set used for the lab was found in UC Irvine Machine Learning Repository (https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29) that contained information regarding breast cancer tumors and information to help predict the diagnoses of the tumors as malignant or benign. The data set contains 10 measurements of each cell nucleus captured using images of cell nuclei gathered from a fine needle aspiration procedure (FNA). The average, standard error, and extreme values of all nuclei in the tumor sample were calculated for each of the following features:
- Texture (standard deviation of gray-scale values)
- Smoothness (local variation in radius length)
- Compactness (perimeter2/area – 1)
- Concavity (severity of concave portions of the contour)
- Concave points (number of concave portions of the contour)
- Fractal dimension (coastline approximation – 1)
After finding a data set, a machine learning model could be created to diagnose breast cancer tumors. In order to do so we needed to set up a Watson Studio account on the IBM Cloud platform( https://console.bluemix.net/registration/ ). Within Watson Studio(https://console.bluemix.net/catalog/services/watson-studio ) we created a Jupyter Notebook which was used to write a python code to work with the data set, create a model, and make the predictions, all using Apache Spark as the analytics engine.
The first step to creating the machine learning model required determining which type of model would be the best fit for the data. Research found that there are different types of models that Spark supports such as Naïve Bayes, Decision Trees, Random Forests, and Regression Models, which are the most common. Because Naïve Bayes required a strong independence assumption between the features, that type of model was ruled out. Ultimately, a Logistic Regression Model was chosen since it is often used for models of binary categorical outcome (exactly what we’re dealing with when trying to diagnose a tumor as malignant or benign) and it is good at measuring the relationship between the labels and features.
To start out, the logistic regression model was set to have the default parameters so that an initial model could be created and improved upon if needed. Once the model was defined, a pipeline was set up which contained a sequence of stages to be run in a specific order. Within a pipeline each stage is either a transformer, which converts a dataframe into other dataframes, or an estimator which calls fit() and trains a model. There are many different options that you can include in your pipeline, including tokenizers, hashes, normalizers, etc. In terms of this dataset and for the sake of creating an easy to follow lab our pipeline started by using StringIndexer to turn the label (diagnosis) into a form that SparkML could use by encoding the input columns to a column of indices based on their frequency. Then a Vector Assembler combined the list of columns into a single vector column to be used in training the model. A normalizer was added to normalize each vector into a standard form to improve the algorithm. Lastly, our defined logistic regression instance was implemented and IndexToString was used to get the results of the model back into human readable form.
Following the definition of a pipeline, the logistic regression model and pipeline could be used to train and test the model. The data set was split with the standard 70/30 split for the training and test dataset, respectively. The training data set was then used to fit the pipeline and train the model to make predictions and the accuracy of the model was tested using a Receiver Operator Characteristic curve for binary classifiers. This value is calculated by plotting the true positive rate (recall/probability of detection) against the false positive rate (fall-out/probability of a false alarm) at various levels. A value when using the ROC curve that is close to 1 suggests that the model performs very well, whereas a value close to 0.5 is about as good as flipping a coin. Once the model was trained and evaluated, the test data set was used to make predictions The logistic regression model that was created in the steps previously described resulted in a value of 0.989, meaning it was able to predict the diagnoses of tumors very well.
Even though the model was already proven to be able to diagnose tumors accurately it could still be improved on. Hyperparameter tuning includes the use of model selection tools that test different parameter values for the pipeline and find the best possible values. There are two main options when working in Spark in terms of model selection tools, a CrossValidator or a Train-Validation Split. For this project we used a CrossValidator because even though they can be more expensive for larger data sets, they are more reliable when the data set isn’t sufficiently large because it evaluates each parameter k times, rather than just once.
CrossValidators first split the data set into “folds” which are used as separate training and test data set pairs. We set the value of the number of folds for this project to be 10 and therefore the CrossValidator generated 10 training/test data set pairs which are all used to test the parameters. The average performance among the 10 instances for each parameter are averaged and compared to other parameter values tested. We defined a paramGrid which stated the values to be used for the parameters within the pipeline. For this pipeline we could define values for maxIter, elasticNetParam, regParam, which are the parameters in the logistic regression model, or the normalizer parameter of the pipeline. Included in our paramGrid for this lab was parameter values for elasticNetParam, which must be between 0 and 1. This is an important parameter in the pipeline because it can make the model closer to a Lasso regression model (coefficients that are not relevant are set to 0) with a value close to 1 or a Ridge regression model (minimize the impact of irrelevant coefficients without setting them to 0) with a value close to 0. Because of this the values to test the elasticNetParam in the grid were set to 0, 0.5, and 1 to see which type of regression model would be best for the data. The second parameter defined in the paramGrid is the normalizer from the pipeline. The normalizer ensures that the algorithm runs correctly and the value set for the parameter represents the p-norm for normalization. The default value of 2 was previously used so within the paramGrid the values to be tested were set to 1 and 3.
After using a CrossValidator to find the best paramMaps, that model was trained using the testing data set and it was evaluated. The model improved 0.058% due to hyperparameter tuning, meaning the newly defined model was 99.5% accurate.
With an almost perfect predictive model defined, the last step was to grab the undiagnosed tumors from the original data set and use the model to predict their diagnoses with a high level of confidence.
The creation of this highly accurate model shows the power of Machine Learning in bettering the lives of people worldwide. It allows for the augmentation of breast cancer diagnosing and ensures that doctors see the patients in dire need of medical attention. Models such as these can help detect cancer earlier and, in more individuals, than doctors can do alone. Machine Learning has already started to be implemented in oncology to diagnose tumors, pathology to analyze bodily fluids, and in diagnosing rare diseases using facial recognition and deep learning to detect rare genetic diseases. Machine Learning serves many purposes from chatbots to augmenting the medical diagnostic process and with the continued advancements in technology and AI its applications are sure to expand even more.
Do today’s MBAs need Analytical Skills? That was the question that a recent Symposium
tried to answer.
On October 21, George
Institute for Integrating Statistics in Decision Sciences (I2SDS)
and IBM’s Analytics
held a Symposium entitled: Analytics and the 21st Century MBA. The abstract provides a good description of
the thesis of the Symposium:
The 21st century belongs to those who can think and act analytically. No
longer is it good enough to make business decisions, no matter what the field,
based on little more than feelings or gut reactions to events. Consumer
products companies, insurance companies, banks, governments, and even sports
teams are turning to Analytics to improve their bottom line and assure their
survivability in this age of hyper-competition and increasingly severe
This Symposium… will demonstrate how Analytics is, a critical component
of 21st Business careers, whether the practitioner's primary responsibility is
in a functional area (Marketing, Operations, Finance, Strategy, International
Business, HR) or a vertical such as Health Care or Tourism.
The Symposium provided talks by leading users of Analytics in Marketing,
Retail, Finance, and the Public Sector.
More on the Symposium is at: http://business.gwu.edu/decisionsciences/i2sds/pdf/GWU%20ASCOutline.pdf
Do you agree with the thesis? Are you
seeing more need for employees with analytical skills? Do you think those with these skills are
having an easier time getting jobs?
I’d like to hear your thoughts.
Frank Stein, Director, Analytics
Watson is the only computer on the planet that can answer a Jeopardy!
question in less than three seconds - fast enough to be competitive with the
world’s best human players.
of you that missed the match click here
to see a video clip from the match.)
But can a Watson-like computer help the government?
Watson was optimized to tackle a specific challenge:
competing against the world’s best Jeopardy! contestants. It does this by sifting through large amounts of unstructured information to find potential answers and assigning a confidence measure to each potential answer. When it has high confidence in an answer, it will buzz in and offer the answer. Beyond Jeopardy!,
IBM is working to deploy this technology
to businesses and governments dealing with the information overload
problem. At work, few of us are like
Ken Jennings, able to instantly answer almost every question thrown at us - -
with an 80-90% success rate. There is
simply too much information and more information is coming in all the
time. Whether we’re in finance, HR, IT,
or another area, our success at work depends upon dealing with huge volumes of
information, sifting through it to find
the “good information”, and then using the information to make decisions to do our
job. Technology like that used in Watson can provide for our consideration potential answers as well as the "evidence" it used to come up with potential answers.
In discussions recently with some of our military colleagues,
they came up with numerous ideas for deploying Watson-like technology. They cited the problem of “request overload” - - dealing with all the
requests for Predator and similar UAV missions.
How could they deploy their limited resources to best effect? Another person mentioned the problem of
sifting through all the intelligence information – most of it in the form of
unstructured information formats such as video and text – to find the relevant
information to a mission they were planning.
Another discussed the problem of monitoring their “situational
awareness” and how hard it was to keep track of all the data coming in. “Could Watson help monitor our security
posture and alert us to potential threats?” asked another.
Are you dealing with massive amounts of information? How could a Watson-like system assist you at
work? Do you want to recruit Watson to
work for your agency? We want to hear
your thoughts either in this blog or directly.
Write to me at email@example.com.
We’re hosting 2 free Watson Overview Briefings
on July 26 and 27. More
information at our website: www.ibm.com/ascdc
Frank Stein, Director, Analytics Solution Center
In these tough fiscal times, all agencies are going to be
focusing on doing more with less. How
does one get more done with less budget and staff? Consider turning to Analytics.
The consulting firm Nucleus Research has been looking at the
Return on Investment (ROI)
for various types of IT projects.
According to David O’Connell, Principal Analyst at Nucleus Research, “projects
involving analytics have some of the highest ROIs of any projects studied.”
Nucleus Research recently studied an analytics project IBM performed at DC
Water, the local water authority for Washington,
DC. In 2008, IBM began a first of a kind project
using advanced analytics to create a smarter water system that analyzes data on
valves, storm drains, service vehicles, truck routes and more to optimize its
infrastructure. With some pipes and other assets that date to the Civil War,
maintaining high levels of service while replacing older infrastructure is an
The project has resulted in the following benefits from a combination of IBM
Asset Management and Analytics technology and services:
Field Services trucks can be automatically
routed to optimize work management. This results in more work orders being
completed each week, as well as up to 20 percent reduction of fuel costs
related to fewer truck rolls and reduced "windshield" time.
Revenue loss from defective or
degrading water meters allowed recapture of $3.8 M because the analytics behind
the advanced metering infrastructure delivers more timely identification and
replacement of those meters. Revenue was
also recaptured because DC Water can now identify and bill locations where
there is unmetered water usage.
DC Water has been able to identify
assets most critically in need of repair using predictive analytics, so aging
infrastructure replacement programs can be more accurately scheduled,
preventing costly incidents that reduce service quality, such as outages and
water main breaks. This reduces both
maintenance labor costs and call center
costs associated with emergency incidents.
Nucleus Research reported in its case
study that the DC Water project resulted in $19.677 M of benefits over 3
years with a cost of $883 K, giving an ROI of 629%.
In 2010, Nucleus Research studied a number of other public
sector analytics projects. The results
from these projects are shown in the chart below. On average, the analytics projects have
resulted in an ROI of almost 600%! This
means that over 3 years, the projects have returned benefits 6 times the
original cost of the projects. The
payback period has been less than a year in all cases. This is important to government agencies because
it means you can see cost savings in the same fiscal year that you invest in an
According to David O’Connell, Principal Analyst at Nucleus
Research, “When government entities adopt
analytics, returns are high for two reasons.
First, waste such as leaky water mains, defective meters, or benefits
overpayments can be identified and eliminated.
Second, by making information more readily available, employees spend
less time looking around for information and more time getting their jobs done.” O’Connell went on to say, “Another improvement is better use of
workers’ time. The more an organization
knows about the public it serves, their needs, and the means of delivering
service, the smarter managers’ decisions are when they hand out workers’
Has your agency implemented any analytics projects? What’s been your experience?
Don't feel comfortable sharing
publicly? I'd be happy to hear your thoughts directly as well (firstname.lastname@example.org).
(net savings year 1 + net savings year 2 + net savings year 3)/3 * 100