How to sharpen Watson Visual Recognition results with simple preprocessing

The IBM Watson Visual Recognition service enables your applications to turn images into actionable data. You can obviously use it to tag images for content, recognize faces, and find similar images, but that’s not all it can do.  You can also create what are known as custom classifiers train the Visual Recognition service to recognize specific content within your images.

How might you use this in a real world case?  Imagine you want to train a system to:

  • Recognize specific structures in satellite images,

  • Spot cracks in pipes,

  • Pinpoint rust or corrosion on infrastructure,

  • Flag known defects in a manufacturing process.

Or any other of the millions of use cases?  Custom classifiers are how you can implement these types of solutions.  You train the service based on collections of images that are both positive and negative of the condition that you want to classify, and then you’ll be able to use it to analyze other images to identify the presence of that condition.  Keep in mind, the quality of your output is directly related to the quality of your training data, so you’ll want to make sure that you have high quality training based on real world sample images.

In many use cases a general/entire-image classification might not be enough.  If the condition you want to identify is only within a smaller region of a larger image, the entire image might not be classified with high enough confidence and a positive result could be missed.  Or, what if you want localized classification – meaning, where within the image can you find the areas recognized by the classifier?

Improving recognition of finer details with image preprocessing

One technique that I have used successfully on several projects is to use slice an image into smaller images, and analyze each one of the images individually against a custom classifier.

This approach lets us analyze each portion of the image independently, and then assemble the results into the bigger picture to provide a view into where within that image there are conditions recognized by your classifier.

Here’s a sample from one insurance-focused solution, which shows areas of hail damage that are recognized on a shingle roof by the Watson Visual Recognition service.

The colorization of the “heat map” is based on the confidence score for each tile images that is coming back in the results of the Watson Visual Recognition service.

Below you can also see a video of another sample, which demonstrates rust detection using Custom Classifiers.  Using image slicing, we can identify where within that image there is the presence of rust.



Try it yourself with this sample code

If you’d like to test this out with your own images and custom classifiers using the Watson Visual recognition service, you’re in luck! The code for this project is available as a learning resource, or for extending into your own solutions at (GitHub).

The sample is a Node.js application that uses the GraphicsMagick module for image preprocessing, submits the tiled images to the Watson Visual Recognition service for analysis, and then display the results within the browser superimposed over the source image.   The colorized squares represent Visual Recognition service’s confidence score for each area.

Confidence Scores

When you’re thinking about this type of scenario, it’s also important to understand confidence scores for custom classifiers returned by the Visual Recognition service.  Many people incorrectly think of confidence scores as a measure of absolute truth; it’s better to think of them as a threshold for action.   For example, consider results with a confidence score of only 60-70%, but the results are consistent, reliable, and do not include false positives or false negatives.  This would be a scenario where a confidence score of 60% is extremely valuable, and you wouldn’t need a 90% confidence score to perform an action.

Confidence scores can be though of much like temperatures in a weather report—I don’t know what the scientific definition of 75 degrees Fahrenheit is, but I know that it’s a pleasant temperature.  Likewise, I don’t know exactly what 50°F means, but I know that I’ll need a jacket.

From a cost-benefit perspective, I know that if I don’t wear a jacket at 50°F, then I am going to be cold.  Likewise, in my custom classifier scenario, I might be able to make a similar type of assumption or assessment. In my workflow, will it cost less if I invoke an action based on a confidence score of 65%?  In many cases, this answer could be yes, and this is where the value of the service comes in.  Of course, confidence score values are subjective.  They vary based upon training images, evaluation images, and the types of criteria that you want to classify. It is up to the developers of each solution to determine what confidences score values are appropriate thresholds for action for a given solution.


With a little bit of preprocessing, your application can improve the detection accuracy of the Watson Visual Recognition service, especially when applied to smaller details within an image. Are you ready to learn more?  Check out these references for more information:

Photo credit: A rusty bridge (


More from

Closing the breach window, from data to action

6 min read - Accelerate threat detection and response (TDR) using AI-powered centralized log management and security observability It is not news to most that cyberattacks have become easier to launch and harder to stop as attackers have gotten smarter and faster. For those defending against cyberthreats, things continue to get more complicated. The list of challenges is long: cloud attack surface sprawl, complex application environments, information overload from disparate tools, noise from false positives and low-risk events, just to name a few. The…

Debunking observability myths – Part 6: Observability is about one part of your stack

3 min read - In our blog series, we’ve debunked the following observability myths so far: Part 1: You can skip monitoring and rely solely on logs Part 2: Observability is built exclusively for SREs Part 3: Observability is only relevant and beneficial for large-scale systems or complex architectures Part 4: Observability is always expensive Part 5: You can create an observable system without observability-driven automation Today, we're delving into another misconception about observability—the belief that it's solely applicable to a specific part of your stack or…

Discovery to delivery: Transform the shopper’s journey

3 min read - The relevance of a promise The fundamental principle of commerce is the concept of a “promise.” A promise of accuracy in product detail, product capabilities, quality, price and delivery. The promise of accuracy in inventory—or, more importantly, in the availability of inventory—is very important when there are competing demands for the same unit of inventory and unpredictable disruptions to sources of supply. It becomes very challenging to uphold the promise when so much is changing in the processes that trigger…

Open source large language models: Benefits, risks and types

6 min read - Large language models (LLMs) are foundation models that use artificial intelligence (AI), deep learning and massive data sets, including websites, articles and books, to generate text, translate between languages and write many types of content. There are two types of these generative AI models: proprietary large language models and open source large language models. In this video, Martin Keen briefly explains large language models, how they relate to foundation models, how they work and how they can be used…