March 17, 2017 By Andrew Trice 4 min read

How to sharpen Watson Visual Recognition results with simple preprocessing

The IBM Watson Visual Recognition service enables your applications to turn images into actionable data. You can obviously use it to tag images for content, recognize faces, and find similar images, but that’s not all it can do.  You can also create what are known as custom classifiers train the Visual Recognition service to recognize specific content within your images.

How might you use this in a real world case?  Imagine you want to train a system to:

  • Recognize specific structures in satellite images,

  • Spot cracks in pipes,

  • Pinpoint rust or corrosion on infrastructure,

  • Flag known defects in a manufacturing process.

Or any other of the millions of use cases?  Custom classifiers are how you can implement these types of solutions.  You train the service based on collections of images that are both positive and negative of the condition that you want to classify, and then you’ll be able to use it to analyze other images to identify the presence of that condition.  Keep in mind, the quality of your output is directly related to the quality of your training data, so you’ll want to make sure that you have high quality training based on real world sample images.

In many use cases a general/entire-image classification might not be enough.  If the condition you want to identify is only within a smaller region of a larger image, the entire image might not be classified with high enough confidence and a positive result could be missed.  Or, what if you want localized classification – meaning, where within the image can you find the areas recognized by the classifier?

Improving recognition of finer details with image preprocessing

One technique that I have used successfully on several projects is to use slice an image into smaller images, and analyze each one of the images individually against a custom classifier.

This approach lets us analyze each portion of the image independently, and then assemble the results into the bigger picture to provide a view into where within that image there are conditions recognized by your classifier.

Here’s a sample from one insurance-focused solution, which shows areas of hail damage that are recognized on a shingle roof by the Watson Visual Recognition service.

The colorization of the “heat map” is based on the confidence score for each tile images that is coming back in the results of the Watson Visual Recognition service.

Below you can also see a video of another sample, which demonstrates rust detection using Custom Classifiers.  Using image slicing, we can identify where within that image there is the presence of rust.



Try it yourself with this sample code

If you’d like to test this out with your own images and custom classifiers using the Watson Visual recognition service, you’re in luck! The code for this project is available as a learning resource, or for extending into your own solutions at (GitHub).

The sample is a Node.js application that uses the GraphicsMagick module for image preprocessing, submits the tiled images to the Watson Visual Recognition service for analysis, and then display the results within the browser superimposed over the source image.   The colorized squares represent Visual Recognition service’s confidence score for each area.

Confidence Scores

When you’re thinking about this type of scenario, it’s also important to understand confidence scores for custom classifiers returned by the Visual Recognition service.  Many people incorrectly think of confidence scores as a measure of absolute truth; it’s better to think of them as a threshold for action.   For example, consider results with a confidence score of only 60-70%, but the results are consistent, reliable, and do not include false positives or false negatives.  This would be a scenario where a confidence score of 60% is extremely valuable, and you wouldn’t need a 90% confidence score to perform an action.

Confidence scores can be though of much like temperatures in a weather report—I don’t know what the scientific definition of 75 degrees Fahrenheit is, but I know that it’s a pleasant temperature.  Likewise, I don’t know exactly what 50°F means, but I know that I’ll need a jacket.

From a cost-benefit perspective, I know that if I don’t wear a jacket at 50°F, then I am going to be cold.  Likewise, in my custom classifier scenario, I might be able to make a similar type of assumption or assessment. In my workflow, will it cost less if I invoke an action based on a confidence score of 65%?  In many cases, this answer could be yes, and this is where the value of the service comes in.  Of course, confidence score values are subjective.  They vary based upon training images, evaluation images, and the types of criteria that you want to classify. It is up to the developers of each solution to determine what confidences score values are appropriate thresholds for action for a given solution.


With a little bit of preprocessing, your application can improve the detection accuracy of the Watson Visual Recognition service, especially when applied to smaller details within an image. Are you ready to learn more?  Check out these references for more information:

Photo credit: A rusty bridge (

Was this article helpful?

More from

Intesa Sanpaolo and IBM secure digital transactions with fully homomorphic encryption

6 min read - This blog was made possible thanks to contributions from Nicola Bertoli, Sandra Grazia Tedesco, Alessio Di Michelangeli, Omri Soceanu, Akram Bitar, Allon Adir, Salvatore Sollami and Liam Chambers. Intesa Sanpaolo is one of the most trusted and profitable European banks. It offers commercial banking, corporate investment banking, asset management and insurance services. It is the leading bank in Italy with approximately 12 million customers served through its digital and traditional channels. The Cybersecurity Lab of Intesa Sanpaolo (ISP) needed to…

Manage the routing of your observability log and event data 

4 min read - Comprehensive environments include many sources of observable data to be aggregated and then analyzed for infrastructure and app performance management. Connecting and aggregating the data sources to observability tools need to be flexible. Some use cases might require all data to be aggregated into one common location while others have narrowed scope. Optimizing where observability data is processed enables businesses to maximize insights while managing to cost, compliance and data residency objectives.  As announced on 29 March 2024, IBM Cloud® released its next-gen observability…

IBM unveils Cloud Pak for Data 5.0

7 min read - Today’s modern technology landscape is experiencing an explosion of data. Organizations need to be able to trust and access this data to generate meaningful insights. Enter IBM Cloud Pak® for Data 5.0, the newest release of the cloud-native insight platform that integrates the tools needed to collect, organize and analyze data within a data fabric architecture. IBM Cloud Pak for Data 5.0 enhances users’ data strategies by including these new features Immersive Experience: Customers can now streamline their IT and day 2 operations with…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters