How to sharpen Watson Visual Recognition results with simple preprocessing

The IBM Watson Visual Recognition service enables your applications to turn images into actionable data. You can obviously use it to tag images for content, recognize faces, and find similar images, but that’s not all it can do.  You can also create what are known as custom classifiers train the Visual Recognition service to recognize specific content within your images.

How might you use this in a real world case?  Imagine you want to train a system to:

  • Recognize specific structures in satellite images,

  • Spot cracks in pipes,

  • Pinpoint rust or corrosion on infrastructure,

  • Flag known defects in a manufacturing process.

Or any other of the millions of use cases?  Custom classifiers are how you can implement these types of solutions.  You train the service based on collections of images that are both positive and negative of the condition that you want to classify, and then you’ll be able to use it to analyze other images to identify the presence of that condition.  Keep in mind, the quality of your output is directly related to the quality of your training data, so you’ll want to make sure that you have high quality training based on real world sample images.

In many use cases a general/entire-image classification might not be enough.  If the condition you want to identify is only within a smaller region of a larger image, the entire image might not be classified with high enough confidence and a positive result could be missed.  Or, what if you want localized classification – meaning, where within the image can you find the areas recognized by the classifier?

Improving recognition of finer details with image preprocessing

One technique that I have used successfully on several projects is to use slice an image into smaller images, and analyze each one of the images individually against a custom classifier.

This approach lets us analyze each portion of the image independently, and then assemble the results into the bigger picture to provide a view into where within that image there are conditions recognized by your classifier.

Here’s a sample from one insurance-focused solution, which shows areas of hail damage that are recognized on a shingle roof by the Watson Visual Recognition service.

The colorization of the “heat map” is based on the confidence score for each tile images that is coming back in the results of the Watson Visual Recognition service.

Below you can also see a video of another sample, which demonstrates rust detection using Custom Classifiers.  Using image slicing, we can identify where within that image there is the presence of rust.


IBM Watson Visual Recognition using image preprocessing


IBM Watson Visual Recognition using image preprocessing


Try it yourself with this sample code

If you’d like to test this out with your own images and custom classifiers using the Watson Visual recognition service, you’re in luck! The code for this project is available as a learning resource, or for extending into your own solutions at (GitHub).

The sample is a Node.js application that uses the GraphicsMagick module for image preprocessing, submits the tiled images to the Watson Visual Recognition service for analysis, and then display the results within the browser superimposed over the source image.   The colorized squares represent Visual Recognition service’s confidence score for each area.

Confidence Scores

When you’re thinking about this type of scenario, it’s also important to understand confidence scores for custom classifiers returned by the Visual Recognition service.  Many people incorrectly think of confidence scores as a measure of absolute truth; it’s better to think of them as a threshold for action.   For example, consider results with a confidence score of only 60-70%, but the results are consistent, reliable, and do not include false positives or false negatives.  This would be a scenario where a confidence score of 60% is extremely valuable, and you wouldn’t need a 90% confidence score to perform an action.

Confidence scores can be though of much like temperatures in a weather report—I don’t know what the scientific definition of 75 degrees Fahrenheit is, but I know that it’s a pleasant temperature.  Likewise, I don’t know exactly what 50°F means, but I know that I’ll need a jacket.

From a cost-benefit perspective, I know that if I don’t wear a jacket at 50°F, then I am going to be cold.  Likewise, in my custom classifier scenario, I might be able to make a similar type of assumption or assessment. In my workflow, will it cost less if I invoke an action based on a confidence score of 65%?  In many cases, this answer could be yes, and this is where the value of the service comes in.  Of course, confidence score values are subjective.  They vary based upon training images, evaluation images, and the types of criteria that you want to classify. It is up to the developers of each solution to determine what confidences score values are appropriate thresholds for action for a given solution.


With a little bit of preprocessing, your application can improve the detection accuracy of the Watson Visual Recognition service, especially when applied to smaller details within an image. Are you ready to learn more?  Check out these references for more information:

Photo credit: A rusty bridge (

More from Cloud

Strengthening cybersecurity in life sciences with IBM and AWS

7 min read - Cloud is transforming the way life sciences organizations are doing business. Cloud computing offers the potential to redefine and personalize customer relationships, transform and optimize operations, improve governance and transparency, and expand business agility and capability. Leading life science companies are leveraging cloud for innovation around operational, revenue and business models. According to a report on mapping the cloud maturity curve from the EIU, 48% of industry executives said cloud has improved data access, analysis and utilization, 45% say cloud…

7 min read

Kubernetes version 1.27 now available in IBM Cloud Kubernetes Service

< 1 min read - We are excited to announce the availability of Kubernetes version 1.27 for your clusters that are running in IBM Cloud Kubernetes Service. This is our 22nd release of Kubernetes. With our Kubernetes service, you can easily upgrade your clusters without the need for deep Kubernetes knowledge. When you deploy new clusters, the default Kubernetes version remains 1.25 (soon to be 1.26); you can also choose to immediately deploy version 1.27. Learn more about deploying clusters here. Kubernetes version 1.27 In…

< 1 min read

Redefining the consumer experience: Diageo partners with SAP and IBM on global digital transformation

3 min read - In an era of evolving consumer preferences and economic uncertainties, the beverage industry stands as a vibrant reflection of changing trends and shifting priorities. Despite the challenges posed by inflation and the cost-of-living crisis, a dichotomy has emerged in consumer behavior, where individuals untouched by the crisis continue to indulge in their favorite beverages, while those directly affected pivot towards more affordable luxuries, such as a bottle of something special. This intriguing juxtaposition highlights the resilient nature of consumers and…

3 min read

IBM Cloud releases 2023 IBM Cloud for Financial Services Agreed-Upon Procedures (AUP) Report

2 min read - IBM Cloud completed its 2023 independent review of IBM Cloud services and processes. The review report demonstrates to its clients, partners and other interested parties that IBM Cloud services have implemented and adhere to the technical, administrative and physical control requirements of IBM Cloud Framework for Financial Services. What is the IBM Cloud Framework for Financial Services? IBM Cloud for Financial Services® is designed to build trust and enable a transparent public cloud ecosystem with features for security, compliance and…

2 min read