Guest blogger: Zoya Yeprem, Senior Technical Solution Specialist Intern
IBM Summit Program, IBM Federal
Computer vision is a technology that acquires, processes and analyzes images, and can automate what human visual analysis can perform. And with recent advances in this field, we are witnessing its use in our personal and professional lives more than ever. Such systems are heavily dependent on highly complex deep learning models called Deep Neural Networks (DNN). These models are capable of successfully performing complex tasks like object detection, image classification, segmentation, etc. without being explicitly programmed.
Numerous applications and systems that we will be using in the future will be using deep learning models behind the scenes to perform high-level cognitive tasks e.g. self-driving cars. In addition to the day-to-day use of such systems, computer vision can be extremely useful for government agencies. It can help agencies categorize and analyze images and gain high level cognitive insights automatically in real time. For instance, the surveillance system in airports can be integrated with computer vision systems that can help detect any abnormal activity that may be considered a threat. Also, when the threat is detected, it can help track down the person responsible by identifying other sightings of similar looking individuals. Another use case is in geospatial technologies. Geospatial imagery encompasses a wide range of graphical products that convey information about natural phenomena and human activities occurring on Earth's surface. This technology uses computer vision to provide complex insights in real time, providing crucial information to humanitarian and disaster relief agencies where accuracy and timeliness are top priorities.
While deep learning models used in computer vision systems are normally very accurate, they are vulnerable to special attacks that use adversarial examples. Adversarial examples are input images that have a carefully crafted noise added to them. While these images appear identical to the originals, they are completely misclassified by the DNN. A simple example can be found below which demonstrates how a stop sign image is misclassified as an “Ahead Only” sign when certain noise is added to it.
Adversarial attacks pose a real threat to the deployment of AI systems in security critical applications. Virtually undetectable manipulations of images, video, speech, and other data have been crafted to confuse these systems. Such manipulations can be crafted even if the attacker doesn’t have exact knowledge of the architecture of the deep learning model or access to its parameters (Black-Box attacks). Even more worrisome, adversarial attacks can be launched in the physical world: researchers have proven that instead of manipulating the pixels of a digital image, adversaries could defeat visual recognition systems in autonomous vehicles by sticking patches to traffic signs, or they can fool facial recognition systems by wearing specially designed glasses. Therefore, it is crucial to protect our deep learning models against such attacks.
IBM Research in Ireland has released the Adversarial Robustness Toolbox (ART) that provides protection against adversarial attacks on DNNs. ART is an open source library written in python that supports most popular deep learning frameworks such as: TensorFlow, Keras, PyTorch, etc. ART can provide protection to a DNN in three stages:
First, we check to see if the DNN model is vulnerable against adversarial attacks as not all DNNs are vulnerable. ART has implementations of state-of-the-art attacks, which can be used to craft an adversarial image and feed it to the DNN. Then, by recording the loss of accuracy on adversarially altered inputs, you can detect how vulnerable your model is to that specific attack. Other approaches measure how much the internal representations and the output of a DNN vary when small changes are applied to its inputs.
Second, after confirming vulnerability of a certain type of attack, a given DNN can be “hardened” to make it more robust against adversarial inputs. Common approaches are to preprocess the inputs of a DNN, to augment the training data with adversarial examples, or to change the DNN architecture to prevent adversarial signals from propagating through the internal representation layers.
Finally, runtime detection methods can be applied to flag any inputs that an adversary might have tempered with. During this stage, ART can somewhat act like an antivirus application where it checks the inputs and flags the one that are adversaries to protect the DNN.
In conclusion, any new technology comes with strengths and weaknesses. Take E-mail technology for instance; it provided fast and convenient way of communication and reduced the need of hard copied documents dramatically. However, in the beginning, users were extremely vulnerable to different worms and viruses spread across mailboxes. But through several years of using them, we learned how to mitigate those vulnerabilities while enhancing its positive capabilities. Same goes with visual recognition technology. no one can deny all the goods that these systems have brought to us but to embrace it, it’s crucial to first: find possible vulnerabilities and second: have tools to protect our system against adversaries, and this is exactly where Adversarial Robustness Toolbox can help.
ART in Action
Open source demo can be found here: ART Demo.
This implementation contains attack and defense against a model trained on GermanTrafficSign dataset. Full documentation on each step of the implementation is included in the notebook file.
ART open source library
To install ART and start using it, check out the open-source release under Adversarial Robustness Toolbox .The release includes extensive documentation and tutorials to help researchers and developers get started.
 Sharif et al. 2016, “Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition”
 Eykholt, et al. 2018, “Robust Physical-World Attacks on Deep Learning Visual Classification” arXiv:1707.08945v5 [cs.CR]
You can contact the author at firstname.lastname@example.org