Big Imagery Data Visualization with Gradient-weighted Class Activation Maps

Create saliency maps in tandem with convolutional neural networks, enabled by PyTorch

Thomas Chen

Algorithms Computer Vision Deep Learning Machine-Learning

See in schedule

Machine learning and artificial intelligence have enabled many solutions that tackle real-world problems, from climate change to autonomous vehicle development. However, with any new technology comes novel issues to address, and in the consideration of machine learning, a recurring concern is the interpretability of the models. Particularly, this refers to the ability of humans, and especially end users of the technology, to understand how the model came to its decision. One way of opening up “black box models” (algorithms that are relatively uninterpretable) that are trained on imagery (computer vision models) is to create saliency maps that are qualitative, visual representations of which parts of each image had the most influence in the deep learning model’s prediction for that image. In this talk, we discuss one interesting way of achieving this. After training a convolutional neural network (CNN) using PyTorch on an image dataset (which can be anything, such as ImageNet), we explore using the GradCam repository to create gradient-weighted class activation maps, which harness the CNN and uses the gradients of any target concept, flowing into the final convolutional layer to produce a coarse localization map highlighting important regions in the image for predicting the concept. It is recommended that attendees of this talk are familiar with at least the basics of machine learning, deep learning, and computer vision, but the presentation will not be too technical as to encourage less experienced folks.

Type: Talk (30 mins); Python level: Intermediate; Domain level: Intermediate


Thomas Chen

Thomas Chen is a research scientist whose primary interests lie in machine learning and computer vision. He serves on the U.S. Technology Policy Committee of the Association for Computing Machinery. As much of his work lies at the nexus of artificial intelligence and earth science, he is also an active early-career scientist member of the European Geosciences Union and the American Geophysical Union. Previously, Thomas has presented work at a number of conferences, workshops, and meetings, from NeurIPS workshops, to AAAI, Applied Machine Learning Days, the Open Data Science Conference, Machine Learning Week Europe, the UK Antarctic Science Conference, and much more.