Using Machine Learning &

Deep Learning With ArcGIS

Imagery

KANNAN Jayaraman
gistec
Recent Innovation
ArcGIS as a Platform
for Deep Learning
 ArcGIS is a Comprehensive Imagery Platform
System of
Engagement

System of
Insight To share imagery products and
information to those that need it
Professional
Imagery & Geospatial
Analysts System of
Record

To extract Information
from Imagery Server

To manage and
Esri also have multiple partners to provide best of breed solutions process all your imagery
Analysis
Extracting Information from Imagery
ArcGIS Pro
Core capabilities
Raster Functions

ArcGIS Spatial Analyst

Raster Functions
Distance modelling
Hydrological modelling

ArcGIS Image Analyst

Raster Functions
Deep Learning
Pixel Editor
Multidimensional data
Stereo
Image Space
Full Motion Video

ArcGIS Image Server

Raster Functions
Scaling using Raster Analytics

ArcGIS API for Python

Recent Innovation
Recent Innovation
Artificial Intelligence
• Artificial Intelligence Natural
Video Game
• AI is the ability of computers to perform Behavioral AI
Language
a task that typically requires some level Processing
of human intelligence. Speech
Computer
Vision
Machine Recognition

• Machine Learning Learning

• Type of engine that uses data driven Random Forest
algorithms to learn from data to give Logistic Regression
users the answers that’s required.
Scikit-learn
SVM
• Deep Learning Deep
• Type of machine learning, and it Learning
refers to deep neural networks, that are
inspired from and loosely resemble the Convolution
Neural Networks
human brain
 Caffe Object
Artificial Intelligence
Detection
Object Tracking
T-SNE
Support Vector Machines
Natural Language
Processing
Machine Neural Networks
Cognitive Learning Computer Vision
Computing Theano

CNTK
GeoAI Deep
Learning TensorFlow
ArcGIS Includes Machine Learning Tools

Classification

Prediction

ArcGIS
Clustering Deep Learning
Machine Learning Tools in ArcGIS

Classification Deep Learning

• Pixel & Object Based • Generate training samples
• Image Segmentation • Detect objects
• Maximum Likelihood • Classify pixels
• Random Trees
• Support Vector Machine

Clustering
Prediction
• Spatially Constrained Multivariate
Clustering • Empirical Bayesian Kriging
• Multivariate Clustering • Areal Interpolation
• Density-based Clustering • EBK Regression Prediction
• Hot Spot Analysis • Ordinary Least Squares Regression and
• Cluster and Outlier Analysis Exploratory Regression
• Space Time Pattern Mining • Geographically Weighted Regression
Deep Learning with Imagery in ArcGIS
ArcGIS supports end-to-end deep learning workflows

• Tools for:
• Labeling training samples
• Preparing data to train models
• Training Models
• Running Inferencing

• Supports all 4 imagery deep learning categories

• Supports image space, leverage GPU

• Clients
• ArcGIS Pro
• Map Viewer Part of ArcGIS Image Analyst
• Notebooks Run distributed on ArcGIS Image Server
Deep Learning
Key imagery tasks for deep learning

Impervious Surface Palm Tree Building Footprint Damaged House

Classification Detection Extraction Classification

Pixel Classification Object Detection Instance Segmentation Image Classification

 ArcGIS – Machine Learning Workflow
Detailed Workflow

Image (Data) Scientist ArcGIS User

ArcGIS Professional

Machine
Machine Learning
Imagery Pixel & Segment Based
Learning
Generate Training Inference
results
Samples
Deep Learning Based Deep
Deep Learning
Learning
Training sites
Training Tools Training Engine Model
Definition

Feedback Loop
Input Images

Pixel & Segment Based: Machine Learning:

- Maximum Likelihood - Support Vector Machine
- Support Vector Machine - Random Forest
- Random Forest Deep Learning:
Deep Learning Based: - TensorFlow*
- TensorFlow* - CNTK*
- CNTK* - PyTorch*
- PyTorch* - Custom*
- Custom* - + External via Python
*Run External to ArcGIS *Requires framework installed
Deep Learning Process
 Samples as Image Chips

Image as viewed in ArcGIS Pro

Density of Sample collection

 SAMPLE IMAGE CHIP CREATION
Samples at multiple locations Samples exported for Deep Learning
 • Support for Deep Learning out of the box

Detect Objects Classify Pixels

Object Detection API DeepLabs
Tensor Flow
Mask RCNN
Keras
fast.ai - SSD
PyTorch
Faster RCNN by Microsoft U-Net by
CNTK Microsoft Azure

Tensor Flow: end-to-end open source platform for machine learning

Keras: Open-source neural-network library written in Python. Capable of running on top of TensorFlow, CNTK and other platform
PyTorch: Open source machine learning library used for computer vision and natural language processing, primarily developed by FB AI Research Lab
CNTK: Microsoft Cognitive Toolkit, previously known as CNTK - a deep learning framework developed by Microsoft Research
Inference Tools

• Classify Pixels using Deep Learning

Runs the model on an input raster to produce classified raster,
each valid pixel has an assigned class label

• Built in Python Raster Function for TensorFlow and CNTK

• Mini-batch support
• Processor type: CPU or GPU
• Parallel Processing in ArcGIS Pro
• Distributed raster analysis on Enterprise
 Inference Tools
• Detect Objects Using Deep Learning
Runs the model on an input raster to produce feature class
containing the object it finds

• Built in Python Raster Function for TensorFlow, Keras,

PyTorch and CNTK
• Mini-batch support
• Optional: Non-Maximum Suppression
• Processor type: CPU or GPU
• Parallel Processing in ArcGIS Pro
• Distributed raster analysis on Enterprise
Object Detection using Deep Learning – Sample Output
 Current
Deep Learning in ArcGIS API for Python Version
Make model training easier using arcgis.learn module
1.7.0

The arcgis.learn module is based on PyTorch and fast.ai and enables fine-tuning of pretrained torchvision
models on satellite imagery

• Prepare Training Data • Model Management

• arcgis.learn.prepare_data • arcgis.learn.list_models
• arcgis.learn.Model
• Model.install
• Export Training Data • Model.Uninstall
• arcgis.learn.export_training_data • Model.query_info

• Train Models • Run Inference at SCALE

• arcgis.learn.SingleShotDetector • arcgis.learn.detect_objects
• arcgis.learn.UnetClassifier • arcgis.learn.classify_pixels
• arcgis.learn.FeatureClassifier
 Take Away

• Easy way to extend Deep Learning capabilities to any support Framework/Model configuration
• Out of the box Support for most common Deep Learning Frameworks
• Leverage the powerful Raster Analytics capabilities to distribute model inference tasks.
• Users can easily train the model with ArcGIS API for Python
Thank you!

Kannan Jayaraman
kannan.jram@gistec.com