Using MMSegmentation with arcgis.learn

Introduction

MMSegmentation, a part of OpenMMLab, is an open-source semantic segmentation toolbox based on PyTorch. arcgis.learn provides the MMSegmentation class which acts as a bridge to train and use the models in OpenMMLab's MMSegmentation toolbox in ArcGIS. The MMSegmentation class of arcgis.learn allows you to train these models using the familiar arcgis.learn API for data preparation, model definition and training. The trained models can then be used in ArcGIS Pro, Enterprise and Online.

Land Cover Classification of San Francisco Bay, California, USA

Setting up the environment

Follow the steps here to install deep learning dependencies in ArcGIS Pro or Anaconda environment respectively.

ArcGIS Pro 2.8 users additionally need to install CUDA toolkit version 11, mmcv-full and mmsegmentation libraries. Follow these steps to do so:

Download and install the latest CUDA toolkit version from here.
Add the installed CUDA toolkit's bin folder path (typically, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin) to the (user or system) Path Environment Variables.
Run the following command in a cloned environment:

conda install -c esri mmcv-full mmsegmentation

Implementation in `arcgis.learn`

With the basic setup done, we are now ready to define a model supported in MMSegmentation. arcgis.learn allows us to use any of the supported models from OpenMMLab's MMSegmentation toolkit through a single line of code. For example:

model = arcgis.learn.MMSegmentation(data, model='resnest')

The parameters required to be passed are:

data is the data object prepared using prepare_data function.
model is name of one of the models from the list of supported models.

The following MMSegmentation models are supported through arcgis.learn:

MMSegmentation.supported_models

['ann',
 'apcnet',
 'ccnet',
 'cgnet',
 'deeplabv3',
 'deeplabv3plus',
 'dmnet',
 'dnlnet',
 'emanet',
 'fastscnn',
 'fcn',
 'gcnet',
 'hrnet',
 'mobilenet_v2',
 'nonlocal_net',
 'ocrnet',
 'psanet',
 'pspnet',
 'resnest',
 'sem_fpn',
 'unet',
 'upernet']

For more information about the API, visit the API reference for MMSegmentation.

Model inferencing in ArcGIS Pro

Trained MMSegmentation models can be used for inferencing using the Classify Pixels Using Deep Learning tool in ArcGIS Pro.

References

[1] OpenMmlab, “openmmlab/mmsegmentation: OpenMMLab Semantic Segmentation Toolbox and Benchmark.,” GitHub. [Online]. Available: https://github.com/open-mmlab/mmsegmentation. [Accessed: 22-Jul-2021].