The constant evolution of AI (Artificial Intelligence) encourages many to believe it may accelerate our progress towards a sustainable and resilient future. After delving into the ways AI is supporting geospatial analysis in ArcGIS, I’m inclined to agree. What you may not know is that AI technology is already working behind the scenes across the ArcGIS suite, quietly improving functionality and efficiency without us realising it…which can probably be said for most of your day-to-day applications!
Today in ArcGIS, AI takes two forms: Generative AI and GeoAI. Generative AI aims to recognise the intent behind a user query, while GeoAI helps us to automate data extraction at scale. This blog will focus on GeoAI, how we can harness it in ArcGIS, and the opportunities it presents for smarter spatial analysis. Whether you’re new to GeoAI like me or just curious to learn more, I will guide you through the basics.
GeoAI is the integration of AI capabilities with Geographic Information Systems. Its purpose is to assist and speed up the extraction of meaningful data. This enables us to keep track of ever-changing spatial patterns, visualise flows of information and better understand real-world problems, all while streamlining our workflows. It may sound daunting, but approaching GeoAI – regardless of your experience, profession or location – has never been easier.
Let’s get into it!
Taking advantage of GeoAI in ArcGIS
GeoAI is composed of Machine Learning and Deep Learning. Machine learning involves training a computer with data-driven algorithms, so that it can recognise patterns in geospatial data and predict outcomes. Deep Learning is a subset of Machine Learning, which uses a human brain-like approach to extract progressively more complex features from geospatial data. If you’re looking for an effective approach to data extraction, you’ve come to the right place.
At its most accessible in ArcGIS, GeoAI forms the basis of a library of pretrained deep learning packages within the ArcGIS Living Atlas of the World. These models are designed to assist Creator (or higher) user types perform tasks relating to a variety of sectors – such as business, insurance and environment. These packages promote smarter spatial analysis through assisted feature extraction, image redaction, object tracking and pixel classification.
Comparing different forms of GeoAI
Do you find yourself needing to automatically detect features from imagery? Wind turbines or power lines? Or maybe a feature of your own choosing? Well, GeoAI could ultimately replace the tiresome and inefficient process of manual digitisation… for good.
I know, this would have been useful for my dissertation too – but new GeoAI capabilities are available all the time!
With ArcGIS, you can get started using a straightforward pretrained model package, extract or classify features of your choice with deep learning, or even build your own unique GeoAI model using Esri tools. While these options are available for Creator, Professional and Professional Plus users, different options require different user type levels – so remember to double check the documentation before you try them out!
Let’s explore these some more…
For the curious
The first option requires little GeoAI experience and possesses minimal computational difficulty, by using pre-trained models that have been tried and tested. These models are accessible via the Living Atlas and are trained to perform specific tasks to solve problems. Examples include building footprint extraction, wind turbine detection, and agricultural field delineation. A deeper dive into using these models can be found in our Machine Learning Made Easy blog.
Note: Some models have been trained on US data (such as cars or buildings) and are therefore accustomed to picking up different shapes. These models are still worth experimenting with outside of the US, but you may want to double check their outputs.
For the confident
The second option requires background knowledge of GIS and is slightly more computationally involved. It allows users to refine pre-trained models, offering you control over feature extraction or classification – a suitable option if your goal is not defined by one of the pre-trained packages. One example is the Text Segment Anything Model (or Text SAM) which does what it says on the tin, allowing you to prompt the model to identify specific objects using free-form text. Another example is the CLIP Zero-Shot Classifier, which can be used to assign an image (or even a folder full of images) to a class based on names and descriptions that you provide. By allowing the user to define custom categories, this unique model is particularly useful for environmental analysis and disaster response.
Note: These models can also be found in the Living Atlas but may require more specific parameters regarding imagery and extensions.
For the GeoAI pro
The final option is for those familiar with incorporating AI into ArcGIS, looking to develop their interest in deep learning and apply its capabilities within their sector. This process involves tailoring deep learning models with your own training data, shaping the model to suit your Area of Interest (AOI), resulting in the most accurate and consistent outputs. One example is the Land Cover Classification (Sentinel-2) package, which can be fine-tuned to suit your own geographic imagery using the Train Deep Learning Model tool in Pro.
Find out more in the Fine-Tune a Pretrained Deep Learning Model blog.
Getting started
I’m going to step through some examples I have experimented with using one of our ‘intermediate’ GeoAI models – Text SAM. This package can be found and downloaded from the Living Atlas here.
Ensure that the supported deep learning frameworks for ArcGIS are installed before using the model. Remember to check that you have the required licensing: in this case ArcGIS Image Analyst and ArcGIS 3D Analyst toolbox extensions for ArcGIS Pro.
A step-by-step guide to using this model can be found on our Use the model page, but below are some tips and tricks I learnt while extracting my own objects from some sample Bluesky aerial imagery.
If you (like me) are new to running deep learning models in Pro, zooming in on your chosen features and then selecting Current Display Extent under Environments > Processing Extent helps to reduce model run times and exclude unnecessary features that may confuse the model.
My experience…
With no prior experience of deep learning packages, I managed to extract six different features with ease using my downloaded TextSAM.dlpk as the model definition for the Detect Objects Using Deep Learning tool.
→ As you can see, the model has returned mostly accurate responses – even ‘ships’ on the bank have been identified! Keep in mind that using generalised terms such as ship instead of boat may return more accurate results.
→ While there is an existing pre-trained model for Car Detection, using Text SAM has allowed me to specify the colour of the car extracted. Be aware that your prompt doesn’t need to be plural to extract more than one object!
→ The Swimming pool prompt has been highly successful, while rectangular objects has produced less accurate results, with some shadows being extracted instead of the objects themselves due to shape similarities. The clearer the imagery, the better the model will perform.
Ultimately, this model is designed for users who want to quickly and effectively extract specific features from imagery, without extensive understanding of GeoAI. Now it’s your turn to give Text SAM a try! What features can you extract?
Looking to the future
I know what you’re thinking… so what’s the catch?
Well, GeoAI is still a developing tool, meaning that its performance is not flawless. Results may require the reinforcement of secondary validation while its accuracy increases. This is due to a variety of factors, including varied quality of geospatial data, computational complexity, and model limitations such as machine assumptions.
The GeoAI roadmap is a long-term one. However, constant accumulation of training data for these models to learn from, coupled with increased training times, facilitates consistent progress in the GeoAI field. Researchers are confident that in the not-too-distant future, GeoAI will be capable of contributing to vital areas of research such as resource allocation and monitoring of real-time environmental hazards. Check out the Geospatial AI Models for Disaster Response blog to find out more about how GeoAI can help identify and respond to critical threats.
To summarise in the words of Mark Wells, CTO at Esri UK: GeoAI helps us do things smarter.
We can’t let GeoAI pass us by!
If you’re looking to get stuck into GeoAI, check out some of the resources below.
- GeoAI help page – for more key concepts
- Capabilities page – for an overview of incorporating GeoAI into your workflows
- Customer Stories – for real-world stories
- All pre-trained models can be found in the Living Atlas