Skip to content
Andrei Nita8/29/24 12:49 PM6 min read

When MIS Met ML : Breaking Down How We Built an AI Damage Classifier to Enhance Assessments

The subject of Artificial Intelligence has permeated many major political, cultural, and technological conversations over the last few years. At this point, there is little doubt that in one form or another, AI is here to stay. 

Whilst a measured approach from technology providers is necessary to find the best way to harness AI to support the insurance industry, there is a major market opportunity for this technology to be integrated with industry practices and generate significant value for insurers and Insurtech's alike. 

In 2023, the AI in insurance market was valued at $5bn, which is astronomical compared to other key industries, given that AI in law was valued at $69m and AI in finance at $1.09bn.  

What’s even more striking is that according to market.us, the AI in insurance market has a projected compound annual growth rate (CAGR) of 32.7% between 2024-2033, which will see the market valued at a whopping $91bn in just nine years' time. 

It was also found that in 2023, 76% of insurance executives felt that pressure to innovate was at an all-time high, especially with very public advances in AI technology and the growing need for operational efficiency and data-driven decision making. 

Currently, the AI in insurance market is driven mostly by a focus on machine learning (ML), which captures 45% of the market share. This is where MIS comes in. 

Back in June, we announced that we had added AI functionality to the GEO platform, which came in the form of a building damage classifier to enhance and support our human-led damage assessments following catastrophic events. 

Objective 

The goal for this building damage classifier was to harness the accuracy and reliability of our team of in-house military/NATO-trained intelligence analysts, then subsequently combine it with the speed and processing power of AI.  

In doing this, it would cut down the time it takes to assess affected locations post-event, whilst also being able to cover more locations across a larger area in that same timeframe. This would provide the team with powerful additional resources when making assessments of multiple ongoing events or tackling major catastrophes. 

Ultimately, this was designed to supplement our human-led damage assessments rather than as a means of replacing it, and there will always be a human in-the-loop to ensure that the AI tool is delivering accurate results that are reflective of the MIS Intelligence team’s very high standards. 

Approach 

To develop such a tool, it was clear that following a machine learning approach was the right choice, given that the nature of our human-led damage assessments is rooted in processing a vast amount of imagery and datasets following an event.  

As these traditional methods for assessments would serve as the ideal way for the AI classifier to learn how to replicate the accuracy of human analysts, we therefore trained the model on an extensive amount of real-life data based on previous damage assessments made by the MIS Intelligence team. 

Applying an ensemble computer vision method towards machine learning, this training material comprised of a combination of satellite/aerial imagery and building footprint data from events that MIS had responded to, with the relevant Intelligence team building damage assessments overlaid. 

The MIS Technology team continuously feeds this data into the ML model, so that it can quickly process the information and piece together an understanding of the relationship between building footprints and damage assessments to provide the relevant output.

A brilliant example of how fast the model can make assessments of a significant number of buildings is that there were instances where it assessed ~20,000 properties in just 30 minutes.

Finally, the last step would then be to correlate the results with the real data to ensure that the assessment is accurate. 

Process 

The development process here was defined by the classic iterative approach of trial and error.  

To determine how to get the best results out of the AI damage classifier, multiple image types from both satellite and aerial imagery were tested to ascertain which was the most effective, relative to the AI classifier’s ability to accurately label properties with the correct damage assessments.  

Once the ideal image type was identified, that narrowed the focus of the model’s development towards getting the best outputs. 

However, there were several other critical factors which had to be considered first if the AI damage classifier was to succeed. 

The most fundamental part of this was teaching it how to be a classifier, ergo, how to define classes of damage.

Typically, the industry applies 3 classes to damage assessments: no damage, damaged, or destroyed. MIS doubles this to provide insurers with greater detail around individual property damage levels, with 6 damage classes: no damage, light damage, moderate damage, substantial damage, severe damage and destroyed. 

In order for the AI classifier to be able to distinguish between the 6 classes, it requires multiple models, which would make classifications, further refine them, then finalise the results for the subsequent output.

Ultimately, determining whether the outcome of this was successful would be a measure of how accurate those damage assessments were. Through trial and error, the team are able to ensure that these factors are accounted for when running the next iteration to try and improve results. 

Using 7.5cm aerial imagery provided by Fugro, the image below depicts how the model is trained to accurately reflect human assessments during it's development, showing a side-by-side example of an MIS Intelligence team assessment (left) and the AI damage classifier assessment (right) of an area in Perry, Florida following Hurricane Idalia.

Results 

These factors influencing results involved several key aspects, such as proximity to the coast to understand potential water damage and wind speeds in each area to explore the potential for wind damage, however there was one which really stood out. 

This was the idea of nearest neighbour. This applied the assumption that typically, building damage tends to cluster, which the model could be taught. 

Whilst there are exceptions, i.e. there could be a building with light damage next to a group of buildings with no damage, it did mean that by and large, accuracy levels when predicting either the same or adjacent damage class compared to the Intelligence team assessments was averaging at around 95%, which so early in the AI classifier’s development, is excellent. 

With the classifier already showing high levels of accuracy relative to the abilities of our own in-house intelligence experts to make damage assessments, our exhaustive, considered approach to AI is already showing major promise. 

Using 15cm aerial imagery provided by NOAA to showcase the accuracy of the model, below is a side-by-side example of an MIS Intelligence team assessment (left) and the AI damage classifier assessment (right) of an area in Fort Myers, Florida following Hurricane Ian.

ian sbs ai

Future 

What’s notable about this AI damage classifier is that the improvement opportunities are infinite, given that we are constantly feeding new data from damage assessments into the machine learning models. 

With every new event MIS responds to, this is more data for the model to learn from in terms of both quantity and quality, which provides the ideal foundation for its continued development. 

In its current stage, it is vital that we already recognise where the AI damage classifier is best suited to supporting our Intelligence team when it comes to event response. 

It is important that we continue to focus the application of the model on these tasks, like dealing with large scale events in a short time frame, which helps the MIS intelligence analysts pour their skills and focus into the areas that matter the most to our clients.

Regardless of how it improves in the future and how effective it becomes, the need for a human will always remain to verify the findings, it just means our Intelligence team will benefit from further support as MIS continues to help insurers enhance their event response processes. 

If you're interested in harnessing the powerful combination of expert human-led damage assessments and the speed of AI, make sure to contact the MIS team.

RELATED ARTICLES