With the assistance of AI, researchers have created a technique that produces satellite images forecasting how areas might appear after a flooding incident.
Anticipating the possible effects of a hurricane on homes can aid residents in getting ready and deciding whether to evacuate in advance.
Scientists at MIT have introduced a new method to create future satellite imagery that illustrates how a region could appear after flooding occurs. This approach merges a generative artificial intelligence model with a physics-driven flood model to generate realistic aerial images of an area, highlighting spots likely to be flooded based on the intensity of a looming storm.
For their initial test, the team focused on Houston, generating satellite images that show how various locations in the city might look after a storm similar to Hurricane Harvey, which affected the area in 2017. They compared the created images with real satellite photos of the same areas taken post-Harvey and also assessed AI-generated images that did not incorporate the physics-based flooding model.
The method enhanced by physics provided more accurate and realistic future flooding images compared to the AI-only method, which produced images showing flooding in areas where it was physically impossible.
This approach acts as proof of concept, showcasing how generative AI models can be trusted to produce reliable content when combined with a physics-based framework. To extend this method to other areas affected by future storms, it will need to be trained with a more extensive dataset of satellite images to understand how flooding appears in different locales.
“The goal is that one day, we could use this before a hurricane hits and provide the public with an additional visualization,” says Björn Lütjens, a postdoctoral fellow in MIT’s Department of Earth, Atmospheric and Planetary Sciences. He led the research while pursuing his doctorate in the Department of Aeronautics and Astronautics (AeroAstro). “A significant challenge is motivating people to evacuate when they are in danger. Perhaps this could serve as an extra visual tool to enhance that readiness.”
To demonstrate the capabilities of their new method, named the “Earth Intelligence Engine,” the team has made it accessible online for others to explore.
The researchers published their findings today in the journal IEEE Transactions on Geoscience and Remote Sensing. The study’s co-authors from MIT include Brandon Leschchinskiy; Aruna Sankaranarayanan; and Dava Newman, a professor of AeroAstro and director of the MIT Media Lab, alongside partners from various institutions.
Generative adversarial images
This new study expands on the team’s efforts to utilize generative AI tools to visualize potential climate conditions.
“Offering a hyper-local view of climate appears to be the most effective way to convey our scientific findings,” says Newman, the principal author of the study. “People connect with their own zip codes, their surroundings, where family and friends reside. Local climate simulations become intuitive, personal, and relatable.”
In this research, the authors employed a conditional generative adversarial network, or GAN, which is a machine-learning approach that creates realistic images using two competing neural networks. The first network, known as the “generator,” is trained on actual data pairs, such as satellite images from before and after a hurricane. The second network, called the “discriminator,” is trained to differentiate between real satellite images and those produced by the generator.
Each network improves as it receives feedback from the other. Therefore, the goal of this adversarial interaction is to generate synthetic images that closely resemble real images. However, GANs can still produce “hallucinations,” leading to inaccuracies where details that shouldn’t be present appear in otherwise realistic images.
“Hallucinations can mislead viewers,” explains Lütjens, who began considering whether such inaccuracies could be avoided so that generative AI tools can be dependable in helping to inform individuals, particularly in high-risk scenarios. “We were contemplating how to leverage these generative AI models within a climate-impact context, where having reliable data is crucial.”
Flood hallucinations
In their latest research, the team examined a risk-sensitive situation in which generative AI creates satellite images of future flooding that might be trustworthy enough to guide preparation and potential evacuation processes for at-risk individuals.
Policymakers normally rely on visual aids, like color-coded maps, to get an idea of potential flooding locations. These maps represent the culmination of a series of physical models, starting with a hurricane trajectory model that is integrated with a wind model simulating wind patterns and strengths in a particular region. This information is combined with a flood or storm surge model that predicts how wind may drive water onto land. Ultimately, a hydraulic model determines the likely flooding areas based on local flood management systems and generates a visual, color-coded map of flood heights across the region.
“The question is: Can satellite imagery visualizations offer an additional layer that feels more tangible and emotionally compelling than just a color-coded map while still being dependable?” Lütjens says.
The team first evaluated how well generative AI alone could produce satellite images depicting future flooding. They trained a GAN on real satellite images captured as satellites surveyed Houston before and after Hurricane Harvey. When tasked with generating new flood images for the same regions, the generator created images that resembled conventional satellite images but, upon closer inspection, revealed hallucinations where floods appeared in areas that should not have been flooded (such as higher elevation locations).
To mitigate these inaccuracies and boost the reliability of the AI-created images, the team combined the GAN with a physics-based flood model that takes into account real physical parameters and phenomena, including the hurricane’s path, storm surge, and flood trends. This method, reinforced by physics, allowed the team to generate satellite images around Houston that accurately depicted the same flood extents, pixel for pixel, as predicted by the flood model.
“We demonstrate a practical way to merge machine learning with physics in a situation that requires risk sensitization, necessitating an analysis of Earth’s complex systems to predict future behaviors and potential scenarios aimed at keeping people safe,” added Newman. “We are eager to equip local community decision-makers with our generative AI tools, which could significantly impact and potentially save lives.”
This research received partial support from the MIT Portugal Program, the DAF-MIT Artificial Intelligence Accelerator, NASA, and Google Cloud.
