Deep Learning and Remote Sensing for Coastal Resilience: Rapid and Automated Coastal Monitoring to Inform Community Recovery from Storm Events (2019-2020)
How can we quickly and effectively assess the effects of storms on our coastal environment? And how can recent technological developments in satellite remote sensing and machine learning support such assessments?
Remote sensing plays a critical role in the study and management of coastal ecosystems. Satellite imagery permits accurate land cover mapping at medium resolution (<5 m), and with the proliferation of cubesats (small satellites, roughly the size of a shoebox), this data is becoming available at weekly and even daily time steps on a global scale.
Unoccupied aerial systems (UAS, also known as drones) complement satellites and are becoming an integral tool in studying and managing coastal systems, providing on-demand remote sensing capabilities at low cost and with modular sensors that can assess a wide variety of variables. In coastal systems, small UAS can collect ultra-high resolution (<0.05 m) multispectral imagery and generate high fidelity 3D models of the landscape.
While powerful tools, satellites and UAS generate massive amounts of data. The expertise and time required to analyze this imagery is an obstacle in using it to inform disaster response and storm recovery – an obstacle machine learning is well poised to mitigate.
The purpose of this project is to use Hurricane Florence coastal impacts and recovery to develop a methodology for rapid and high-resolution monitoring of North Carolina’s coastline. We will do so by leveraging convolution neural networks (CNNs) to automate change detection in satellite imagery. The change detected via satellite will trigger a time-series of UAS surveys for areas that experienced substantial change.
CNNs are a popular deep learning model inspired by the neural connections in the human brain and visual cortex. CNNs have been extremely successful in analyzing imagery and are a promising method for automating the analysis of remote sensing data. At their simplest, CNNs transform complex input data into more useful output data (e.g. transforming satellite imagery into a habitat map).
Our team will build on related projects using deep learning for remote sensing data analysis and implement a CNN to create automated habitat classification and change detection maps of both satellite and UAS imagery, using the satellite-based change map to inform UAS survey locations. This work will lead to a tool capable of creating accurate habitat maps and detecting ecosystem change from multiple remote sensing platforms.
Built in coordination with local collaborators, our tool will demonstrate a semi-autonomous system capable of monitoring North Carolina’s coast at a scale relevant for statewide management yet sensitive to local storm responses. These tool and map products will be shared with our partners and online through an Environment Systems Research Institute (ESRI) story map.
Ultimately, faster detection of coastal change in the face of more frequent intense storms will facilitate disaster response, conservation management and coastal resilience – aiding communities in immediate recovery and planning long-term responses to climate change-driven threats.
Habitat classification and change detection satellite mapping tool; publication in peer-reviewed journal; foundation for future research and grants to explore using a coordinated satellite and drone system to monitor and evaluate ecosystem services provided by coastal habitats
Spring 2019 – Fall 2019
- Spring 2019: Begin weekly meetings; begin CNN implementation (January-June), satellite change detection (January-September) and UAS surveys (January-September)
- Summer 2019: Select team members begin summer fieldwork (May-August); continue developing and refining remote sensing and machine learning techniques; begin data analysis (April-December); begin manuscript preparation (July-December)
- Fall 2019: Resume weekly meetings; continue analyzing data and manuscript preparation and revisions
Team Outputs to Date
Creating Custom Loss Functions for Multiclass Classification (poster by Yousuf Rehman)
Deep Learning for Land Cover Classification (poster by Diego Chamorro)
Deep Learning and Remote Sensing for Costal Resilience: Scikit-Learn Model Performance (poster by Sofia Nieto)
NLCD Analysis of Wetland and Tree Canopy Dynamics in Eastern North Carolina (poster by Kendall Jefferys)
This Team in the News
- David Johnston, Nicholas School of the Environment-Marine Science and Conservation
- Justin Ridge, Nicholas School of the Environment-Marine Science and Conservation
/graduate Team Members
Patrick Gray, Marine Sci & Conservation-PHD
/undergraduate Team Members
Diego Chamorro, Electrical & Computer Egr(BSE), Computer Science (BSE2)
Kendall Jefferys, Environmental Sci/Policy (AB), English (AB2)
Feroze Mohideen, Electrical & Computer Egr(BSE), Computer Science (BSE2)
Sofia Nieto, Interdepartmental Major
Yousuf Rehman, Electrical & Computer Egr(BSE)
/yfaculty/staff Team Members
Guillermo Sapiro, Pratt School of Engineering-Electrical & Computer Engineering
Brian Silliman, Nicholas School of the Environment-Marine Science and Conservation
/zcommunity Team Members
Brian Boutin, The Nature Conservancy
Carolyn Currin, National Oceanic and Atmospheric Administration (NOAA)
Jenny Davis, National Oceanic and Atmospheric Administration (NOAA)
Paula Gilikin, North Carolina Division of Coastal Management
Aaron McCall, The Nature Conservancy
Brandon Puckett, North Carolina Division of Coastal Management
Sarah Spiegler, North Carolina Sentinel Site Cooperative