Deep Learning and Remote Sensing for Coastal Resilience: Rapid and Automated Coastal Monitoring to Inform Community Recovery from Storm Events (2019-2020)

This is an off-cycle project team developed in response to Hurricane Florence. Applications are now closed.

Background

How can we quickly and effectively assess the effects of storms on our coastal environment? And how can recent technological developments in satellite remote sensing and machine learning support such assessments?

Remote sensing plays a critical role in the study and management of coastal ecosystems. Satellite imagery permits accurate land cover mapping at medium resolution (<5 m), and with the proliferation of cubesats (small satellites, roughly the size of a shoebox), this data is becoming available at weekly and even daily time steps on a global scale.

Unoccupied aerial systems (UAS, aka drones) complement satellites and are becoming an integral tool in studying and managing coastal systems, providing on-demand remote sensing capabilities at low cost and with modular sensors that can assess a wide variety of variables. In coastal systems, small UAS can collect ultra-high resolution (<0.05 m) multispectral imagery and generate high fidelity 3D models of the landscape.

While powerful tools, satellites and UAS generate massive amounts of data. The expertise and time required to analyze this imagery is an obstacle in using it to inform disaster response and storm recovery – an obstacle machine learning is well poised to mitigate.

Project Description

The purpose of this project is to use Hurricane Florence coastal impacts and recovery to develop a methodology for rapid and high-resolution monitoring of North Carolina’s coastline. We will do so by leveraging convolution neural networks (CNNs) to automate change detection in satellite imagery. The change detected via satellite will trigger a time-series of UAS surveys for areas that experienced substantial change.

CNNs are a popular deep learning model inspired by the neural connections in the human brain and visual cortex. CNNs have been extremely successful in analyzing imagery and are a promising method for automating the analysis of remote sensing data. At their simplest, CNNs transform complex input data into more useful output data (e.g. transforming satellite imagery into a habitat map).

Our team will build on related projects using deep learning for remote sensing data analysis and implement a CNN to create automated habitat classification and change detection maps of both satellite and UAS imagery, using the satellite-based change map to inform UAS survey locations. This work will lead to a tool capable of creating accurate habitat maps and detecting ecosystem change from multiple remote sensing platforms.

Built in coordination with local collaborators, our tool will demonstrate a semi-autonomous system capable of monitoring North Carolina’s coast at a scale relevant for statewide management yet sensitive to local storm responses. These tool and map products will be shared with our partners and online through an Environment Systems Research Institute (ESRI) story map.

Ultimately, faster detection of coastal change in the face of more frequent intense storms will facilitate disaster response, conservation management and coastal resilience – aiding communities in immediate recovery and planning long-term responses to climate change-driven threats.

Anticipated Outputs

Habitat classification and change detection satellite mapping tool; publication in peer-reviewed journal; foundation for future research and grants to explore using a coordinated satellite and drone system to monitor and evaluate ecosystem services provided by coastal habitats

Timing

Spring 2019 – Fall 2019

  • Spring 2019: Begin weekly meetings; begin CNN implementation (January-June), satellite change detection (January-September) and UAS surveys (January-September)
  • Summer 2019: Select team members begin summer fieldwork (May-August); continue developing and refining remote sensing and machine learning techniques; begin data analysis (April-December); begin manuscript preparation (July-December)
  • Fall 2019: Resume weekly meetings; continue analyzing data and manuscript preparation and revisions

This Team in the News

New Project Teams Will Tackle Research to Inform Hurricane Preparedness and Resiliency Efforts

David Johnston catches a drone during field work in Antarctica; Rett Newton readies to catch a drone at Cape Lookout.

/faculty/staff Team Members

  • Patrick Gray, Nicholas School of the Environment-Marine Science and Conservation-PHD*
  • David Johnston, Nicholas School of the Environment-Marine Science and Conservation*
  • Justin Ridge, Nicholas School of the Environment-Marine Science and Conservation*
  • Guillermo Sapiro, Pratt School of Engineering-Electrical & Computer Engineering
  • Brian Silliman, Nicholas School of the Environment-Marine Science and Conservation

/zcommunity Team Members

  • Brian Boutin, The Nature Conservancy
  • Carolyn Currin, National Oceanic and Atmospheric Administration (NOAA)
  • Jenny Davis, National Oceanic and Atmospheric Administration (NOAA)
  • Paula Gilikin, N.C. Division of Coastal Management
  • Brandon Puckett, North Carolina Division of Coastal Management
  • Sarah Spiegler, North Carolina Sentinel Site Cooperative (NCSSC)