Team profile by Ada Ye, Tyler Feldman, Jose Moscoso, Jessie Ou, Wendy Zhang and Eddy Lin

This team has demonstrated the effectiveness of using synthetic imagery to enhance the ability of energy infrastructure detection algorithms to automatically map the locations of infrastructure across diverse geographic domains. This is a major step towards enabling algorithms for energy infrastructure (and other objects visible overhead) to be accurately and automatically detected in overhead imagery in situations when there is limited labeled training imagery data in the region of interest.

This project is part of a larger effort to develop detailed, up-to-date maps of energy infrastructure through using deep learning techniques that can automatically and rapidly scan massive volumes of remotely sensed data, such as satellite imagery. Key stakeholders in electrification planning processes do not have up-to-date information on energy infrastructure limiting energy access planning efforts. In addition, traditional methods of collecting this locational data, such as on-ground surveys, are time-consuming and expensive. Deep learning approaches provide a powerful tool for researchers, policy-makers, and governments to continuously collect energy systems information and make informed decisions about providing electricity to underserved communities.

In past years, this Bass Connections project team has developed deep learning models that can detect energy infrastructure in satellite imagery, including power transmission lines and off-grid distributed energy generation sources such as solar photovoltaics. Deep learning models often need massive amounts of hand-labeled real images to achieve reasonable performance. However, most energy infrastructure is rare both in the total number of installations and the density of those installations, so collecting and manually annotating large amounts of satellite images is an expensive and time-consuming process.

In addition to the lack of data, object detection models typically perform worse on images that differ from the ones it has previously seen in training data: this is commonly the case that the area of interest differs from the training data. In our case, this means the model will struggle when deployed to new geographies and to different types of energy infrastructure.

To address the problem of data scarcity and to help our model overcome differences in geography, our team created synthetic overhead imagery that supplements real training data. Synthetic images can be made rapidly, cheaply, and come with automatic labels for the objects. The team created an efficient pipeline for synthetic data generation that can be applied to many different types of energy infrastructure. At this stage, our experiments focus on wind turbines, although this can be expanded to other types of infrastructure. The team conducted a set of experiments to test the impact of augmenting the training dataset with synthetic data. We found that synthetic imagery data significantly improved the model’s performance accuracy and generalizability. The results show that adding synthetic imagery can be particularly useful when training data is limited and when it is cost-prohibitive to collect and label more data for the region of interest.

Given the object detection performance improvement delivered by synthetic imagery, our project has great potential to be applied to less developed regions of the world where there is limited or no access to electricity. When there is a lack of real labeled data in target locations where we aim to deploy energy resources to, synthetic imagery can be easily generated by incorporating unlabeled overhead images of the regions and adding synthetically generated infrastructure on top of them. The resulting improved model can then be used to map out the energy infrastructure of that region and help local stakeholders optimize energy access planning activities.

Future work of the project includes expanding to other types of energy infrastructure, such as solar panels and transmission towers, exploring alternative data generation methodologies, and evaluating the model in situations with no labeled training data for the region of interest.

Visit the team’s website for more information on the project.