Image Processing Algorithms for Art Conservation (2016-2017)

Background

The 14th century St. John altarpiece by Francescuccio Ghissi was removed from its church over 100 years ago. It was sawn apart into nine panels corresponding to the individual scenes, before being sold to collectors. In the process one of the panels, the final illustration of the life of St. John the Evangelist, was lost.

For an exhibition of the altarpiece panels, the North Carolina Museum of Art contracted with three other museums to bring to Raleigh the five panels that would complement the three in its own collection. It also commissioned art reconstruction expert Charlotte Caspers to paint a replacement ninth panel using 14th-century techniques and materials as close as possible to those used by Ghissi.

The new panel, with vivid, untarnished colors and richly reflecting burnished and punchmarked gold surfaces, demonstrates how wonderfully bright and sparkling these altarpieces were in their own time. Yet if this replacement panel was simply added to the altarpiece frame among the eight original panels, it would make those panels look dull and faded by contrast.

Project Description

Therefore, the panel needed to be virtually aged, and the technical analysis needed to achieve the virtual aging of the replacement ninth panel could also be applied in the reverse direction. Once the team determined the exact correspondence between “old” and “new” for each pigment mixture used in the altarpiece, and had fine-tuned the digital image manipulations to make the transition from new to old, the team could also take a high-resolution image of the old panels and map their old, aged color planes to corresponding “freshly painted” versions, thus rejuvenating the 14th-century panels.

In order to give a convincing “new” impression, the team also needed to remove the craquelure—that is, use image analysis techniques to find all the cracks in the paintings, remove the corresponding pixels and “inpaint” them digitally. The results of this digital rejuvenation work are shown at the NCMA exhibition.

The goal of this Bass Connections project is to expose undergraduate students to the concrete problems posed by image processing for art conservation and art history, and have them produce results that are useful to art conservators.

Undergraduate team members Geena Gomez, Raphael Kim, Mitchell Parekh and Sam Slack worked with postdoc Bruno Cornelis and graduate team member Rachel Yin on several image processing tasks connected with the North Carolina Museum of Art’s Reunited exhibition. This included crack removal, color remapping and modeling the gold surfaces to rejuvenate them. Students learned about approaches done by others and improved upon them. The team met regularly and consulted often with the art conservators to get feedback and fine-tune the results.

Gomez was able to spend time interacting with Dr. Laura Alba, head of the imaging lab at the Prado Museum in Madrid, and received training by Cornelis (now in Brussels) on the use of the software Platypus, which Cornelis developed while at Duke, for the removal of cradle artifacts in digitized X-ray photos of paintings on wood panels. Parekh worked on images of a two-sided painting on canvas that emerged recently in Italy, in an attempt to separate the front and verso parts of the X-ray image in which both paintings are superimposed.

By studying the old as well as the new panels, the Bass Connections team virtually aged the new panel and made a digital copy in which the gold looks duller, the colors were altered to mimic 650 years of aging of the pigments and small cracks were added. A print of this virtually aged panel now completes the reunited Ghissi altarpiece without distracting from the authentic panels, and the new panel is exhibited separately in the exhibition.

In Spring 2017, the team will use machine learning tools to dig further into problems around the crack patterns that Slack detected and removed in the Ghissi panels as part of the rejuvenation process. The team will try to detect automatically different types of patterns in the fine network of cracks, which could provide valuable information about the state of conservation of the panel and its gesso layers. Kim will explore how to make his color remapping techniques more systematic. Gomez will work on the 3D-modeling to produce realistic gold surface images.

Timing

Summer 2016 – Spring 2017

Team Outcomes to Date

Reunited: Francescuccio Ghissi’s St. John Altarpiece, exhibition at the North Carolina Museum of Art, September 10, 2016 – March 5, 2017

Reunited: The Ghissi Altarpiece

Reunited: An Art Historical and Digital Adventure

This Team in the News

Graduate Fellowship Winners Describe Their Research

Fellowships Snapshots 2016 (Rujie Yin)

Using Math to Repair a 650-Year-Old Masterpiece

Using Mathematics to Repair a Masterpiece

A Tour of the NC Art Museum’s “Reunited: Francescuccio Ghissi’s St. John Altarpiece”

NCMA Recreates Missing Panel in Ghissi Altarpiece Exhibition

N.C. Museum of Art Presents Completed Ghissi Altarpiece for First Time in Over 100 Years

Software Helps Art Conservators Clear the Cradling

How a Duke University Technology Solved a Decades-old Irritation for Art Conservators

Faculty/Staff Team Members

Ingrid Daubechies, Trinity - Mathematics*
Ed Triplett, Wired! Lab*

Graduate Team Members

Rachel (Rujie) Yin, Graduate School - PhD in Mathematics

Undergraduate Team Members

Geena Gomez, Civil & Environmental Engineering
Raphael Kim, Mechanical Engineering, Computer Science (BS2)
Mitchell Parekh
Sam Slack

Community Team Members

Multiple Contributors, North Carolina Museum of Art
Bruno Cornelis, Vrije Universiteit Brussel

* denotes team leader

Status

Active