Image Processing Algorithms for Art Conservation (2016-2017)
The 14th century St. John altarpiece by Francescuccio Ghissi was removed from its church over 100 years ago. It was sawn apart into nine panels corresponding to the individual scenes, before being sold to collectors. In the process one of the panels, the final illustration of the life of St. John the Evangelist, was lost.
For an exhibition of the altarpiece panels, the North Carolina Museum of Art contracted with three other museums to bring to Raleigh the five panels that would complement the three in its own collection. It also commissioned art reconstruction expert Charlotte Caspers to paint a replacement ninth panel using 14th-century techniques and materials as close as possible to those used by Ghissi.
The new panel, with vivid, untarnished colors and richly reflecting burnished and punchmarked gold surfaces, demonstrates how wonderfully bright and sparkling these altarpieces were in their own time. Yet if this replacement panel was simply added to the altarpiece frame among the eight original panels, it would make those panels look dull and faded by contrast. Therefore, the panel needed to be virtually aged, and the technical analysis needed to achieve the virtual aging of the replacement ninth panel could also be applied in the reverse direction.
By studying the old as well as the new panels, the Bass Connections team virtually aged the new panel and made a digital copy in which the gold looks duller, the colors were altered to mimic 650 years of aging of the pigments and small cracks were added. A print of this virtually aged panel completes the reunited Ghissi altarpiece without distracting from the authentic panels, and the new panel was exhibited separately in the exhibition.
Team members took on a variety of image processing tasks connected with the North Carolina Museum of Art’s Reunited exhibition. The team’s achievements included crack removal, color remapping and modeling the gold surfaces to rejuvenate them. Team members learned about approaches done by others and improved upon them. They consulted often with the art conservators to get feedback and fine-tune the results.
Once the team determined the exact correspondence between “old” and “new” for each pigment mixture used in the altarpiece, and had fine-tuned the digital image manipulations to make the transition from new to old, the team could also take a high-resolution image of the old panels and map their old, aged color planes to corresponding “freshly painted” versions, thus rejuvenating the 14th-century panels.
In order to give a convincing “new” impression, the team also needed to remove the craquelure—that is, use image analysis techniques to find all the cracks in the paintings, remove the corresponding pixels and “inpaint” them digitally. The results of this digital rejuvenation work were shown at the North Carolina Museum of Art exhibition.
One team member spent time interacting with Dr. Laura Alba, head of the imaging lab at the Prado Museum in Madrid, and received training by Bruno Cornelis on the use of Platypus software for the removal of cradle artifacts in digitized X-ray photos of paintings on wood panels. Another worked on images of a two-sided painting on canvas that emerged recently in Italy, in an attempt to separate the front and verso parts of the X-ray image in which both paintings are superimposed.
Following the exhibition, the team used machine learning tools to dig further into problems around the crack patterns that a team member detected and removed in the Ghissi panels as part of the rejuvenation process. They attempted to detect automatically different types of patterns in the fine network of cracks, which could provide valuable information about the state of conservation of the panel and its gesso layers. Other refinements included making the color remapping techniques more systematic and producing realistic gold surface images in the 3D-modeling.
Summer 2016 – Spring 2017
Reunited: Francescuccio Ghissi’s St. John Altarpiece, exhibition at the North Carolina Museum of Art, September 10, 2016 – March 5, 2017
Reuniting and Rejuvenating the Ghissi Altarpiece (presentation by Geena Gomez at Bass Connections Showcase, April 20, 2017)
Presentation by Ingrid Daubechies on the virtual aging and rejuvenation of the Ghissi panels, North Carolina Museum of Art, January 21, 2017
Observations on Reconstructing Old Master Paintings and Ghissi’s Missing Panel (presentation by Charlotte Caspers at the Franklin Humanities Institute, January 23, 2017)
This Team in the News
Fellowships Snapshots 2016 (Rujie Yin)
/faculty/staff Team Members
Ingrid Daubechies, Arts & Sciences-Mathematics*
Ed Triplett, Arts & Sciences-Art, Art History, and Visual Studies*
/graduate Team Members
Rachel (Rujie) Yin, Mathematics-PHD
/undergraduate Team Members
Geena Gomez, Mechanical Engineering (BSE), Electrical Engineering (BSE2)
Raphael Kim, Mechanical Engineering (BSE), Computer Science (BSE2)
Mitchell Parekh, Computer Science (BS)
/zcommunity Team Members
North Carolina Museum of Art
Bruno Cornelis, Vrije Universiteit Brussel