Step One
Madonna
Oxia Palus
2020 / GANs, GPUs, Multispectral Imaging
Oxia Palus is on a mission to uncover masterpieces lost to the ages using the power of AI. It’s a powerful dichotomy of history and innovative technology that can give us true insight into art lost to the ages.
The Process
The Leonardeschi were a group of artists that worked in the studio of or under the influence Leonardo da Vinci, and 225 of their paintings were used to train the NVIDIA pix2pixHD model.
The Madonna of the Carnation diagram is one example of an application of the model in which the model can map an edgemap back to a painting.
Training the pix2pixHD model with a coarse-grained labeling semantic helps to guide the model in recognizing attributes such as hair and skin.
Step Two
Oxia Palus manually co-registered (aligned) the x-ray of da Vinci's Virgin on the Rocks, from Imperial College London, with the x-ray trace produced by the National Gallery.
They then further manually edited the co-registered image, adding in missing trace components and labeling attributes such as skin, hair, and clothing.
Translucent colored labeling was used to prevent saturation of the underlying co-registered image.
Step Three
The left video shows a varying set of input color maps of a segment of the Virgin of the Rocks x-ray. The right video shows the output of the pix2pixHD Leonardeschi model, with a set of input color maps on the left. The X-ray is not a perfect ghostly outline. But by varying it slightly, a smooth video of potential reconstructions is created.
The Painting
The labeled co-registered image is segmented (zoomed in on Madonna) and subsequently a set of augmented segments are produced that vary in brightness, contrast, and sharpness. These augmented segments are then passed to the trained Leonardeschi model and inference is run, transforming the augmented segments in a set of paintings. After an exploration of the set of paintings (possible solutions) the most aesthetically pleasing result is manually selected. The final painting, Madonna, shows a resurrected Leonardo da Vinci.
About George Cann
George's journey into the world of AI art is circular, from art to space science to machine learning to art. His work reconstructing the past with AI offers an alternative to how AI is creating new value in art. George is also a PhD candidate at University College London researching biosignatures in the Martian atmosphere and he holds a Master’s in Space Science and Engineering from University College London and a Bachelor’s in Mathematics and Physics from the University of Warwick.
Featured Sessions
The AI Art Race: Resurrecting Lost Art with NVIDIA GPUs
Oxia Palus has a lofty mission: to uncover masterpieces lost to the ages using artificial intelligence. Learn how they translate from X-rays to paintings using datasets, standard image augmentations, holistically-nested edge detection, and other modern technologies to recreate Leonardo Da Vinci's Virgin of the Rocks.
