Every day, millions of Snapchat users contort their faces on camera, using filters to turn into cartoon dogs, cats, drag queens and sunflowers. The social media filters are impossibly popular, and a Dublin researcher hopes that portraits in the style of Vincent Van Gogh or Rembrandt will soon join them.
Dr Ahmed Selim from Trinity College Dublin’s Connect Centre, in collaboration with Dr Mohamed Elgharib, Qatar Computing Research Institute, and Prof Linda Doyle, director of Connect, has developed a technique that takes a photo and turns it into a faithful portrait in the style of a given painter.
The resulting pictures, imitating the styles of celebrated artists, rely on machine learning technology informed by two mathematical models. One is charged with identifying facial features and retaining their integrity, while another classifies distinctive features of artists’ styles and transfers those textures.
“This technique uses complex algorithms to combine information about the identity of an individual with information from the style of an artist to produce a faithful head portrait,” Doyle said. “In essence, the machine is being taught to do the job of the painter.”
Learning by images
More simply, a machine is doing exactly what a young child does when it learns to distinguish between cats and dogs through building a database of recognisable features, Selim said. “It is learning by images, features,” he said. “Learning by mistake.”
Selim’s deep learning technique is distinct from existing portrait technology, and much more complex than the distorted birds, bees and butterflies already on Snapchat, because its final product remains faithful to both the original face and the given artistic style.
“Automated portrait painting is particularly challenging,” Selim said. “There are already generic painting techniques available but these often deform facial features. Our technique avoids these pitfalls completely.”
Artistic encounter
While artists and historians are impressed with the technology’s commercial potential, some have reservations about its removal of artistic agency.
On a practical level, these two-dimensional portraits do away with the layers, density and texture that characterise original paintings, said Dr Peter Cherry, an art historian at Trinity College. The digital process itself also lacks the intuition, intentionality and attribution of the original artists, he added.
“There seems to be a sort of rational move from one two-dimensional image to another,” Cherry said. “Whereas portraiture was all about the sitter and the artist – the encounter didn’t always result in rational marks on a canvas.”
Still portraits take about 100 seconds to be completed. Selim is working on extending the technology to work seamlessly in video format and in real time. Once he successfully does that, he can see Van Goghs and Frida Kahlos flooding social-media platforms.
“I think there’s a huge commercial potential,” Selim said. “For social networks, social media like Snapchat, it could be really cool.”