When Jillian York, a 36-year-old American activist, was on vacation in February, she received an unexpected text. Her friend Adam Harvey, another activist and researcher, had discovered photos of her in a US government database used to train facial-recognition algorithms, and wondered whether she knew about it.
York, who works in Berlin for the Electronic Frontier Foundation, a digital rights non-profit group, did not. She was stunned to discover that the database contained nearly a dozen images of her, a mixture of photos and YouTube video stills, taken over a period of almost a decade.
When she dug into what the database was used for, it dawned on her that her face had helped to build systems used by the federal government to recognise faces of interest, including suspected criminals, terrorists and illegal aliens.
"What struck me immediately was the range of times they cover," York says. "The first images were from 2008, all the way through to 2015." Two of the photos, by a photographer friend, had been scraped from Google. "They were taken at closed meetings. They were definitely private in the sense that it was me goofing around with friends, rather than me on stage," she adds.
Another half-dozen photos had been clipped from YouTube videos of York speaking at events, on topics including freedom of expression, digital privacy and security. “It troubles me that someone was watching videos of me and clipping stills for this purpose,” she says.
Competitive advantage
York is one of 3,500 subjects in this database, which is known as Iarpa Janus Benchmark-C (IJB-C). Iarpa is a US government body that funds innovative research aimed at giving the US intelligence community a competitive advantage; Janus – named after the two-faced Roman god – is its facial-recognition initiative.
The dataset, which was compiled by a government subcontractor called Noblis, includes a total of 21,294 images of faces (there are other body parts too), averaging six pictures and three videos per person, and is available on application to researchers in the field. By their own admission, its creators picked “subjects with diverse occupations, avoiding one pitfall of “celebrity-only” media [which] may be less representative of the global population.”
Other subjects in the dataset include three EFF board members, an Al-Jazeera journalist, a technology futurist and writer, and at least three Middle Eastern political activists, including an Egyptian scientist who participated in the Tahrir Square protests in 2011.
The primary use of facial-recognition technology is in security and surveillance
None of the people described above was aware of their inclusion in the database. Their images were obtained without their explicit consent, as they had been uploaded under the terms of Creative Commons licences, an online copyright agreement that allows images to be copied and reused for academic and commercial purposes by anyone.
The primary use of facial-recognition technology is in security and surveillance, whether by private companies such as retailers and events venues, or by public bodies such as police forces to track criminals. Governments increasingly use it to identify people for national and border security.
The biggest technical obstacle to achieving accurate facial recognition thus far has been the inability of machines to identify human faces when they are only partially visible, shrouded in shadow or covered by clothing, as opposed to the high-resolution, front-facing portrait photos the computers were trained on.
Unposed
To teach a machine how to better read and recognise a human face in these conditions, it has to be trained using hundreds of thousands of faces of all shapes, sizes, colours, ages and genders. The more natural, varied and unposed the faces are, the better they simulate real-life scenarios in which surveillance might take place, and the more accurate the resulting models for facial recognition.
In order to feed this hungry system, a plethora of face repositories – such as IJB-C – have sprung up, containing images manually culled and bound together from sources as varied as university campuses, town squares, markets, cafes, mugshots and social-media sites such as Flickr, Instagram and YouTube.
To understand what these faces have been helping to build, we worked with Adam Harvey, the researcher who first spotted Jillian York’s face in IJB-C. An American based in Berlin, he has spent years amassing more than 300 face datasets and has identified some 5,000 academic papers that cite them.
The images, we found, are used to train and benchmark algorithms that serve a variety of biometric-related purposes – recognising faces at passport control, crowd surveillance, automated driving, robotics, even emotion analysis for advertising. They have been cited in papers by commercial companies including Facebook, Microsoft, Baidu, SenseTime and IBM, as well as by academics around the world, from Japan to the United Arab Emirates and Israel.
Shocked
"We've seen facial recognition shifting in purpose," says Dave Maass, a senior investigative researcher at the EFF, who was shocked to discover that his own colleagues' faces were in the Iarpa database. "It was originally being used for identification purposes...Now somebody's face is used as a tracking number to watch them as they move across locations on video, which is a huge shift. [researchers] don't have to pay people for consent, they don't have to find models, no firm has to pay to collect it, everyone gets it for free."
The dataset containing Jillian York's face is one of a series compiled on behalf of Iarpa (earlier iterations are IJB-A and -B), which have been cited by academics in 21 different countries, including China, Russia, Israel, Turkey and Australia.
They have been used by companies such as the Chinese AI firm SenseTime, which sells facial-recognition products to the Chinese police, and the Japanese IT company NEC, which supplies software to law enforcement agencies in the US, UK and India.
The images in them have even been scraped by the National University of Defense Technology in China, which is controlled by China's top military body, the Central Military Commission. One of its academics collaborated last year in a project that used IJB-A, among other sets, to build a system that would, its architects wrote, "[enable] more detailed understanding of humans in crowded scenes", with applications including "group behaviour analysis" and "person re-identification".
In China, facial scanning software has played a significant role in the government's mass surveillance and detention of Muslim Uighurs in the far-western region of Xinjiang. Cameras made by Hikvision, one of the world's biggest CCTV companies, and Leon, a former partner of SenseTime, have been used to track Muslims all over Xinjiang, playing a part in what human-rights campaigners describe as the systematic repression of millions of people.
International outcry
Earlier this week, it emerged that SenseTime had sold its 51 per cent stake in a security joint venture with Leon in Xinjiang after the growing international outcry over the treatment of the Uighurs.
“That was the shocking part,” York says, as she considers the ways multiple companies and agencies have used the database. “It’s not that my image is being used, it’s about how it’s being used.”
Researchers point out that facial analysis technologies aren’t just for surveillance – they could be used for health monitoring; for instance, scanning faces to see if someone is developing dementia or type 2 diabetes, or to check for drowsiness or inebriation in drivers.
I'm not worried about government, I'm worried about Google and Facebook
If datasets weren’t shareable, corporations such as Facebook and Google, which have billions of user photos and videos uploaded to their sites each day, would be the only organisations with access to an ocean of high-quality face data and so would have the best face recognition algorithms, some researchers argue.
"I'm not worried about government, I'm worried about Google and Facebook," says Karl Ricanek, a professor at the University of North Carolina Wilmington who has built two publicly accessible face datasets. "In my opinion, they have more info on citizens than governments themselves, and we can't affect leadership at these companies. I think our government at least has a good mission. From an academic perspective we are trying to solve problems that we think will make life better in our world. Most of us aren't attempting to make money."
Extensive data pools
Despite commercial companies often having their own extensive data pools, they too have increasingly turned to the internet to cull larger and more natural datasets used to benchmark and train algorithms. For instance, Facebook created a dataset called People in Photo Albums, consisting of more than 37,000 photos of 2,000 individuals, including children, from personal Flickr photo albums. “While a lot of progress has been made recently in recognition from a frontal face, non-frontal views are a lot more common in photo albums than people might suspect,” Facebook researchers wrote in their paper.
Their dataset specifically picks out photos with “large variations in pose, clothing, camera viewpoint, image resolution and illumination”, and the paper describes a new algorithm that can recognise even these partially hidden faces with high accuracy.
“We hope our dataset will steer the vision community towards the very important and largely unsolved problem of person recognition in the wild,” the Facebook researchers conclude. This dataset has now been reused all over the world, including by the National University of Defense Technology in China to improve video surveillance technology.
“To be clear, we are not collaborating with the Chinese government on facial recognition and never have,” a Facebook spokesperson said. “However, there will always be a question if advancements in technology should be shared widely or closely held. Facebook and other leading technology companies believe that the scientific community can share learnings to advance technology, while also working to prevent abuse.”
‘Morals and ethics’
Maass, the Electronic Frontier Foundation researcher, says: “This is not a question of legality but of morals and ethics. I’m not sure where a research project like this would fall, but I wonder if creating a dataset to train surveillance tech is the same as conducting surveillance yourself.”
Where images have been scraped off the internet, researchers – even at large companies such as Microsoft, IBM and Facebook – have relied on the Creative Commons licence to stand in for user consent. But the application of these photos as training data for surveillance and facial analysis is so far removed from what the licence was originally intended to cover that Creative Commons itself, a non-profit, recently put out a statement to clarify the change.
"CC licences were designed to address a specific constraint, which they do very well: unlocking restrictive copyright. But copyright is not a good tool to protect individual privacy, to address research ethics in AI development, or to regulate the use of surveillance tools employed online," chief executive Ryan Merkley wrote in March. "Those issues rightly belong in the public policy space...[we hope to] engage in discussion with those using our content in objectionable ways, and to speak out on...the important issues of privacy, surveillance, and AI that impact the sharing of works on the web."
Ultimately, experts believe it is too late to put the face data back in a box, or to restrict its movement across geographic borders. “We may trust the entity that gathers or captures that information, and we may have consented to an initial capture, but custody over that dataset leaks,” says Greenfield, the tech writer who features in Microsoft’s celebrity dataset. “It can leak through hacking, corporate acquisition, simple clumsiness, it can leak through regime change. Even if [creators] attempted to control access, there’s no way they could stop it coming into the hands of the Israeli, American or Chinese state, or anyone who wants to train up facial-recognition algorithms.”
‘Brazen abuses’
For Harvey, who has spent almost a decade trying to illustrate the scale of the issue, there doesn’t seem to be an end in sight. “There are so many egregious examples in these datasets that are just brazen abuses of privacy,” he says. “Some of them come from public cameras pointed at the street, and there are even a few that came from cameras in cafes. After looking at these, you never know when you could walk in front of a camera that may one day be part of a training dataset.”
In fact, recognising a face is only the first step of biometric surveillance, he suggests. “It’s really like an entry-level term to much broader, deeper analysis of people’s biometrics. There’s jaw recognition – the width of your jaw can be used to infer success as CEO, for example. Companies such as Boston-based Affectiva are doing research that analyses faces in real time, to determine from a webcam or in-store camera if someone is going to buy something in your store.”
Other analyses, he adds, can be used to determine people’s tiredness, skin quality and heart rate, or even to lip-read what they are saying. “Face recognition is a very deceiving term, technically, because there’s no limit,” he concludes. “It ends ultimately only with your DNA.”
- Copyright The Financial Times Limited 2019
