At RealSelf, I give a LOT of presentations on topics like protecting user privacy, maintaining a psychologically safe culture and computer security awareness. At the beginning of each slide deck, I’ll drop in these two slides. I find that they’re applicable in the vast majority of my topics and they really set the mood.
If you’re not terrified, you’re not paying attention. Hopefully the picture of Missy softened the blow! Truthfully, the threats lurking out there in cyberspace exist whether or not you are aware of them. I don’t expect everyone at RealSelf to stay on top of the latest data breaches, hacks and vulnerabilities – that’s what I’m here for!
More dangerous than being unaware of these threats is to think that these threats don’t apply to you. You may think your data is not valuable, your identity not worth stealing or that it just can’t happen to you. This mentality makes you an excellent target for victimization.
Facial Recognition Technology
So why should we be concerned about selfies? Enter Facial Recognition Technology! Most of us use Facial Recognition Technologies (FRT) daily; built-in to our phones, tablets, workplaces and security systems. FRT is not threatening by definition, but is highly susceptible to biases and inaccuracies driven by homogenous development teams and a historical, systemic low regard for user privacy.
What does this look like in the real world? We see large, private organizations with the capability to harvest public information on the internet for use in their FRT models. When you posted that selfie on Instagram did you consent to your face being used in this manner? Of course not, you were just sharing content with your followers! What about your drivers license photo; have you ever consented to allowing that image to be shared with private companies for the purpose of facial recognition model improvement? I seriously doubt it! Unfortunately, these are common occurrences in 2020.
But really, what’s the risk of your image being used without your consent? It’s inarguably an invasion of privacy and a breach of trust – but is there any real damage done? Absolutely. I mentioned earlier that FRT is highly susceptible to implicit bias, but what does that mean?
Facial Recognition Primer
Before we can fully understand the biases of Facial Recognition Technology, let’s look at how FRT is developed. Generally, FRTs use a model developed by a team intended to serve the team’s objective. This objective could be to identify specific individuals in a crowd, calculate the likelihood that detected faces in multiple samples are the same person, or detect emotional state based on facial expression.
These models incorporate mathematical algorithms and machine learning to help computers detect and calculate the position of facial landmarks including eye and mouth corners, distance between eyes, nose position and more. The model can take these calculations, apply logic to them as determined by the development team, and output the results. The development team feeds sample data to the model, helping the model “learn” and improve. These teams may incorporate human review in order to reduce errors and further increase the accuracy of the model.
This process seems pretty straightforward and resilient against errors, so where does it go sideways? Unfortunately, the biases existing within the development team, implicit or explicit, will be directly transferred to the FRT model by way of the supplied sample data and model logic. Let’s dig into some real world examples.
Threats posed by Facial Recognition
Joy Buolamwini, a researcher at the M.I.T. Media Lab and head of the Algorithmic Justice League, found that models by Microsoft, IBM and Face++ had error rates of:
- 1 percent of lighter-skinned males
- 7 percent of lighter-skinned females
- 12 percent of darker-skinned males
- 35 percent of darker-skinned females
While there are some FRT legislative actions currently awaiting a vote, there are no federal oversight or regulatory powers in effect at this time. Local jurisdictions may impose restrictions on FRT use, but these restrictions are few and far between. I encourage you to check your local jurisdiction at perpetuallineup.org and to understand how LEOs are legally permitted to utilize FRT in your area.
In Seattle, for instance, SPD can search mugshots but need reasonable suspicion to run a search. South Sound 911 sought real-time capabilities but SPD banned its use. Despite the ban, SPD has been using Clearview.AI since at least September 2019 – a clear violation of active surveillance laws. The documentation for their system indicates that it “does not see race, sex, orientation or age”. This contradicts a 2012 FBI co-authored study, and does not reflect the fact that minorities are likely overrepresented in the system. In King County, for example, Blacks are arrested at a rate 294% higher than their share of the population.
How do these biases transfer from the development team to the model? Like many biases, they exist implicitly within all humans. If the development team is homogenous in their gender, sexuality, ethnicity, geographic locale or culture, those biases will be present in any system developed by the team. Most commonly, we see the same results that Joy Buolamwini found at M.I.T., that FRTs are very good at detecting lighter-skinned males but experience dramatically higher error rates on any other group. This bias is not exclusive to FRTs developed in predominantly lighter-skinned regions. FRTs, regardless of objective or area of origin, fall victim to the biases present within the development team.
If you couple the inescapable biases with unchecked use of FRT by private security organizations and law enforcement agencies, the end result is a recipe for the suppression of minority speech. So the question changes from “What’s the risk of my image being used without my consent?” and “What damage is being done?” to “Am I comfortable with my image being used to suppress speech?”. For me, the answer is unequivocally “No.”
So… What now?
If you made it this far, you’re probably terrified – but that means you’re paying attention! We do have some avenues to fix these problems.
- Increased diversity and inclusion
This involves having diverse development teams over all aspects of our technical world, not just limited to FRT. Any system that makes automated decisions that have material impact on human lives should be open for public review and should incorporate diversity and inclusion metrics. The Algorithmic Justice League provides diverse sets of sample images for developers to use when teaching their FRT model which removes the possibility of sample data bias. Development teams should incorporate individuals from many backgrounds.
- Psychological Safety
However, diversity alone is not the solution. For diverse teams to function, a workplace culture of psychological safety is required for the open sharing of concerns and ideas, but that’s a subject for an entirely separate paper.
- Increased oversight
We see how easily FRTs and other AI/ML driven systems can fall into bias when unregulated. It is my opinion that true, systemic change can only be accomplished by federal oversight in this space. Requiring commercial and government entities to
- Allow oversight of their models by independent diversity and inclusion panels
- Obtain user consent before using user data in model
- Submit their models to discrimination testing
- Open their model for public comment
- Use diverse and inclusive sample data
- Prohibit sharing of user data with third-parties
There are several federal bills that have been introduced to address some of these actions, including the Facial Recognition and Biometric Technology Moratorium Act of 2020 and Commercial Facial Recognition Privacy Act of 2019. Hopefully we will see some traction on FRT legislation in 2021.
What about individuals in the meantime?
Until we can fix this problem at a systemic level, individual users are mostly on their own when it comes to protection from FRTs. One of the tools RealSelf is working with is the FAWKES image cloak. Run your selfie through FAWKES before posting and any FRT model that ingests it will be effectively “poisoned”, meaning the accuracy and performance of the FRT is negatively impacted. If enough poisoned images are ingested, the FRT will become less and less useful to the point of uselessness. FAWKES works by making mathematical changes to the landmarks of all detected faces in a photo. These changes are imperceptible to the human eye, but dramatically alter the faces when viewed through a FRT model.
FAWKES was developed by the University of Chicago SANDS lab. You can download Windows, Mac and Linux versions of FAWKES from the FAWKES homepage.
What is RealSelf doing?
At RealSelf, we believe that we have the moral obligation to continuously push the envelope of user security and privacy. This means keeping up with emerging threats, privacy legislation and the latest technology available.
Most recently, this work has led us to look at how we treat face photos and what we can do on our side to keep these contributors safe. Deploying FAWKES as part of our image upload service has become a real possibility for RealSelf and is one that we are continuing to test for future implementation.
RealSelf also plans to increase its contribution to the Security and Privacy communities through authoring technical blog posts, releasing our Security and Privacy work as open source, and directly supporting nonprofit organizations like the Algorithmic Justice League. We want to empower our users – both doctors and patients – with the tools they need to keep themselves safe.