Its a startling image that illustrates the deep-rooted biases of AI research. Input a low-resolution picture of Barack Obama, the first black president of the United States, into an algorithm designed to generate depixelated faces, and the output is a white man.
Its not just Obama, either. Get the same algorithm to generate high-resolution images of actress Lucy Liu or congresswoman Alexandria Ocasio-Cortez from low-resolution inputs, and the resulting faces look distinctly white. As one popular tweet quoting the Obama example put it: This image speaks volumes about the dangers of bias in AI.
But whats causing these outputs and what do they really tell us about AI bias?
First, we need to know a little a bit about the technology being used here. The program generating these images is an algorithm called PULSE, which uses a technique known as upscaling to process visual data. Upscaling is like the zoom and enhance tropes you see in TV and film, but, unlike in Hollywood, real software cant just generate new data from nothing. In order to turn a low-resolution image into a high-resolution one, the software has to fill in the blanks using machine learning.
In the case of PULSE, the algorithm doing this work is StyleGAN, which was created by researchers from NVIDIA. Although you might not have heard of StyleGAN before, youre probably familiar with its work. Its the algorithm responsible for making those eerily realistic human faces that you can see on websites like ThisPersonDoesNotExist.com; faces so realistic theyre often used to generate fake social media profiles.
What PULSE does is use StyleGAN to imagine the high-res version of pixelated inputs. It does this not by enhancing the original low-res image, but by generating a completely new high-res face that, when pixelated, looks the same as the one inputted by the user.
This means each depixelated image can be upscaled in a variety of ways, the same way a single set of ingredients makes different dishes. Its also why you can use PULSE to see what Doom guy, or the hero of Wolfenstein 3D, or even the crying emoji look like at high resolution. Its not that the algorithm is finding new detail in the image as in the zoom and enhance trope; its instead inventing new faces that revert to the input data.
This sort of work has been theoretically possible for a few years now, but, as is often the case in the AI world, it reached a larger audience when an easy-to-run version of the code was shared online this weekend. Thats when the racial disparities started to leap out.
PULSEs creators say the trend is clear: when using the algorithm to scale up pixelated images, the algorithm more often generates faces with Caucasian features.
It does appear that PULSE is producing white faces much more frequently than faces of people of color, wrote the algorithms creators on Github. This bias is likely inherited from the dataset StyleGAN was trained on [...] though there could be other factors that we are unaware of.
In other words, because of the data StyleGAN was trained on, when its trying to come up with a face that looks like the pixelated input image, it defaults to white features.
This problem is extremely common in machine learning, and its one of the reasons facial recognition algorithms perform worse on non-white and female faces. Data used to train AI is often skewed toward a single demographic, white men, and when a program sees data not in that demographic it performs poorly. Not coincidentally, its white men who dominate AI research.
But exactly what the Obama example reveals about bias and how the problems it represents might be fixed are complicated questions. Indeed, theyre so complicated that this single image has sparked heated disagreement among AI academics, engineers, and researchers.
On a technical level, some experts arent sure this is even an example of dataset bias. The AI artist Mario Klingemann suggests that the PULSE selection algorithm itself, rather than the data, is to blame. Klingemann notes that he was able to use StyleGAN to generate more non-white outputs from the same pixelated Obama image, as shown below:
These faces were generated using the same concept and the same StyleGAN model but different search methods to Pulse, says Klingemann, who says we cant really judge an algorithm from just a few samples. There are probably millions of possible faces that will all reduce to the same pixel pattern and all of them are equally correct, he told The Verge.
(Incidentally, this is also the reason why tools like this are unlikely to be of use for surveillance purposes. The faces created by these processes are imaginary and, as the above examples show, have little relation to the ground truth of the input. However, its not like huge technical flaws have stopped police from adopting technology in the past.)
But regardless of the cause, the outputs of the algorithm seem biased something that the researchers didnt notice before the tool became widely accessible. This speaks to a different and more pervasive sort of bias: one that operates on a social level.
Deborah Raji, a researcher in AI accountability, tells The Verge that this sort of bias is all too typical in the AI world. Given the basic existence of people of color, the negligence of not testing for this situation is astounding, and likely reflects the lack of diversity we continue to see with respect to who gets to build such systems, says Raji. People of color are not outliers. Were not edge cases authors can just forget.
The fact that some researchers seem keen to only address the data side of the bias problem is what sparked larger arguments about the Obama image. Facebooks chief AI scientist Yann LeCun became a flashpoint for these conversations after tweeting a response to the image saying that ML systems are biased when data is biased, and adding that this sort of bias is a far more serious problem in a deployed product than in an academic paper. The implication being: lets not worry too much about this particular example.
Many researchers, Raji among them, took issue with LeCuns framing, pointing out that bias in AI is affected by wider social injustices and prejudices, and that simply using correct data does not deal with the larger injustices.
Others noted that even from the point of view of a purely technical fix, fair datasets can often be anything but. For example, a dataset of faces that accurately reflected the demographics of the UK would be predominantly white because the UK is predominantly white. An algorithm trained on this data would perform better on white faces than non-white faces. In other words, fair datasets can still created biased systems. (In a later thread on Twitter, LeCun acknowledged there were multiple causes for AI bias.)
Raji tells The Verge she was also surprised by LeCuns suggestion that researchers should worry about bias less than engineers producing commercial systems, and that this reflected a lack of awareness at the very highest levels of the industry.
Yann LeCun leads an industry lab known for working on many applied research problems that they regularly seek to productize, says Raji. I literally cannot understand how someone in that position doesnt acknowledge the role that research has in setting up norms for engineering deployments.
When contacted by The Verge about these comments, LeCun noted that hed helped set up a number of groups, inside and outside of Facebook, that focus on AI fairness and safety, including the Partnership on AI. I absolutely never, ever said or even hinted at the fact that research does not play a role is setting up norms, he told The Verge.
Many commercial AI systems, though, are built directly from research data and algorithms without any adjustment for racial or gender disparities. Failing to address the problem of bias at the research stage just perpetuates existing problems.
In this sense, then, the value of the Obama image isnt that it exposes a single flaw in a single algorithm; its that it communicates, at an intuitive level, the pervasive nature of AI bias. What it hides, however, is that the problem of bias goes far deeper than any dataset or algorithm. Its a pervasive issue that requires much more than technical fixes.
As one researcher, Vidushi Marda, responded on Twitter to the white faces produced by the algorithm: In case it needed to be said explicitly - This isnt a call for diversity in datasets or improved accuracy in performance - its a call for a fundamental reconsideration of the institutions and individuals that design, develop, deploy this tech in the first place.
Update, Wednesday, June 24: This piece has been updated to include additional comment from Yann LeCun.
- Machine Learning Answers: Facebook Stock Is Down 20% In A Month, What Are The Chances It'll Rebound? - Trefis - September 22nd, 2020
- Machine Learning in Education Market Incredible Possibilities, Growth Analysis and Forecast To 2025 - The Daily Chronicle - September 22nd, 2020
- Proximity matters: Using machine learning and geospatial analytics to reduce COVID-19 exposure risk - Healthcare IT News - September 22nd, 2020
- Global Machine Learning Market Tends To Show Steady Growth Post Pandemic With Regional Overview and Top Key Players - Verdant News - September 22nd, 2020
- PREDICTING THE OPTIMUM PATH - Port Strategy - September 22nd, 2020
- AI/ML Remains The Most In-Demand Tech Skill Post COVID - Analytics India Magazine - September 22nd, 2020
- Panalgo Brings the Power of Machine-Learning to the Healthcare Industry Via Its IHD Software - AiThority - September 15th, 2020
- Microchip Partners with Machine-Learning (ML) Software Leaders to Simplify AI-at-the-Edge Design Using its 32-Bit Microcontrollers (MCUs) - EE Journal - September 15th, 2020
- What is 'custom machine learning' and why is it important for programmatic optimisation? - The Drum - September 15th, 2020
- PODCAST: NVIDIA's Director of Data Science Talks Machine Learning for Airlines and Aerospace - Aviation Today - September 15th, 2020
- The Use of Machine Learning to Forecast Progression to Advanced AMD - DocWire News - September 15th, 2020
- How Can Machine Learning Help the Teaching Profession? - FE News - September 15th, 2020
- Global Machine Learning in Automobile Market: Development History, Current Analysis and Estimated Forecast to 2024 - The Market Correspondent - September 15th, 2020
- Using machine learning to organize the chemical diversity - Tech Explorist - September 15th, 2020
- Dashboard AI Announces Its Technology Vision for the Foodservice and Hospitality Industry - PRNewswire - September 15th, 2020
- Alfa Releases Second Paper on AI, Using Machine Learning in the Wild - Monitor Daily - September 10th, 2020
- Combatting COVID-19 misinformation with machine learning (VB Live) - VentureBeat - September 10th, 2020
- This artist used machine learning to create realistic portraits of Roman emperors - The World - September 10th, 2020
- Domino Data Lab Named a Leader in Notebook-Based Predictive Analytics and Machine Learning Evaluation by Global Research Firm - Business Wire - September 10th, 2020
- Demonstration Of What-If Tool For Machine Learning Model Investigation - Analytics India Magazine - September 10th, 2020
- RXA to Participate in 2nd Annual A2.AI Conference focused on Machine Learning & Applied AI - PR Web - September 10th, 2020
- 50 Data Science and Analysts Jobs That Opened Just Last Week - Analytics India Magazine - September 10th, 2020
- FSS Launches Next Gen Recon with Machine Learning and Cloud Support - TechGenyz - September 10th, 2020
- Getting to the heart of machine learning and complex humans - The Irish Times - August 28th, 2020
- Global Machine Learning Courses Market Trends, Key Driven Factors, Segmentation And Forecast To 2020-2026 - The Scarlet - August 28th, 2020
- AI and Machine Learning Network Fetch.ai Partners Open-Source Blockchain Protocol Waves to Conduct R&D on DLT - Crowdfund Insider - August 28th, 2020
- UT Austin Selected as Home of National AI Institute Focused on Machine Learning - UT News | The University of Texas at Austin - August 26th, 2020
- Participation-washing could be the next dangerous fad in machine learning - MIT Technology Review - August 26th, 2020
- The Role of Artificial Intelligence and Machine Learning in the... - Insurance CIO Outlook - August 26th, 2020
- Machine Learning Artificial intelligence Market Size and Growth By Leading Vendors, By Types and Application, By End Users and Forecast to 2020-2027 -... - August 26th, 2020
- Air Force Taps Machine Learning to Speed Up Flight Certifications - Nextgov - August 26th, 2020
- What is AutoML and Why Should Your Business Consider It - BizTech Magazine - August 26th, 2020
- Chatbots Are Machine Learning Their Way To Human Language - Forbes - August 26th, 2020
- Explainable AI: From the peak of inflated expectations to the pitfalls of interpreting machine learning models - ZDNet - August 26th, 2020
- Focusing on ethical AI in business and government - FierceElectronics - August 26th, 2020
- Amazon's Machine Learning University To Make Its Online Courses Available To The Public - Analytics India Magazine - August 14th, 2020
- Watch 3 Videos from Coursera's New "Machine Learning for Everyone" - Machine Learning Times - machine learning & data science news - The... - August 14th, 2020
- PhD Research Fellowship in Machine Learning for Cognitive Power Management job with NORWEGIAN UNIVERSITY OF SCIENCE & TECHNOLOGY - NTNU | 219138 -... - August 14th, 2020
- Machine learning is pivotal to every line of business, every organisation must have an ML strategy - BusinessLine - August 14th, 2020
- CORRECTING and REPLACING Anyscale Hosts Inaugural Ray Summit on Scalable Python and Scalable Machine Learning - Yahoo Finance - August 14th, 2020
- Why GPT-3 Heralds a Democratic Revolution in Tech - Built In - August 14th, 2020
- BMW, Red Hat, and Malong Share Insights on AI and Machine Learning During Transform 2020 - ENGINEERING.com - August 14th, 2020
- Algorithm created by deep learning finds potential therapeutic targets throughout the human genome - National Science Foundation - August 14th, 2020
- Ensighten Launches Client-Side Threat Intelligence Initiative and Invests in Machine Learning - WFMZ Allentown - August 6th, 2020
- Hey software developers, youre approaching machine learning the wrong way - The Next Web - August 6th, 2020
- Introducing The AI & Machine Learning Imperative - MIT Sloan - August 6th, 2020
- Who Does the Machine Learning and Data Science Work? - Customer Think - August 6th, 2020
- Artificial Intelligence and Machine Learning Path to Intelligent Automation - Embedded Computing Design - August 6th, 2020
- Blacklight Solutions Unveils Software to Simplify Business Analytics with AI and Machine Learning - PRNewswire - August 6th, 2020
- AI is learning when it should and shouldnt defer to a human - MIT Technology Review - August 6th, 2020
- Moderna Announced Partnership With Amazon Web Services for Their Analytics and Machine Learning Services - Science Times - August 6th, 2020
- Surprisingly Recent Galaxy Discovered Using Machine Learning May Be the Last Generation Galaxy in the Long Cosmic History - SciTechDaily - August 6th, 2020
- STMicroelectronics Releases STM32 Condition-Monitoring Function Pack Leveraging Tools from Cartesiam for Simplified Machine Learning - ELE Times - August 6th, 2020
- Machine Learning Reveals What Makes People Happy In A Relationship - Forbes - August 4th, 2020
- Benefits Of AI And Machine Learning | Expert Panel | Security News - SecurityInformed - August 4th, 2020
- Preparing new machine learning models used to take weeks Activeloop teams up with NVIDIA to reduce that time to hours - MENAFN.COM - August 4th, 2020
- IoT automation trend rides the next wave of machine learning, Big Data - Urgent Communications - August 4th, 2020
- Decoding Practical Problems and Business Implications of Machine Learning - Analytics Insight - August 4th, 2020
- Artificial Intelligence and Machine Learning Industry 2020 Market Manufacturers Analysis, Share, Size, Growth, Trends and Research Report 2026 -... - August 4th, 2020
- Could this software help users trust machine learning decisions? - C4ISRNet - July 27th, 2020
- Top Five Data Privacy Issues that Artificial Intelligence and Machine Learning Startups Need to Know - insideBIGDATA - July 27th, 2020
- COVID-19 Impacts: Machine Learning Market will Accelerate at a CAGR of about 39% through 2020-2024 | The Increasing Adoption of Cloud-based Offerings... - July 27th, 2020
- Deep learning's role in the evolution of machine learning - TechTarget - July 1st, 2020
- 2 books to deepen your command of python machine learning - TechTalks - July 1st, 2020
- What I Learned From Looking at 200 Machine Learning Tools - Machine Learning Times - machine learning & data science news - The Predictive... - July 1st, 2020
- Protecting inventions which use Machine Learning and Artificial Intelligence - Lexology - July 1st, 2020
- Machine learning finds use in creating sharper maps of 'ecosystem' lines in the ocean - Firstpost - July 1st, 2020
- Fake data is great data when it comes to machine learning - Stacey on IoT - July 1st, 2020
- Decisions and NLP Logix Announce Partnership to bring the Power of Machine Learning to Business Process Management - Benzinga - July 1st, 2020
- Machine Learning in Medical Imaging Market Strategies and Insight Driven Transformation 2020-2030 - Cole of Duty - July 1st, 2020
- Impact of COVID-19 Outbreak on Artificial Intelligence and Machine Learning Market to Witness AIBrain, Amazon, Anki, CloudMinds - Cole of Duty - July 1st, 2020
- Machine Learning Market Projected to Register 43.5% CAGR to 2030 Intel, H2Oai - 3rd Watch News - July 1st, 2020
- Learn the business value of AI's various techniques - TechTarget - July 1st, 2020
- Machine Learning As A Service In Manufacturing Market Augmented Expansion to Be Registered by 2018-2023 - 3rd Watch News - July 1st, 2020
- COVID 19 Impact on Machine Learning in Medicine Market Outlook 2020 Industry Size, Top Key Manufacturers, Growth Insights, Demand Analysis and... - July 1st, 2020
- Machine learning algorithm from RaySearch enhances workflow at Swedish radiation therapy clinic - DOTmed HealthCare Business News - July 1st, 2020
- AI and Machine Learning Are Changing Everything. Here's How You Can Get In On The Fun - ExtremeTech - June 25th, 2020
- SLAM + Machine Learning Ushers in the "Age of Perception - Robotics Business Review - June 25th, 2020
- Googles new ML Kit SDK keeps all machine learning on the device - SlashGear - June 25th, 2020
- Machine Learning vs Predictive Analytics: Are they same? - Analytics Insight - June 25th, 2020