The Bias Embedded in Algorithms

Algorithms and the data that drive them are designed and created by people, which means those systems can carry biases based on who builds them and how they’re ultimately deployed. Safiya Umoja Noble, author of Algorithms of Oppression: How Search Engines Reinforce Racism, offers a curated reading list exploring how technology can replicate and reinforce racist and sexist beliefs, how that bias can affect everything from health outcomes to financial credit to criminal justice, and why data discrimination is a major 21st century challenge.

Google Has a Striking History of Bias Against Black Girls
My first encounter with racism in search was in 2009 when I was talking to a friend who causally mentioned one day, “You should see what happens when you Google ‘black girls.’” I did and was stunned. (An excerpt from Algorithms of Oppression.)

Machine Bias
There’s software used across the country to predict future criminals. And it’s biased against blacks.

There Is a Racial Divide in Speech-Recognition Systems, Researchers Say
Technology from Amazon, Apple, Google, IBM and Microsoft misidentified 35 percent of words from people who were black. White people fared much better.

Technology Can’t Fix Algorithmic Injustice
We need greater democratic oversight of AI not just from developers and designers, but from all members of society.

Why Algorithms Can Be Racist and Sexist
A computer can make a decision faster. That doesn’t make it fair.

Social Inequality Will Not Be Solved by an App
We need more intense attention on how artificial intelligence forestalls the ability to see what kinds of choices we are making.

Racial Bias Skews Algorithms Widely Used to Guide Care from Heart Surgery to Birth, Study Finds
Algorithms used by hospitals and physicians to guide the health care given to tens of millions of Americans are shot through with implicit racism that their creators are often unaware of, according to a study.

Can Racist Algorithms Be Fixed?
A new study adds to the debate over racial bias in risk assessment tools widely used in courtrooms.

Human Biases Are Baked into Algorithms. Now What?
The Apple Card debut stumbled as good credit scores didn’t help boost womens’ credit limits.

15 Books by Black Scholars the Tech Industry Needs to Read Now
From the way search engines reinforce racism to how digital culture has reshaped the way Black identity is expressed.

The Underworld of Online Content Moderation
How the work of content moderation takes a psychological toll.
Videos

Black Future Month
Is the future more Black Panther or Black Mirror? Guest correspondent Sasheer Zamata investigates Afrofuturism and the journey to make Snapchat filters actually work.

Before Algorithms: A History of Bias and Oppression in Computing
A conversation about the underlying systems of power, privilege, and oppression as they relate to present-day conversations about algorithms.

Ruha Benjamin on ”The New Jim Code? Race, Carceral Technoscience, and Liberatory Imagination“
From everyday apps to complex algorithms, technology has the potential to hide, speed, and even deepen discrimination, while appearing neutral and even benevolent when compared to racist practices of a previous era.

Surveillance and the Public Sphere: Confronting a Democratic Dilemma
The increasingly precise segmentation and targeting of commercial messages has been enabled in large part through the analysis of massive amounts of transaction-generated-information.

Technologies of Control and Our Right of Refusal
Most of us don’t realize how much digital systems govern access to our basic public services, like education, health and housing. Even more terrifying is how much data is collected about us and used without our knowledge.

Examining Black Feminism in the Digital Era
It is important to examine the digital manifestations of misogynoir – or what it means to be a Woman of Color existing in the hegemonic spaces of digital technology. But our conceptual frameworks fail to capture the everyday practices that Women of Color exhibit online.

About the curator
Safiya Umoja Noble is an Associate Professor at the University of California, Los Angeles (UCLA) in the Department of Information Studies, where she serves as the Co-Director of the UCLA Center for Critical Internet Inquiry. She is the author of a best-selling book on racist and sexist algorithmic bias in commercial search engines, Algorithms of Oppression: How Search Engines Reinforce Racism (NYU Press).

Get fascinating articles like these directly in your inbox.
Join millions and sign up for Pocket’s daily newsletter.