The Bias Embedded in Algorithms

​Algorithms and the data that drive them are designed and created by people, which means those systems can carry biases based on who builds them and how they’re ultimately deployed. Safiya Umoja Noble, author of Algorithms of Oppression: How Search Engines Reinforce Racism, offers a curated reading list exploring how technology can replicate and reinforce racist and sexist beliefs, how that bias can affect everything from health outcomes to financial credit to criminal justice, and why data discrimination is a major 21st century challenge.

Google Has a Striking History of Bias Against Black Girls

Safiya Umoja Noble • Time

My first encounter with racism in search was in 2009 when I was talking to a friend who causally mentioned one day, “You should see what happens when you Google ‘black girls.’” I did and was stunned. (An excerpt from Algorithms of Oppression.)

Machine Bias

by Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner • ProPublica

There’s software used across the country to predict future criminals. And it’s biased against blacks.

There Is a Racial Divide in Speech-Recognition Systems, Researchers Say

Cade Metz • The New York Times

Technology from Amazon, Apple, Google, IBM and Microsoft misidentified 35 percent of words from people who were black. White people fared much better.

Technology Can’t Fix Algorithmic Injustice

Annette Zimmermann, Elena DiRosa, and Hochan Kim • Boston Review

We need greater democratic oversight of AI not just from developers and designers, but from all members of society.

Why Algorithms Can Be Racist and Sexist

Rebecca Heilweil • Recode

A computer can make a decision faster. That doesn’t make it fair.

Social Inequality Will Not Be Solved by an App

Safiya Noble • Wired

We need more intense attention on how artificial intelligence forestalls the ability to see what kinds of choices we are making.

Racial Bias Skews Algorithms Widely Used to Guide Care from Heart Surgery to Birth, Study Finds

Sharon Begley • STAT

Algorithms used by hospitals and physicians to guide the health care given to tens of millions of Americans are shot through with implicit racism that their creators are often unaware of, according to a study.

Can Racist Algorithms Be Fixed?

Beth Schwartzapfel • The Marshall Project

A new study adds to the debate over racial bias in risk assessment tools widely used in courtrooms.

Human Biases Are Baked into Algorithms. Now What?

Marketplace

The Apple Card debut stumbled as good credit scores didn’t help boost womens’ credit limits.

15 Books by Black Scholars the Tech Industry Needs to Read Now

UCLA Center for Critical Internet Inquiry

From the way search engines reinforce racism to how digital culture has reshaped the way Black identity is expressed.

The Underworld of Online Content Moderation

Isaac Chotiner and Sarah T. Roberts • The New Yorker

How the work of content moderation takes a psychological toll.

Videos

Black Future Month

Sasheer Zamata • Full Frontal with Samantha Bee

Is the future more Black Panther or Black Mirror? Guest correspondent Sasheer Zamata investigates Afrofuturism and the journey to make Snapchat filters actually work.

Before Algorithms: A History of Bias and Oppression in Computing

Harvard Kennedy School

A conversation about the underlying systems of power, privilege, and oppression as they relate to present-day conversations about algorithms.

Ruha Benjamin on ”The New Jim Code? Race, Carceral Technoscience, and Liberatory Imagination“

Othering & Belonging Institute

From everyday apps to complex algorithms, technology has the potential to hide, speed, and even deepen discrimination, while appearing neutral and even benevolent when compared to racist practices of a previous era.

Surveillance and the Public Sphere: Confronting a Democratic Dilemma

LSE’s Media Policy Project

The increasingly precise segmentation and targeting of commercial messages has been enabled in large part through the analysis of massive amounts of transaction-generated-information.

Technologies of Control and Our Right of Refusal

Dr. Seeta Peña Gangadharan • TEDxLondon

Most of us don’t realize how much digital systems govern access to our basic public services, like education, health and housing. Even more terrifying is how much data is collected about us and used without our knowledge.

Examining Black Feminism in the Digital Era

Kishonna L. Gray • The Berkman Klein Center for Internet & Society

It is important to examine the digital manifestations of misogynoir – or what it means to be a Woman of Color existing in the hegemonic spaces of digital technology. But our conceptual frameworks fail to capture the everyday practices that Women of Color exhibit online.

About the curator

Safiya Umoja Noble is an Associate Professor at the University of California, Los Angeles (UCLA) in the Department of Information Studies, where she serves as the Co-Director of the UCLA Center for Critical Internet Inquiry. She is the author of a best-selling book on racist and sexist algorithmic bias in commercial search engines, Algorithms of Oppression: How Search Engines Reinforce Racism (NYU Press).

Get fascinating articles like these directly in your inbox.

Join millions and sign up for Pocket’s daily newsletter.