The Hidden Biases in AI

The Hidden Biases in AI

Written by Vivian Wang
Illustrated by Amy Zhang

From customized Instagram ads to video games, artificial intelligence (AI) plays an increasingly powerful role in our everyday lives. When we think of these algorithms, it’s easy to picture them as complex mathematical functions that make automated, impartial decisions for us. However, since humans are inevitably biased, AI systems can easily pick up biases from the people who build them and the processes through which they are developed and used.

One major contributor to AI bias is the data we “train” algorithms to make decisions with. A recent study in Science showed significant racial bias in an algorithm widely used by US healthcare systems. The researchers found that for white and Black patients assigned the same level of risk by the algorithm, the Black patients are usually sicker. This disparity reduces more than half of the number of Black patients identified for extra care. Where does this bias come from? Turns out, the algorithm predicts healthcare costs using past records, and less money is usually spent caring for Black patients than for white patients. So, the bias roots from the ongoing issue of unequal access to healthcare; this is just one of many AI systems that learn to make decisions based on data that reflect historical and social inequities.

For consumers who use algorithms all the time, it can be difficult to tell whether any biases are at play. Luckily, many researchers are trying to raise awareness of the hidden biases in AI, and even analyze existing datasets to reengineer algorithms that produce fairer results. In 2012, a project called ImageNet curated a vast visual database (with over 14 million labeled images!) that programmers could use for free to train image-recognition algorithms. However, some researchers noticed bias patterns within the data. For example, an algorithm trained with ImageNet data might identify programmers as white men, reflecting the majority of the pool of images labeled “programmer.” Moreover, some images have been found to be associated with slurs and derogatory terms. The ImageNet team has since analyzed the data set to track down and remove sources of bias. The researchers are also working on a new tool to diversify the database. The term “programmer,” for instance, would produce images of greater diversity in terms of gender, race and age. While debiasing algorithms may be a tricky process, experts believe these efforts will help reduce discrimination by AI bias.

What else can we do to limit the harmful effects of AI bias? As tech companies continue to introduce new algorithms, policymakers are trying to catch up. Facial recognition, employment discrimination, and web searches are just a few areas that governments aim to study and regulate with both existing and new laws.

At the end of the day, algorithms are only as biased as we allow them to be. By combining technical and legal efforts, we can further recognize and reduce the human biases that creep into AI technology.


  1. Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science366(6464), 447–453. doi: 10.1126/science.aax2342