Artificial Intelligence: How the Internet’s Gatekeeper Could Affect Your Civil Rights

Lindsay Nako, Impact Fund Executive Director

In the modern world, artificial intelligence goes far beyond ChatGPT – it curates our entire online experience. Which song will I want to listen to next? Is an email important or SPAM? What TikToks show up on my feed? What ads will interest me? Artificial intelligence prevents us from being inundated with irrelevant information – and that raises a important questions.

Who determines what is relevant or irrelevant?  And how do they decide?

If an artificial intelligence program delivers me a song by Chopin instead of Bad Bunny, it might be surprising.  But if an artificial intelligence program delivers me a job ad for administrative assistant and prevents me from seeing one for car mechanic, that could be illegal.  

Artificial intelligence, the science and engineering of making intelligent machines, combines algorithms (unambiguous instructions) with data to perform functions similar to human decision-making. The first generation of artificial intelligence in the 1980s and 1990s could apply rules written by humans to data to create outputs. The second generation in the 2000s could “learn,” meaning that programs could take data and guidance provided by humans, independently identify rules, and then apply those rules to new data to create outputs. The third generation of artificial intelligence to incorporate “deep learning.”  Deep learning permits programs to autonomously learn rules and automatically judge new data to create outputs, without human intervention. The fourth generation –  generative AI such as ChatGPT – can ‘talk’ to us like another person would and has been trained on the sum total of humanity’s written record.

AI-based products may screen out vulnerable groups – people of color, women, people with disabilities, older people, members of the LGBTQ community

Over the past ten years, products have been emerging that incorporate artificial intelligence to streamline processes for delivering employment and housing advertisements, identifying resumes with relevant experience, interviewing job candidates, evaluating tenant applications, and more.  These products range from first-generation AI, scanning documents for relevant words or background records for criminal convictions, to more advanced models, such as those purporting to use video analysis to identify “trustworthy” or “enthusiastic” candidates.  

Many of these products advertise their use of artificial intelligence as a way to avoid human bias, but minimizing the role of humans does not guarantee equitable results. Artificial intelligence prioritizes user preference, while our civil rights laws prioritize equality of opportunity.  In the quest for compatibility, AI-based products may screen out vulnerable groups – people of color, women, people with disabilities, older people, members of the LGBTQ community – and run afoul of the very laws they claim to satisfy.  Our civil rights laws prohibit targeting advertisements and making employment or housing decisions based on personal characteristics.  This would also prohibit use of these characteristics as relevant data for AI decision-making.

Reuters and others reported an AI failure by retail giant Amazon in 2018.  After multiple years of building machine-learning computer programs to evaluate resumes and identify top candidates, Amazon reportedly abandoned the project because it could not create a program that was able to evaluate candidates for technical positions without disadvantaging women.  For example, Reuters reported that the system penalized resumes that included the word “women’s” and downgraded graduates of all-women’s colleges.

Deep learning will permit programs to autonomously learn rules and automatically judge new data to create outputs, without human intervention.

Or it may be as simple as a platform allowing advertisers to select the users it wants to see ads for housing, credit, and employment opportunities.  A lawsuit brought by the ACLU and others challenged Facebook’s use of ad targeting based on gender, age, zip code, and other demographic information.  The settlement reached in 2017 removes gender, age, and multiple demographic proxies from ad targeting options.  

Since this piece was written in 2020, a new generation of AI tools has entered the mainstream. Systems like ChatGPT, Google's Gemini, and others, built on a technology called large language models, or LLMs, have become embedded irreversibly into our lives. Millions of people now use these tools daily. This has opened the door to an even greater risk of widespread discrimination.

LLMs base their outputs on historical data, and history is not neutral. Left unchecked, LLMs have the potential to replicate and amplify the same patterns of discrimination in hiring, lending, healthcare, and housing that civil rights law has spent decades trying to dismantle. A biased algorithm embedded in a widely used platform can affect millions of people before anyone raises a flag. That is why AI transparency and oversight are more critical than they’ve ever been before.

At this point, it is impossible to imagine our society without artificial intelligence.  But we also have a shared responsibility to ensure that our principles of fairness, equity, and inclusion remain at the forefront of our collective advancement.  Technology provides an opportunity to generate greater visibility and access for historically disadvantaged communities.  We cannot let that opportunity wither in the black box of artificial intelligence.

--

Many thanks to Professor Niloufar Salehi, UC Berkeley School of Information; Professor Catherine Albiston, UC Berkeley School of Law; Christine Webber, Cohen Milstein Sellers & Toll; and Galen Sherwin, ACLU Women’s Rights Project for our discussion on “Artificial Intelligence & the Future of Discrimination” at the 2020 Impact Fund Class Action Conference. 

This blog post, originally published in March of 2020, was updated and republished in March of 2026.