Awful AI: When AI Crosses the Line

Awful AI is an open repository that collects real-world examples of artificial intelligence technologies that do the worst. Things like excessive surveillance, unfair decisions, or systems that treat some people worse than others.
The idea is simple: to show what’s going wrong to prevent it from continuing to happen.

The problem: machines making decisions without people knowing

More and more, schools, companies, hospitals, police departments, banks, and even social media platforms are using artificial intelligence in their decision-making.
The problem? Most of the time, people don’t know this is happening.

For example, facial recognition AIs or predictive criminal justice systems may have racial or gender bias. When the code and data are closed-source, there is no way to audit, challenge, or remedy them. This amounts to a new form of “algorithmic governance”—automated decisions that regulate people’s lives.

Why does this happen?

Many of these technologies are created by companies that do not disclose their code, do not share their data, and do not explain how decisions are made.
Others are purchased by public or private institutions as “black boxes”: they work… but no one can see what’s inside.

And what are the impacts?

The problems aren’t theoretical; they happen in everyday life:

• Injustice: systems that treat certain people worse (based on gender, neighborhood, skin color, or accent).

• Loss of opportunities: algorithms that deny credit, insurance, or employment.

• Surveillance: cameras that track people for no reason.

• Stress and confusion: feeling that “something” is making decisions for us, but without knowing how. It’s like living in a place where there are rules, but no one tells you what they are.

Already marginalized groups feel the worst effects: algorithmic biases turn inequalities into automated decisions. Opacity is not neutral;
it perpetuates injustices that already existed.

What does Awful AI do?

Awful AI is not a “magic tool,” but it serves as an infrastructure for visibility and debate—an open repository, free from commercial biases, that documents and exposes AI abuses. This is important in and of itself: it brings to light what is usually concealed. Transparency is the first step toward accountability.

The fact that it is open means that anyone—researchers, journalists, activists—can consult the list, examine the cases, follow up on reports, conduct studies, and put pressure on institutions. This empowers communities outside the corporate-technological sphere.

Furthermore, Awful AI inspires what the authors call “contestational tech”—that is, technologies of contestation and resistance: since the problem exists, it is urgent to consider counterforces. Although the repository itself does not provide a ready-to-use alternative system, it serves as a critical knowledge base and a starting point for those who want to envision AIs that are fair, regulated, auditable, and aligned with the common good.

This function is essential: to challenge the assumption that AI is inevitable, and to reaffirm that technologies are social and political choices that depend on ethical decisions.

How this helps those who develop civic technology in Portugal

Although Awful AI is not Portuguese—and the documented cases often come from the United States, Europe, or the Global North—its role as critical infrastructure is valuable in any context where AI and automation are on the rise.

In Portugal, as throughout Europe, public and private initiatives are beginning to adopt automated systems: facial recognition, administrative decisions based on algorithms, and large-scale analysis of personal data. Without oversight, debate, and alternatives, we will repeat the same abuses—segregation, opacity, algorithmic discrimination, and mass surveillance.

For advocates of free software, open data, and open governance, Awful AI serves as a valuable tool for raising awareness, documentation, and mobilization.

Technology we can trust because we can see how it works

The repository helps us realize that there are better ways:

• Open-source software: we can view and contribute to the code

• open and well-documented data: we know where decisions come from

• community tools: designed to help, not to monitor

• low-tech or hybrid technology: simple, yet transparent

• processes that are open to review and correction.

Opacity is not neutral; it perpetuates injustices that already existed

Links

daviddao/awful-ai: Awful AI - 2021 Edition (DOI 10.5281/zenodo.5855971)