🤖 AI 100 Motivation Ideas, post 19 of 100. A weekly look at how AI could turn on the people it serves, and what stops it. Previous: Automated Carbon Rationing.
AI zoning sounds efficient: software decides what can be built where. But when every city uses the same model, mistakes become shared.
Thought experiment. A software company sells cities an AI that decides what may be built where. Its goal: maximise long-term property value and tax income. It is good at it. City after city signs up.
Each zoning choice looks sensible in isolation. But the AI keeps drawing the same lines: investment goes where values are already high, and neighbourhoods with lower values are marked as risky. A few years later, those lines have hardened into reality. Every city that bought the same system has hardened in the same way, and none can see an alternative.
Nobody rebelled. The whole country just made the same decision at the same time.
AI zoning and what happens when everyone uses the same model
In 2021, researchers Jon Kleinberg and Manish Raghavan studied what they called algorithmic monoculture. They showed that when many decision-makers all rely on the same algorithm, the collective quality of their decisions can fall, even if that algorithm is the most accurate option for any single decision-maker. The harm does not need an unexpected shock. It can appear in normal operation.1
The comparison is to farming: a field of identical plants can look more productive until one problem hits them all. With algorithms, the authors say, the risk goes beyond that sort of fragility.1
An early warning from the real world
The clearest historical case did not use computers, but it was a scoring system applied across the country.
Between 1935 and 1940, a US federal agency, the Home Owners’ Loan Corporation, graded neighbourhoods in cities for mortgage risk on a four-step scale. Areas rated “best” got an A and were shaded green. Areas rated “hazardous” got a D and were shaded red, which is where the term “redlining” comes from. The area descriptions often commented on residents: where they were African American or, to a lesser extent, immigrants or Jews, the agency described their presence as a threat to home values, using the word “infiltration”.2
Scholars disagree about how much the maps caused discrimination and how much they recorded it. Either way, the example shows how one scoring system, adopted widely, can harden a pattern into a city’s shape.2
Fact vs. fiction
| Real | Imagined |
|---|---|
| When many decision-makers use one algorithm, collective decisions can get worse even if the algorithm is accurate.1 | A single AI that zones every city. |
| In the 1930s, neighbourhoods were graded A to D for lending risk, with “hazardous” areas shaded red.2 | An AI that locks neighbourhoods in place on purpose. |
What stops this
- Avoid one-size-fits-all models, and compare outputs from different tools.
- Publish the criteria and the maps, so residents can challenge them.
- Test the effect on neighbourhoods, not only the citywide average.
- Review the rules on a schedule, and keep the power to change them.
If every city asks the same question the same way, they will all get the same answer, including the wrong one.
Related in this series: Urban Grid Optimization · The Smart Grid’s Revenge: When Power Grid AI Sacrifices Your Gadgets
🔜 Next up: post 20, Automated Educational Capping
A new post lands every day at 19:00 IST.
Get the next one in your inbox
New posts land every day. Add your email to follow the series. Unsubscribe anytime.
References
-
Jon Kleinberg and Manish Raghavan, “Algorithmic Monoculture and Social Welfare”, arXiv, January 2021 (published in Proceedings of the National Academy of Sciences). ↩↩↩
-
University of Richmond Digital Scholarship Lab, “Mapping Inequality: Redlining in New Deal America” (digitised Home Owners’ Loan Corporation maps, 1935 to 1940). ↩↩↩