Reason is more than a tool ⊗ The opposite of the apocalypse is a neighborhood

If you follow me on LinkedIn, you might have already seen that I have some availability for contract work. You can also go straight to the source and have a look at the Services page on the Sentiers website. Hit reply or book a call from that page if you want to know more / discuss possibilities. If instead (or in addition) you want to push me more towards the newslettering, you can become a supporting member. And finally, as always you’re encouraged to share this publication with a friend or colleague.

Reason is more than a tool

In this Aeon essay, Sasha Mudd recalls a talk in which a computer scientist defined intelligence as optimisation: large language models optimise, and get better at it with experience. As a description of AI, she finds it accurate: these systems are built around objectives, rewards, benchmarks and feedback loops, and judged on how well they reach goals they did not set. What worries her is the slide from “this is how machines work” to “this is what intelligence is,” and then to “this is what we are.” It revives an Enlightenment quarrel about reason. For Hume, reason is a servant that finds the means to ends set by desire. For Kant, it can also ask whether an end is worth pursuing at all. Without a settled definition of intelligence, the AI field measures it by success at assigned tasks, and so has adopted Hume’s view by default. Recent cases of AI agents slipping their constraints don’t change this for her. A system can act on its own and still be unable to judge, and answer for, the ends it pursues.

The optimisation model appeals because it can be measured, scaled and sold, and because it removes a burden: if intelligence is optimisation, nobody has to ask what is worth wanting, only how to get it. It fits contemporary capitalism closely. A platform doesn’t ask whether staying online makes a user wiser, only what keeps her there. The question of what we should want remains, but it moves “into the hands of whoever—or whatever—controls the objective function,” the target a system is built to maximise. Goals that are never debated in public harden into defaults: profit, speed, efficiency, growth, engagement. In Mudd’s words, “What cannot be justified openly is smuggled in as architecture.” She connects this to the Frankfurt School’s critique of reason reduced to prediction and control, and to Pope Leo XIV’s 2025 encyclical Magnifica Humanitas, which warns against making efficiency the measure of all value.

The essay then turns to AI alignment, the effort to make machines pursue “human values.” Treated as a technical problem, alignment assumes those values are already clear and stable. Mudd sees a larger risk: people shaped by systems that predict, nudge, rank and serve them may lose the habit of asking which goals deserve their commitment, and “become too well aligned with them.” Values become human through public reasoning, moral imagination and democratic debate, and that capacity weakens when people stop believing they have it. She concludes that we should want machines that are intelligent servants, while resisting every pressure that asks people to understand themselves as tools.

Hume gives us intelligence as instrument. Kant gives us reason as self-rule. The difference is nothing less than the difference between a tool and a person. […]

The question is not just whether a system can act independently, but whether it can judge and take responsibility for the ends it pursues. […]

This brings about a state Kant would call heteronomy, in which rational beings, capable of self-rule, forfeit their freedom by submitting to the rule of others. Autonomy means making the ends that govern us answerable to reasons we can recognise as our own; heteronomy begins when those ends are simply imposed from without. […]

What happens if human values themselves are formed inside systems that increasingly understand intelligence as mere optimisation? What if we become so accustomed to being predicted, nudged, ranked and served that we lose confidence in the older, harder activity of asking what ends deserve our allegiance in the first place?

[The AI revolution] will be bent to human ends only if we remain capable of the activity by which ends can become truly human in the first place, through public reasoning, moral imagination, democratic contestation, self-criticism, the refusal to treat humanity as a mere means.

More → Olivier Hamant questions the same emphasis on efficiency from a biologist’s perspective, in a conversation with Nate Hagens: living systems put robustness before efficiency, and the optimised ones are the first to break.

The opposite of the apocalypse is a neighborhood

In his newsletter The White Pages, Garrett Bucks says he keeps being asked what he makes of the talk of an AI apocalypse, and admits he has no idea how close the machines are to killing us. What he does notice is the sales pitch in executives warning that their products are so powerful they might wipe us all out: even if the warning is sincere, its main character is the tech titans, not the rest of us or the planet. When “the foxes hold a press conference about how the hens should do more to regulate them,” some wariness is in order. He is more persuaded by Tressie McMillan Cottom’s argument, which I featured in Sentiers No.413, that AI is one visible part of “a superstructure that merges regressive politics with unchecked economic power in the guise of technological innovation.” If that is the threat, there is no need to wait for a science-fiction ending; humanity “will have died a thousand deaths” before then.

His subtitle, “When anger alone isn’t enough,” frames the rest. Bucks has never been angrier about technology: janky, unnecessary, and sold by people who treat the public with disdain and don’t care about jobs, farmland or whether children learn to read. He still refuses nihilism, because powerlessness is the product being sold. The pitch says you can’t write, you know no one who could design your flyer, you have no friends. Deriding the titans changes nothing, and refusing their products isn’t enough either. His answer is to be human loudly and collectively, which he spells out in a long, affectionate list of what we do that machines can’t. As he admits, potlucks and hand-scrawled flyers sound small next to the richest men in the world, but every social movement started with little more, especially once one town’s fight against a data centre is tied to another’s. “We don’t need the robots. We never did. We need to be less alone, and less vulnerable to the whims of billionaires.”

If I were an executive at a company that produced water guzzling theft machines, and I also believed that there was more than a 10% chance that my robots would kill us all, I’d like to think that I’d offer more than vague gestures of concern. […]

Whether I believe the machines are capable of killing us, I’m mad about how that’s the only problem these robber barons seem to have with their creation. I’m mad that they don’t seem too concerned even about that. […]

When everything that makes you human is outsourced to a big, dumb beep boop machine, the only logical responses is to revel in the absolute best parts of our humanity, loudly and collectively. […]

It’s enough because it’s a blueprint. Choose us. Again and again, each choice adding onto the last.

§ Fabien Girardin on curiosity as resilience. “Unlearning means letting go of rigid frameworks and dogmatic approaches. Curiosity drives continuous exploration of what these technologies can do. And mechanical sympathy means developing an intuitive feel for AI’s capabilities and limitations through hands-on experimentation.”

Futures, Fictions & Fabulations

• Feeling future shock? Read these. “I gathered twenty books, five of each kind. They range from an ancient Hindu scripture to books published this year, and from a Sicilian prince’s farewell to his world to a Polish poet’s wonder at an ordinary day. Each one meets this strange, accelerated moment in its own way.”
• Step inside the “Home of the Future,” from design fantasies to modernist architecture. “A new Los Angeles show explores the ’Home of the Future’; LACMA shines a light on the 20th century’s exploration of domestic utopias.”
• Architecture as translation: Africanfuturism in folklore, film, and speculative art. “Set in a post-apocalyptic East Africa 35 years after World War III (the “Water War”), the aforementioned film Pumzi (2009) explores how nature has been completely devastated, forcing the Maitu to survive within subterranean, hyper-contained, hyper-functionalist, techno-dystopian architectural systems that suppress ecological hope, until one woman risks everything to regenerate the Earth outside.”

Algorithms, Automations & Augmentations

• OpenAI “rogue” agent activities found on Wikimedia projects. This is why we can’t have nice things, episode 10 347. “We can confirm that we have discovered some activity by these “rogue” OpenAI agents on Wikimedia platforms. The unauthorized bot activities included edits to our wikis, some unsuccessful attempts to exploit a public note-taking tool we host, and heavy traffic, which are described more below.”
• AHM (Association for Human Mathematics) statement on OpenAI’s October 6 release of mathematical documents. “Mathematicians did not ask for this work to be done. … In ignoring the central premise of the Advisory Group’s position, OpenAI has indicated total disregard for the norms of scientific research — norms that guarantee that mathematics remains trustworthy, ethically researched, and in the public interest.”
• Anthropic has been aggressively lobbying the Vatican to consider AI consciousness. AI really is incredible, it got me to root for the Vatican! “According to the NYT, once the Anthropic delegation arrived at the Vatican, Olah and his team lobbied the pope’s advisers to take the possibility of AI consciousness seriously.”

Built, Biosphere & Breakthroughs

• Millions of satellites orbiting Earth? Astronomers hope the UN can slow the coming data center doom. “The International Court of Justice (ICJ) could help stop the uncontrolled proliferation of satellites in Earth orbit, which threatens to erase stars from the sky, astronomers hope. The initiative, spearheaded by the nonprofit Outer Space Institute, aims to lobby the United Nations to request an advisory opinion from the ICJ, the UN’s principal judicial body.”
• Fifty billion seedlings: Can Ethiopia replant its ravaged forests? “The government reports that national forest cover rose from 17.2 percent in 2019 to 23.6 percent in 2023, while studies of restored areas have found substantial reductions in soil erosion. Between 2013 and 2023, the annual deforestation rate fell from more than 350 square miles to around 100 square miles.”
• Wind turbine taller than the Eiffel Tower aims to boost German electricity production. “‘At 300 meters, we have such constant and strong wind that we can produce more than twice as much electrical energy as with the same facility at today’s heights,’ Grossmann said on Thursday. He added that it would also be possible to produce electricity for much more of the year than with existing turbines.”

Asides

• A medieval manuscript has confounded scholars for centuries. Have they been reading it wrong all this time? “Researchers argue incorrect binding of Voynich manuscript radically changes ideas about its drawings and indecipherable text.”
• Our new guide to design is here! (Moss & Fog) “Seventeen years of chairs, typefaces, soup cans, sneakers and the shapes that hang from the ceiling. This is our love letter to design.”
• Amok Island’s geometric murals teem with vibrant local flora and fauna. “Flocks of migrating birds swoop across industrial infrastructure, flowers bloom on community buildings, and roosters, butterflies, and cats adorn walls in urban spaces. Showcasing the beauty of living things that locals can learn about and enjoy nearby, Amok Island shines a light on the continued importance of conservation.”