The Hidden Costs of Automated Thinking / Sci-Fi and an Uncertain Future / Past futures / ‘Mind-blowing potential’ to tackle climate crisis — No.87

Welcome back! A few admin notes to begin; I redid the Sentiers homepage with some new copy and more emphasis on the idea of Thought Partnership, please have a look if you’re curious. I also wrote an article about how I see that practice. There’s also a new version of the patron support page, if you are getting value from this weekly newsletter, please consider sharing with your friends and colleagues or joining our group of patrons.

I read three books over the break. As you can see below, I accidentally didn’t stray far from Sentiers topics, all three are stories about a “jackpottish” near future and all three are pretty bleak. They are also all recommended and all by interesting people doing excellent non-fiction work as well, decrypting our current situation and futures.

  • Infinite Detail, Tim Maughan. Gibsonian (sorry!) in topics and in the pleasure of reading as well as the feeling of being there.
  • Autonomous, Annalee Newitz. Going on the Cory Doctorow – Rudy Rucker shelf. Fun and makes you think. (Also interesting for the uncommon Canadian settings.)
  • Everything Belongs to the Future, Laurie Penny. ”Time is a weapon wielded by the rich, who have excess of it.”

Since I caught up on three weeks of readings, this is a pretty hefty issue. The miscellany section is especially strong this time around. To read even more, head over to during the week, where I’ll be doing a second stint as guest editor (actually, a few articles in this issue come from Jason).

The Hidden Costs of Automated Thinking

It’s the first time I’ve seen an argument about the blackboxes of AI presented this way. Jonathan Zittrain makes a parallel with drug discovery. In many cases, we don’t now why they work, or we didn’t for a long while. He calls this intellectual debt and frames the AI blackboxes in the same way; what we don’t know about how they work is a debt, and the missing understanding opens the door to clashes with reality, abuses, and complex interactions and consequences.

This approach to discovery—answers first, explanations later—accrues what I call intellectual debt. It’s possible to discover what works without knowing why it works, and then to put that insight to use immediately, assuming that the underlying mechanism will be figured out later. […]

And yet, most machine-learning systems don’t uncover causal mechanisms. They are statistical-correlation engines. They can’t explain why they think some patients are more likely to die, because they don’t “think” in any colloquial sense of the word—they only answer. As we begin to integrate their insights into our lives, we will, collectively, begin to rack up more and more intellectual debt. […]

But intellectual debts don’t exist in isolation. Answers without theory, found and deployed in different areas, can complicate one another in unpredictable ways. […]

Intellectual debt can accumulate in the interstices where systems bump into each other, even when they don’t formally interconnect. Without anything resembling a balance sheet, there’s no way to determine—either in advance or retrospectively—whether any particular quantity of intellectual debt is worth taking on. […]

A world of knowledge without understanding becomes a world without discernible cause and effect, in which we grow dependent on our digital concierges to tell us what to do and when.

More: I’m definitely article around for future use and attaching it with this favourite from last year; Half Life: The Decay of Knowledge. Some of those changes in our knowledge are (or result from) payments of intellectual debt.

Can Sci-Fi Writers Prepare Us for an Uncertain Future?

Rose Eveleth on the difference between scifi writers producing futures scenarios for clients, and professional futurists / forecasters doing “the same.” Good overview and includes quotes from some smart people with hybrid profiles.

But true Futurism is often pretty unsexy. It involves sifting through a lot of data and research and models and spreadsheets. Nobody is going to write a profile of your company or your government project based on a dry series of models outlining carefully caveated possibilities. On the other hand, worldbuilding—the process of imagining a universe in which your fictional stories can exist—is fun. People want stories, and science fiction writers can provide them.

More: Eveleth also mentions in passing a piece by Doctorow, Fake News Is an Oracle, which includes this quote on scifi as ouija board.

In the same way, science fiction responds to our societal ideomotor responses. First, the authors write the stories about the futures they fear and rel­ish. These futures are not drawn from a wide-open field; rather, they make use of the writer’s (and audience’s) existing vocabulary of futuristic ideas: robots, internets and AIs, spaceships and surveil­lance devices. Writers can only get away with so much exposition in their fiction (though I’ve been known to push the limits) and so the imaginative leaps of a work of fiction are constrained by the base knowledge the writer feels safe in assuming their readers share.

Also: This is the Muslim tradition of sci-fi and speculative fiction.

Past futures / participatory panopticon revisited

Paul Graham Raven thinking about his practice and quoting Jamais Cascio’s work. A very interesting take on “black-sky thinking,” tactical foresight, and generally a more critical version of futurism. Also including it as a counterpoint to some of the rosier visions I’ve linked to before, and because his “scout” angle at the end is adjacent to my view of Thought Partnership.

Cascio was one of the first people I can recall reading who was doing what I think of as “black-sky thinking” – contemplating the darker possibilities of sociotechnical change, in a way that seemed to me to combine the best and most interesting aspects of sf worldbuilding along with the real-world critique that I was slowly coming to see as an urgent political project in reality. […]

“Foresight (forecasts, scenarios, futurism, etc.) is the most useful when it alerts us to emerging possible developments that we had not otherwise imagined. Not just as a ‘distant early warning,’ but as a vaccination. A way to become sensitive to changes that we may have missed.”

Tree planting ‘has mind-blowing potential’ to tackle climate crisis

Not as clear cut as the title might lead us to believe. How would such a massive project be paid, by whom, how long it would take, how truly doable is it, etc. All important questions but at least it’s something we know works, the questions are more around scale and speed.

The study, published in the journal Science, determines the potential for tree planting but does not address how a global tree planting programme would be paid for and delivered. […]

The research is based on the measurement of the tree cover by hundreds of people in 80,000 high-resolution satellite images from Google Earth. Artificial intelligence computing then combined this data with 10 key soil, topography and climate factors to create a global map of where trees could grow.

Artificial intelligence and art at Exponential View

Marko Ahtisaari, subbing for Azeem, looks at AI and art. Noting for this quote which mirrors other articles and opinions of mine in previous issues.

Art and AI is a much-hyped, poorly understood and little experienced area. The breakthroughs will come, I believe, from the centaurs, the artist(s) working together with non-human intelligences, not machines emulating styles or replacing human artists.

And for this great one in the “Short morsels” section:

Many US-educated students from China are headed back East, claiming they can move faster and break more things in Shenzhen than in Silicon Valley.

The “they looked promising but I ran out of time and besides this thing is already long enough” section: My manifesto for a post-carbon future (Paul Mason), Was the Automotive Era a Terrible Mistake? (I’m going to go with yes), Networked Dream Worlds (5G at Real Life), and The Desire for Full Automation (I actually read a good chunk of this one and it’s interesting, but it’s a 20 min read and I felt I had to re-read to even begin to compress it, but have a look.) Oh, also Writing the Future With Utopias and Building Utopia in Space (on Fred Scharmen’s book).