Discussion about this post

User's avatar
Kit Noussis's avatar

It strikes me that the two stories from this post have a link. Banks' ship-minds are eminently rational, yet they have just enough quirks to have (mostly peaceful) disagreements. If I recall correctly, they are utilitarian, often directing the morally gray 'Special Circumstances' agents through grisly missions where the ends justify the means. Ship-minds are the benevolent caretakers of the humans in the Culture, but they occasionally convince adventurous people to take massive personal risks to further their grand civilizational plans.

As you say, the only forms of violence that can hurt now are the very large and very small scale. These too-online bombers see themselves as a sort of SC agent for their message-boards, who can be seen as the 'minds' calling the shots. In a corresponding manner, I think Elon sees himself as a sort of ship-mind: he is shepherding us into the better future he envisions for us, and deciding which precious cargo can be jettisoned to get us there faster. Picard, at least, would detach the civilians from the officers before making a risky charge.

If your goal is truly to get us into AI anarcho utopia, then be a bit more rational: you will certainly die before glimpsing anything remotely resembling Banks novels. We should focus less on the Culture and more on the primitive, flawed societies in his books, because they are much closer to what we are dealing with.

Expand full comment
Robert Polik's avatar

A great book on "Radical Utilitarianism" of a different strain is Strangers Drowning by Larissa MacFarquhar. The title might make it sound like it's all about Peter Singer and Effective Altruism, and these concepts do get name-checked throughout the book, but it's mostly profiles of the lives of extreme altruists and how they arrived at their worldview.

Profiling these individuals back to back to back makes their similarities really clear: they all share a worldview that definitely matches your phrase "Radical Utilitarianism." Unlike the Silicon Valley effective altruists who basically see utilitarianism as one big linear optimization problem, these extreme altruists seemed to have over-developed skills of abstract thought. One example I still remember is that three different profiles mentioned the subjects stopped believing in God when they realized that so much of the world followed some other religion, and there was no reason to believe one was "correct" over the others.

I feel like EA zealots, the extremists described in your post, or these extreme altruists all think about the world in an abstract manner that's strange and off-putting to most people. The book does a great job of exploring worldview while profiling extremely interesting, and often admirable people. Abstract thought like this is more accessible than ever, and not just on forums.

I picked up the book because it inspired the "Bad Art Friend" situation several years ago, I definitely didn't think I'd be talking about it in contexts like this.

Expand full comment
27 more comments...

No posts