AI search chatbots output lies, nonsense and hallucinations

Once again we find ourselves heading toward a Neal Stephenson future, one where only people who can afford to hire a human editor (presumably one who owns dead-tree books) can dredge anything of value out of the Miasma.

PCMag: So many tech and digital culture concepts are packed into the first few parts of Fall, but I want to start with the “Miasma.” At the beginning of the book, life is essentially as it is today. There are smartphones, social media, and the internet, with ubiquitous sites like Amazon, Facebook, Twitter, and Wikipedia namedropped throughout. How would you describe the current state of the internet? Just in a general sense of its role in our daily lives, and where that concept of the Miasma came from for you.**

Neal Stephenson: I ended up having a pretty dark view of it, as you can kind of tell from the book. I saw someone recently describe social media in its current state as a doomsday machine, and I think that’s not far off. We’ve turned over our perception of what’s real to algorithmically driven systems that are designed not to have humans in the loop, because if humans are in the loop they’re not scalable and if they’re not scalable they can’t make tons and tons of money.

The result is the situation we see today where no one agrees on what factual reality is and everyone is driven in the direction of content that is “more engaging,” which almost always means that it’s more emotional, it’s less factually based, it’s less rational, and kind of destructive from a basic civics standpoint.

7 Likes