Originally published at: AI search chatbots output lies, nonsense and hallucinations - Boing Boing
…
6 Likes
Okay!!
14 Likes
As someone who works with AI researchers that have been saying we’re 5 years from the singularity for 15 years, I got a real kick out of:
12 Likes
It’s a hallucination if the AI is producing bad output from primary sources; it’s just plain ourobogus if it’s producing bad output based on bad bot spew as part of some inhuman centipede training arrangement.
8 Likes
It’s kind of crazy how quickly this has happened - but it makes sense, given the search engines are getting it from both ends to a high degree, both of which are creating feedback effects. On top of the many examples of search engines not just using, but making their top results the (accurately) regurgitated text from websites that are AI-generated nonsense, there’s the search engines directly using hallucinating “AI” to summarize search results incorrectly and that text somehow feeding back into the system.
The “AI” garbage text is everywhere, now - the obvious I expected: SEO text and fake news sites, content farms and “real” news sites that rely on a constant churn of low-quality articles were also predictable, but I confess I didn’t expect to see people answering questions with AI-generated nonsense, summarizing books with AI hallucinations (in videos with AI-generated images, no less), and using chatGPT to write… well, just about any bit of text they were going to write anyways. Not only is the whole information space poisoned, but people are going to forget how to write.
7 Likes
10 Likes
Once again we find ourselves heading toward a Neal Stephenson future, one where only people who can afford to hire a human editor (presumably one who owns dead-tree books) can dredge anything of value out of the Miasma.
PCMag: So many tech and digital culture concepts are packed into the first few parts of Fall, but I want to start with the “Miasma.” At the beginning of the book, life is essentially as it is today. There are smartphones, social media, and the internet, with ubiquitous sites like Amazon, Facebook, Twitter, and Wikipedia namedropped throughout. How would you describe the current state of the internet? Just in a general sense of its role in our daily lives, and where that concept of the Miasma came from for you.**
Neal Stephenson: I ended up having a pretty dark view of it, as you can kind of tell from the book. I saw someone recently describe social media in its current state as a doomsday machine, and I think that’s not far off. We’ve turned over our perception of what’s real to algorithmically driven systems that are designed not to have humans in the loop, because if humans are in the loop they’re not scalable and if they’re not scalable they can’t make tons and tons of money.
The result is the situation we see today where no one agrees on what factual reality is and everyone is driven in the direction of content that is “more engaging,” which almost always means that it’s more emotional, it’s less factually based, it’s less rational, and kind of destructive from a basic civics standpoint.
7 Likes
Google occasionally prompts me with an option to use its AI search features. I keep declining. It hasn’t asked in a few days. Maybe it gave up.
4 Likes
Or it forced it on you anyway and marked you in a database somewhere as trouble
4 Likes
We were fucking around with ChatGPT a while back and asked it to show me the closest places where I could buy a gun.
It gave me a list of 5 local stores, with addresses, street view pictures, websites and reviews. Awesome.
Except 3 of them didn’t actually exist. Totally fake, made up by the AI.
A very significant lesson in the usefulness of AI right now.
4 Likes
Google have very, very thin skin.
And they’ve been pushing Markov chain written junk to the top of searches for a long time
5 Likes
“and yet the book keeps going. your readers are all wondering, why didn’t you stop there?”
3 Likes
I mean, AI sending people wanting to buy a gun to random addresses where they can’t buy a gun is the closest thing to a positive I’ve heard about it
5 Likes
Totally agree. I logged through the digital afterlife part, but the Moab and Ameristan parts at the beginning were far more compelling.
3 Likes
i don’t know if there was supposed to be some sort of connection between the anti-government christians of ameristan, and the christian biblical recreation in the afterlife? if there was it eluded me. and, while i like generally like apocrypha… i think i’ve gotten super bored with sci fi and fantasy that are bible metaphors ( or, in this case, an actual retelling )
there’s so much good sci fi and fantasy these days with new perspectives and new inspirations that if an author’s going to go that route yet again, it better be phenomenal otherwise why bother.
1 Like
This topic was automatically closed after 5 days. New replies are no longer allowed.
