By submitting User Content, you grant us an unlimited license to use your content in any way we choose.
This is fair enough, we build the software but “the powers that be” eg @orenwolf etc… decide which pieces to run not Discourse.
On a technical level I very much want to improve summaries, they can be very powerful, but also LLMs hallucinate, this is something that really irks people a lot, I understand it. It is an imperfect system.
Also worth noting, none of your data is being used for training here, we are running the mixtral LLM on our own hardware to power summaries, AI based search and AI helper in the composer.
All of these features can be indevidually disabled by BBS admins.
Y’know, I’m pretty sure I already addressed that here:
Unlike a lot of folks, Idid actually read the ToS when I joined. It just reflected different technical capabilities back then, and that was my main point.
That is the nature of my gripe, yes: a ‘hallucinating’ program badly misinterpreting posts and possibly causing even more miscommunication.
Doing summaries accurately on BBS topics is a GPT-5 level problem there are tons of tangents, stuff goes in lots and lots of directions. Even a human would have a hell of a time summarizing.
I would totally understand it if @orenwolf disabled summaries here for a few more months while we refine things.
Plenty of the AI features we rolled out here are huge net benefit…
In this case @Falco1 specifically reached out to ask for us to be a testbed for this feature, and as we have a decade-plus tradition of helping the Discourse team test and experiment with features on the BBS once they reach “beta”, I’m more than happy to oblige. I would much rather see this feature become more useful by becoming more useful to the BBS first, because we helped test it, after all.
Our main problem is that for every topic in the boing category we don’t have access to the OP. That means we are missing the most important part. Changing this requires a big change to the way this blog post comments integration work, but it is something we have in the queue.
I’m fine with BB being a testbed because it’s made Discourse a better experience for all stakeholders. That’s precisely why the excellent feedback from @anon85524460, @Melizmatic and others should be seriously considered by the devs.
I’m in full agreement with them that some equivalent of a “no robots” tag for AI should be implemented on a per-user basis for all comments by that user (topics created by a user excepted). I assume that the “AI” would be smart enough to spot such a tag in a post’s metadata and follow a prompt to exclude it from the summary. If or when it’s done, the AI Summary functionality on BB BBS could be reset based on the new filtered corpus.
Doing so would help everyone, including the devs who’'d continue to cement Discourse’s reputation as a BBS platform with features designed to actually respect users (a rare thing in the tech industry).
Right?! This is what I meant with my post in that other thread. (Thanks for the follow-up, BTW.) The getting it wrong part bothers me most. And there are lots of ways here to get it wrong.
The BBS community relies heavily on context, rhetorical devices, experience, familiarity with other posters, etc etc. I tend to short comments myself, but many comments are complex, nuanced, thoughtful. Is it honest to throw away the bulk of posters’ efforts in order to (mis)represent them in a single sentence?
Sarcasm is not always marked with an “/s” or an emoji. Does the AI “know” and account for this? I think we’re going to see some examples of Poe’s Law in these AI summaries.
User121 shares a graphic of their Wordle result, where their grid matches their avatar.
Or at least, in my opinion it bore a resemblance to my avatar. It’s hard to tell whether the AI is reporting on what I posted, or giving its own opinion.
One thing I’m curious about [did I miss it here?] is: how does the AI pick and choose who or what to mention? Will it wind up boosting certain viewpoints or users?
Also, I noticed in the Wordle topic summary, it “summarizes” up to User1242, but there are 4741 replies in the topic. So it seems to stop a bit short.
This is kind of what I’ve noticed in using AI at work as a software engineer. Yes, it works. To a point. Where that point is is indiscriminate in literally every single case, and thus you need to , as a professional using it as a tool, go back and validate it’s work from beginning to end. Use the parts that are correct, fix the parts that aren’t, or toss the whole thing if it’s not helpful. (AI does awful with optimized code… the amount of times I’ve seen O(n^2) or even O(n^n) functions is high) So, for something like summarizing long threads, it almost defeats the purpose because if it’s only 70% accurate and 30% hallucinatory, you have to read 100% of the thread to find out what it got right and what it got wrong.
there are tons of tangents, stuff goes in lots and lots of directions. Even a human would have a hell of a time summarizing.
