There is a lot of strangeness being talked here. Or is it just me?
This was an experiment. If you know the outcome, or you fiddle with the data to get a predicted result then it’s not an experiment. This doesn’t mean you can never tinker with an experiment in progress. When Kasparov was beaten by Deep Blue, the Deep Blue chess algorithm was being tweaked by lots of chess and computing experts, so you could argue that Kasparov was not wholly beaten by a computer, but that was still a computing achievement as well as an advert for IBM. The achievement today would be to build a chatbot with as little human interference was possible. Which is partly what IBM were trying to do, I guess. Maybe, out of this, they will come up with a chatbot which will distinguish between a genuine consensus and one person saying something a lot or lots of people (or lots of apparently separate characters) chanting the same thing in the same way.
I don’t believe the chatbot is ‘racist’. I don’t believe it had much access to the general internet, let alone directly experiencing the world. To a chatbot, ‘Hitler was right’ and ‘Bananas are yellow’ are both grammatically correct, and capable of being correct or incorrect. If one of these is re-enforced many times over a few hours by some trolls, it may well repeat it. When I was a student in the seventies, the God Squad used to feed the version of Eliza on the university IBM 360 with bible quotes, that it might preach to others. I don’t think they were genuinely trying to convert Eliza, or I hope not. But trolls have been force-feeding chatbots for the last 40 years to my personal knowledge.
The actual story is it is still hard to make a chatbot that learns without human support, and progress is happening in its usual zig-zag fashion. The public who don’t understand computing still rush out to buy tinned food and ammunition against the rise of the machines; because they hate what they fear, and they fear what they don’t understand.
