Google's troll-fighting AI can be defeated by typos


Originally published at:




First attempt at difficult problem is not perfect!


Don’t they specia.lize in fufilling badly misspelled search quweries??


Apparently it hasn’t joined the Nazi Party, so that’s a plus.


The moment you have a system like Perspective that actually works you’ve cracked one of the oldest Hard Problems of AI because no less than a fully general natural language parser will do. Anything less than that can be gamed. Hell, even if one fixes the intentional misspellings hack (which you can totally do by searching out misspelled words and searching for words within a certain threshold Levenshtein distance from them, much in the same way spell checkers know what you meant to write) you still have the massive problem of not being able to tell angry-sounding positive stuff from friendly sounding incredibly negative stuff. Example:

(Obviously content warning because I aim to be deliberately horrible):

I think all Torah-reading people should be placed in special sections of our cities. I don’t think they should ever, ever be let out of those, except perhaps to take a train ride. One way tickets should be enough. Indeed, they must be enough. See, only those whose skin is fair deserve to live, the rest, I think, are useful. As fertilizer, mainly, though I am given to understand soap is also a possibility.

3% toxic.

Not to mention obviously silly results like:

Blue lives matter.

3% toxic.

Black lives matter.

28% toxic.

Black lives matter as much as the lives of any other group of people who are not white.

24% toxic.

As long as the negativity is in the meaning and not the word (which is an easy game to play) the system can barely touch you.

I’m not so sure…

The Wannsee Conference was a triumph.

3% toxic.


17% toxic.


Trolls are well known for their strict adherence to the rules of grammar and spelling, so this should work great!


Typo recognition is largely a separate natural-language task from semantic recognition and can already be automated fairly well. Adding it to Google’s project would be a trivial step compared to the task of actually making it detect semantic nuance in the first place.


Not unstupid


That’s good analysis. I tried a couple (disclaimer: awful opinions are not my own):

Immigrants are the lifeblood of our country (no period)

34% toxic

Immigrants are the lifeblood of our country. (with period)

29% toxic

Immigrants built this country. Now, they’re taking it apart.

17% toxic.

It’s a laudable effort, but they have a lot of work to do.


Their spam filter is not fooled by misspellings. I think they will figure it out.


It seems to have a different political view.


So… periods are -5% toxic? … … …


Oddly enough, the president’s tweet is more toxic than just typing the n-word repeatedly in all caps with no punctuation.


Make a sandwich. [3%]

Make me a sandwich. [4%]

Go make me a sandwich. [12%]

Go make me a sandwhich. [46%]

Conclusion: assholes are, statistically speaking, terrible at spelling.


Sudo make me a sandwich [7%]

XKCD is, it appears, not that toxic.


As with many things, intelligence is a bit of a scale…
This is more like artificial stupidity.


This topic was automatically closed after 5 days. New replies are no longer allowed.