A catalog of ingenious cheats developed by machine-learning systems

Originally published at: https://boingboing.net/2018/11/12/local-optima-r-us.html


You want grey goo as a response to building a better mousetrap? Because that’s how you get grey goo.


AI trained to classify skin lesions as potentially cancerous learns that lesions photographed next to a ruler are more likely to be malignant.

Yeah, it learned to identify rulers in pictures, not skin cancer. I think about that one a lot, because how do you not see that coming?

I recollect an old example of machine learning where the system had evolved to use the analog properties of the chips it was running on. It worked, but it not if you tried to run it on any other hardware.


Not perfectly topical to AI, but still possibly relevant:

When the training for a police Drug Recognition Expert never ever includes any subjects who are stone cold sober, the so-called “drug whisperers” tend to see impairment in literally every driver they examine.


A.I. suggests modest proposal.


Agent kills itself at the end of level 1 to avoid losing in level 2

We’re now pushing AI as hard as students? (As a depressed ex-student, ha-ha-only-serious)


This sounds a lot like teaching lazy yet intelligent children. They’ll (not so) cleverly find a way to thwart your acceptance criteria:



And to be fair, if I was shown a series of photos and asked to guess if they were cancer, I’d assume anything important enough to measure must be serious.

1 Like

If you’re curious about the video:

1 Like

Shall we play a game?

A strange game. The only winning move is not to play. How about a nice game of chess?


In first year computer science I encountered a quiz that required writing a program to calculate a table of values for the hyperbolic sine function. Never having seen sinh x before, I interpreted it as sin(hx). My program thus started with a request for the user to “enter a value for the parameter h”. I’m sure the marker responded much the same way as the teacher in your “find x” example above.

I also famously followed this up by raising my hand in a physics quiz where we had to find the period of a simple pendulum constructed of a weight attached to the end of a meter stick and asking, “Exactly how long is this meter stick?” To which the teaching assistant calmly answered “1 meter”.

11 years later I was granted a PhD :wink:

  • A robotic arm trained to slide a block to a target position on a table achieves the goal by moving the table itself.

That’s intelligence? That’s a “how many engineers” joke!


In an artificial life simulation where survival required energy but giving birth had no energy cost, one species evolved a sedentary lifestyle that consisted mostly of mating in order to produce new children which could be eaten (or used as mates to produce more edible children).

That was a great Rick & Morty epsiode!

Genetic algorithm is supposed to configure a circuit into an oscillator, but instead makes a radio to pick up signals from neighboring computers

Great. Now I have another project idea for my eurorack modules.


For work (and fun, cuz I like what I do) I’ve been trying to learn a bit more about ML, because it’s of utility in my industry (media technology). So I got into some of those basic intro to ML books you can find in two seconds on Amazon.

I found it interesting that it seems to be pretty common to use the design of a basic stock-trading app using a certain algorithmic approach, as an example problem in these types of resources. There is always a disclaimer that this is for example purposes only! Obviously you need pretty damn sophisticated algorithms to predict market activity, not the kind of stuff you find as “my first ML app” in a starter book on Amazon. And even those sophisticated approaches can fail to predict Black Swan events, and in fact may even exacerbate them!

But isn’t it interesting that the market has gone into looney-ville, roughly over the same period of time such Intro to ML books have been on the market, with such example exercises. One wonders how many people actually put these things to use. No rational activity describes our current market value, with about 100 potential Black Swans in our immediate periphery. Sure seems like a lot of “my first ML app” BS to me…


This is a slightly edited version, missing a few rows from the original (reproduced here)


A Dutch roboticist involved with our world champion robosoccer team told me they once tried to teach the robots to avoid collsions with opponents, as the rules forbid collisions. One robot decided it could best avoid collisions by moving itself out of the field entirely.


Go on and win a Noble prize and suddenly that anecdote becomes early evidence of your genius…


So kind of a digital “Clever Hans,” effect.


My college physics teacher told the story of seeing somebody taking a test looking at their left hand while writing with their right…“Your OTHER right hand.” (for the right hand rule) he told his student.

1 Like

For a second I thought this was an older post from a little while ago: