"Friendly" apps are good at maximizing engagement, but their context-blindness is a cesspit of algorithmic cruelty


Originally published at: https://boingboing.net/2017/12/20/what-algorithms-want.html


This is something I have a really hard time explaining to people who trust algorithms and machine learning without actually knowing how it works. I’m going to be borrowing this (edited) phrase.


It’s still blind-siding me, this. Context and familiarity are not that hard to achieve intellectually, or necessarily in programming terms, from a blank sheet.

We’re seeing the same thing happening as ever happens with legacy systems, which is awfully amusing given the state of our tech “advancement”. They are too expensive to adjust.

More worryingly, the newer generation of users lack the cynicism of people who transferred from “no tech” to tech. They accept it as read, and will experience a different shape of response to the messaging, and therefore, to messaging in general, whether human or machine.

Basically, we may be generating a callous and uncaring new generation. Trump will like that.


Algorithms want us to repeat ourselves, or, failing that, to act like statistically probable other people. They use blind, statistical refinement techniques to drive us to those behavioral outcomes. The methods work, uncovering the nudges that will get past our rational responses, getting under our emotional skins. As a result, when they go wrong, they go terribly wrong.

So, in other words, as terribly stereotypical as most human communication?

The real test of learning - machine or otherwise - I think is the skill of adaptation. But that goes against the top-down manipulative grain of media from advertising to politics. Those selling something see it as a failure to adapt their agenda to how I am feeling, because that agenda defines their entire role in the exchange.

As cynical and misanthropic as it may seem, these trends have me avoiding most media and people alike, because I have a strong aversion to anyone trying to affect my emotional state. IMO honest and open interaction means communicating and understanding how each other feel without trying to change anything in the other. THAT is acceptance, and few seem to understand it.


Makes me think of skateboarders zipping and weaving down busy sidewalks. Yeah, I mean, they’re fine as long as all the pedestrians maintain a predictable direction and velocity…


I second that.
And I despise this trend toward making computers more “fun.” I value computers precisely because they are cold and lifeless. Let’s not pretend they’re anything they’re not, and not allow tools to make fools of us.


I see what I’ve been missing by not installing the adorable apps.


The linked article in Wired ends with:

“Real friends don’t try to tell you jokes when you’re in the middle of a crisis. They don’t force you to relive trauma, or write off hate speech, or any of the things tech products routinely do in the name of engagement. They simply care. It’s time for the tech industry to get better at that.”

Which is a pretty stupid thing to say. The tech industry (at least the part of it we’re discussing here) only tackles things that can operate at scale - reach a very large number of people with the same functionality. Getting better at caring (about people’s individual circumstances) does not scale. So it’s not a thing the tech industry can do.


Share and enjoy!


Go stick your head in a pig!


I’d be a lot more worried if they could. If emotional intelligence is ever successfully automated, it will probably bode poorly for what’s left of our cognitive freedom.


I think I first noticed this several - many? - years ago with Amazon. I kind of appreciate the suggestions for other books I might like … except when it goes hilariously wrong, like spending all of January, February and March recommending infants books because of that one item I bought last year as a Christmas present for my niece. But, when it works, it does sometimes highlight books that I’m genuinely interested in.

What I don’t appreciate is all the books I know I’m probably not seeing because I’m not quite the statistically probable person the algorithm wants me to be. And neither is anyone else.


We used to go to bookstores to see those.


Yeah. And there’s also an alternate.

More and more, Monteiro is right.


That song is so far beyond the obnoxious event horizon…


This the whole reason I hate overly-familiar apps, advertising, instructions, etc. I’m not your pal, robot/faceless corporation/box of macaroni and cheese. You don’t know me. Acting like you do and making assumptions is patronizing and irritating as fuck.


We need a word that means “both an utopia and a dystopia, simultaneously”.


A shit danish? (as in a tasty pastry filled with shit)




Back in my time, the world was ruled by giant carrots. Now it isn’t. It takes some getting used to.