Algorithmic cruelty: when Gmail adds your harasser to your speed-dial


[Read the post]


This is what a white heterosexual male homogenous workplace does: it makes people blind to issues someone with a more diverse background would spot. When asked about it, they usually answer “this will never happen” - - because it has never happened to THEM.


Lighter example of algorithm cruelty: every social network that gathers information about people in common offers me my abuser ex-husband as a connection. We have a teenager son in common.

I don’t want to be friends with the guy that tortured me psychologically (I jumped out when it got physical) and I avoid contact like he was the plague. I shudder about him stalking me again. So every time I get to a new social network that can be popular I try to preemptively block him.


True but we all have trouble knowing about stuff that we don’t know. Hopefully G still believes in “Don’t be evil” and will fix this, now that is brought up.


Yes, but, see, when you design an app you have a process. A good design process takes into account the final user and tries to cover a wide range of don’ts. So “in which cases the speed dial would be problematic?” or “make sure our name brand doesn’t mean tiny dick in a foreign language”.

As far as don’t be evil this ship has sailed many years ago never to be seen again.


Lately google has been adding a ‘helpful’ menu of possible replies when I get a message. It feels creepy the same way as when a grocery store clerk calls me by name when I’ve never shopped there before.

Eventually, I suppose, all of these little streamlining measures will just decide to interact with each other, and I won’t even have to show up at my social life.


This issue helps show why for-profit corporations should not control all the productivity software.

When someone is profitably injured or subjected to increased risk of injury from code, the victim lacks adequate democratic controls over the remedy.

Thank you for this article @doctorow. If the opportunity should present itself, please write one about FOSSy Google Apps alternatives that are operated democratically, not-for-profit, locally and/or cooperatively.

Are there any such unicorns left in digital Middle Earth?

Thank you again for this eagle-eyed analysis of an issue about how victims of gender-based violence are subjected to foreseeable risk from nonconsensual corporate code.


And when you have a non-diverse workplace where everyone “knows” the same things…

Although this is a nice concrete example of why letting an algorithm determine something like this, and being unable to manually change it, is a dumb, dumb idea. We have enough other concrete examples that they shouldn’t have done this in the first place, even if they were unable to foreseen this consequence.


For me, Google’s sin is less about failing to specifically anticipate this happening, and more about incompetent design that forces changes on the user without any way to veto them. If you give the user control like you’re supposed to, they can fix these problems themselves and you never even have to hear about it.

But no, they’re convinced that they know better than us.


Depending on what you mean by ‘alternatives’, there may or may not be any.

If you want productivity software: word processing, spreadsheets, email clients, etc. some of the FOSS stuff is a bit rough; but its Office compatibility is typically within the same range as Google’s(it remains the case that if you really need Office, there isn’t anything non-Office that is a close enough approximation; but if you are using Google apps already, you apparently don’t fall into that category) and it’s all pretty usable.

The trouble starts if you want a google-like experience without the google-like orwellian data overlord. There are various options for running a webmail interface(squirrelmail, roundcube, Horde, etc.); but getting into the ‘running a mailserver’ business is a major pain in the ass(and good luck getting major services to take your traffic seriously). There are orwellian data overlords that aren’t Google; but they all tend to be subject to similar incentives and blind spots and to make similar mistakes(facebook is arguably more aggressive that Google about screwing up, Apple less so; but at least as hardline about the ‘this is the UI and you’ll take it and like it becuase it’s genius’ position, probably more so).

Going all or largely FOSS(or proprietary-but-treats-your-purchase-as-the-transaction-and-gets-out-of-your-life-thereafter if you prefer) isn’t difficult if you are willing to operate according to tradition; but if you want a fancy all-web-based experience and cross platform integration with a smartphone ecosystem and such, you have much, much, narrower options.


Thank you for responding and helping.

I’ve used Libreoffice and Ubuntu for professional work for years. I only boot Windows or use IE if I encounter a proprietary vendor app I can’t avoid.

With colleagues, I use Google Docs, gmail and Google Calendar. The data are trapped there and not as confidential as I’d like.

I endured a lot to migrate over a period of years from Windows and MS Word to linux and openoffice and then libreoffice. I’m comfortable learning new things.

It may take me longer, partly because my professional background isn’t coding.

What are the email alternatives that avoid the major privacy issues?


i’m glad i don’t use gmail (major privacy issues), so much so that i’m really surprised that cory uses it.

i think any speed dial should be opt in, the user should be able to control and edit entries, block entries, add entries, etc.

maybe there needs to be a universal do not connect list? where anyone can add people they don’t want to connect with or block if possible, then all social media networks and apps can query this list and filter using it. seems like it would be a good idea.


A few weeks ago I read this great quote, that I can’t find right now, that basically said “If you haven’t thought about how some asshole can use your product to make another person miserable, then you haven’t really finished designing your product.”


Gmail says in its user agreement that they parse all your private email, and not just the messages you send, but also all the email you receive from anyone, so not only does it screw you, it screws anyone who emails you. Google has long abandoned the “no evil” creed.

Your best bet is probably:

Some lists of alternatives:

(Tracking what people search for is one thing, but reading their private email crosses a lot of lines…grrrrr…with email and instant messages/chats there is an expectation of privacy, imho.)


You are awesome! :smiley_cat: Thank you. Bookmarking.


While it might prevent some unpleasant auto-suggestions when the social-network-of-the-week starts spamming you; wouldn’t such a list, and its contents, be a goldmine of delicious user data? Even if you could prevent malicious users from tampering with it and adding or deleting items to the filter to silently black-hole people or arrange surprise re-appearances once somebody had let their guard down on a given service, having a “person X does not want to talk to the following people” list is really just trying to fix the epidemic of data mining and oversharing by sharing even more data(and, if the intention is for it to be broadly used, that effectively means that it’s public, and at least one ‘social’ service is going to be querying it to show their users “all the people who have requested to not be connected to you, for your harassing convenience”, since that’s what their users will want.


that is an excellent point. hmmmm…

it could be private and encrypted and require you to enter a key only you have in order to allow access from an app, but once you open the door to that app, then it is potentially out there. maybe it isn’t such a great idea. i wonder if anyone has already mined such data from the facebook block feature?


As demonstrated by the fact that stories like this are being written about major internet companies doing painfully idiotic things to the very users they are trying to algorithmically cozy-up to; it’s clear that sometimes the adversary is way, way, dumber than we give them credit for; but as a general rule I am always inclined to assume that anything that might provide information about you is a probable target for mining; and who you block is probably treated as a markedly stronger indicator than who you ‘like’ or ‘friend’ or whatever, since the norm appears to be to allow a lot of vaguely-amicable-strangers to be ‘friends’ rather than do them the discourtesy of turning down their requests.

Given that javascript makes grabbing substantially more granular data(observing how long you hover over a ‘like’, ‘block’, ‘friend’ or whatever other element, say) pretty easy; and requesting assorted page elements to be displayed obviously involves talking to the operator’s server, I’d be strongly inclined to suspect that not only is this being mined; but it is being mined with considerably higher resolution. How many times, and how long, did you see that friend request before taking one action or the other? Did you hover over both options? Go right for one? Does someone with the same browser fingerprint as you(but ‘logged out’, hahaha) still examine the page of that user you blocked from time to time?

I suspect that this is occasionally paranoid; but it’s hard to tell which occasions, so that suspicion doesn’t help me much; and the cost of gathering the data are so low that you just can’t safely assume that the adversary isn’t(unless they’ve tried it for a while and their data miners determined that it didn’t seem to correlate with anything useful).


Are there not-for-profit options for non-FB users?

I don’t understand why more coders don’t band together with more creatives and lawyers to make non-commodifiable alternatives?

If I were a coder, I’d want as much FOSS as possible for job security. All the licensing, copyright and patent overreach is for the birds.


There is Diaspora, along wtih a few even more obscure ones mentioned in the ‘see also’ section on that page; but seeing someone actually use one of those in the wild is slightly less common than seeing a PGP-encrypted exchange of email, which is really saying something.

I’m not sure how much of this is just a network effect problem, with even people who think facebook is a miserable cesspool mostly stuck with it because that is where everyone else is, or whether the decentralized alternatives are still really rough around the edges, or some of both.

The only thing I can say is that email/SMTP appears to have been the last open, multiparty, messaging mechanism to have achieved wide adoption, with virtually everything following it being either heavily tilted in favor of the operator’s account system(eg. blogging sites that offer RSS/Atom but would very much prefer that you create an account and ‘follow’ that way) or just plain closed(eg. Skype). Sometimes the protocols used are open standards, or derivatives of them(XMPP and SIP show up a lot; more and less mangled); but interconnection between services tends to be a no-go; and even 3rd party clients usually get a beatdown when the service thinks that they can get away with it(looking at you, twitter…)

I may just be getting old and bitter; but this ‘social’ nonsense tends to make me either angry or sad depending on the day. Time was when silicon valley actually invented and built stuff; and now it’s all a bunch of VC circlejerk and billion-dollar valuations for shoddy mobile phone software that pretends to be your friend in order to show you ads more efficiently.