Weapons of Math Destruction: how Big Data threatens democracy

[Read the post]

Perhpas more at “How the mis-use of Big Data threatens democracy” ? [I’m complaining about the title of a book, not the title of the post that replicates the title of the book.]

Unless we accept that the phrase “Big Data” only applies to the poor application of algorithms to large datasets.

Because railing against the application of algorithms to datasets is like railing against the power-loom.


I heard a similar analysis on cbc radio last month:

I had always heard an algorithm was the spacing of a Tennessee Senator’s children.
(Thanks, I’m here all week, try the veal and be sure to tip your waitstaff.)

1 Like

Isn’t this what is happening in China with their “social credit” system? The nutty thing is that it seems like the citizens are into it. Gotta say, first dates will be super streamlined if all you need to do is tell each other your numbers… in bed.

1 Like

Thought I’d chime in with my first post because I feel the need to agree. Although I’m a little concerned with both of the titles (I know the post is emulating the same titular form).

I wonder if the author would consider re-naming the post “How Big Data can threaten democracy”. There are plenty of Big Data initiatives, some that are civic-minded, that do not meet Cathy O’Neil’s 5 categories. And not all Big Data meets those categories either.

I find this talk echoes the spirit of John Oliver’s “if you want to do something evil, put it inside something boring”.


Are you trying to battle a manufactred outrage with facts and common sense?


Manufactured, but not out of thin air. If a competently developed and implemented algorithm institutionalizes prejudice and safeguards the interests of the strong at the expense of the weak… more or less invisibly… does that sound like a good thing?


Does that sound competently developed and implemented?

Your definitions of competence and implement must differ from mine.

Unfortunately, yes. Algorithms that assess, say, your ‘creditworthiness’ aren’t designed to make your life easier or fairer. They’re designed to minimize risk for your lender(s). That is the only ‘competence’ required of these devices. An unintended consequence of routine use of such algorithms is a Kafkaesque society. A society where those who are born into less well-ordered, ‘polite’ families are systematically shut out of opportunity from day one.


Video of the talk:


During the past year, my UK bank has paid me over £ 150 in compensation, because their idiotic algorithm has at random declined my credit card at time critical moments, causing us endless work and frustration.

In one particular stupid instance, I had the audacity to try to make a payment of £ 749 to the UK Home Office. The payment was the fee for my daughter’s naturalisation as a British Citizen. The ingenious algorithm declined it, because I hadn’t made any similar previous payments to the Home Office.

To make this farce of a security theatre even more amusing: We bank with the very same bank as the Home Office thus they could have easily checked the legitimacy of the recipient as well as the sum. Also in the case of the Home Office / Nationality Service, common sense would suggest, that similar multiple payments rather than a single payment should raise the alarms as normal, law abiding citizens seldom need more than one identity / naturalisations papers.

A few months later I had the audacity to think that I am particularly bright, and tried to alert the bank in advance regarding a similar payment. Except that the system is not set up to process such information, so even if I call in advance and say I am intending to pay x amount to the UK Home Office, no one in that vast bank can tell the algorithm to adjust itself to my reality. It just runs its course and throws out the payment. Then I call up, complain, then some poor sod tries to sort it out, sends me £ 50 and we are non the wiser.

Idiotic algorithms are idiotic, and they are increasingly running our lives, creating kafkaesque feedback loops of institutionalised insanity.


I’m pretty sure that algorithms aren’t racist. The people who create and supply them with data on the other hand…

Yup. My bank nixed my debit card because of a ‘pattern of regular payments’. It was the phone bill.
(ETA: They DIDNT nix it when someone fraudulently debited my account from some shady business registered in the fucking Cayman Islands Oh, no. That charge was clearly legit. Totally)


This topic was automatically closed after 5 days. New replies are no longer allowed.