The predictable dystopian trajectory of China's Citizen Scores

boing
21

Funny enough, for them to really make it dystopian, they had to add in features that would almost certainly never be added in an IRL version - e.g. showing you who “rated” you, asking you to rate every single interaction, having cops/petty bureaucrats be able to “punish” your rating on a whim just 'cuz (that would probably be reserved for rather higher-ranking people, because they don’t want their toy taken away if too many low-ranking ones abuse it, which of course they would).

I still maintain that a true crowdsourced social rating tool would be a GREAT way to weed out narcissists and similar personality disorder types, because they rely on being able to drift from victim to victim.

Yes, you would need to build in guardrails and create all kinds of trust webs to make it very, very difficult to “buy” trust (or blackmail people to avoid being dinged by a malicious group), but those would have pretty obvious “fingerprints”, 99% of the time. You wouldn’t want to 100% rely on it, but it would be really good first-glance way to see if there’s a huge disconnect between a first impression and what people are saying about someone that have interacted with them over a long period. Kind of like a Rotten Tomatoes score, or, y’know, a credit score. If you pick the right metrics, you can get a pretty good idea of what something/someone is going to be like.

22

So having your life controlled by a score that’s generated with opaque, proprietary algorithms, with no clear rules or any appeals process, is less distopian in your opinion?

9 Likes
23

Or you could just regular social media search someone to get a feel for their already publicly available online presence

24

Yes, outlaw cascades are definitely a real risk here, but governments keep betting on having enough overwhelming force/control to prevent them, and are usually right. Until they aren’t.

3 Likes
25

Eh, I already know far too many people that maintain both “public” and “private” accounts. The public one is the one that you’ll find if you search them, and has only wholesome and un-objectionable material.

The private one has a similar name, but usually with an intentional misspelling or fake-but-similar last or first name, doesn’t share pictures with the public account (to avoid linking them), and is used for more casual / political discussion and/or being a dick online, if that’s their thing.

Basically, because there’s no formal 1:1 linking of person to account, there’s no way to verify that what you’re seeing is the real deal. But if you have an account made for you, about you, that you can’t control directly except via your behavior and interactions with others? That’s at least a bit more objective. So like the difference between a restaurant’s website and their Yelp page - one they can control, the other they can only influence. (Ignoring the frequent accusations of Yelp monetizing this, of course; this would have to be publicly funded and non-commercial to work decently).

1 Like
26

To be fair, there’s a lot to be said about somebody by what they consider “wholesome unobjectionable material”.
But the big problem with Yelp, beyond monetization, is the same as with any online ratings program- verifying that someone has ACTUALLY interacted/purchased/been a customer. How would a social program like the one you propose account for that?

27

…And this is different in what way than credit scores, that everyone accepts as annoying but generally fundamentally necessary and acceptable?

And I think that, again, if this were to be instituted, you would need something like an appeals process and some parts of it transparent. Like:

"Your score is linked to X, Y, and Z social media accounts. Click Here if you do not control these accounts to verify this.

Your score has been linked to these criminal convictions. Click Here if this is not you.

Your score has been linked to bad grades and juvenile criminal mischief at ABC High Scool, Click Here if this is not you."

Etc.

28

We do the same thing with credit scores. It’s the same system, privatized, giving the citizen less control over whatever agency distributes secret information about him or her.

3 Likes
29

You’re correct. Perhaps not to the same extent, I’m sure you could still purchase airline tickets if your credit score was poor, but who knows and for how long?

1 Like
30

Is this another episode of Black Mirror? I don’t get it.

31

Okay, a TV show, of our lives in social media displayed on a social media blog. My brain hurts. Thanks for setting me straight, not there’s anything wrong with being straight.

32

It’s all so clear now, my life goals are set. Thanks!

33

Sort of like SSNs (or equivalent depending on your country) work. Everyone would have a unique ID generated for them at birth, naturalization, citizenship, upon obtaining a work or travel visa (if for a longer stay than a couple weeks or so), and so on.

Upon hitting a certain age, you would be asked to register / verify your account, ideally linking it to a password and biometric data, also requiring a physical token. A picture would also be obtained at the same time, and updated whenever you update a government-issued ID like a driver’s license (this would also have a nice side effect of making these IDs more secure because of the multifactor authentication).

So now you have an account that’s pretty well-secured, which is linked to you (for others to rate you) and that requires biometric data (face, fingerprint, semen sample, whatever the current hotness is) to rate others.

Presumably, because there is provable / tortious harm in falsely rating people, it would be both criminally and civilly punishable for giving false ratings, or “renting” out your own ratings to ding or push up the ratings of others. Getting caught (or having a suspicious rating pattern) would either overtly or covertly lower the impact of your own rating power.

So, now you have a pretty well-linked and secure rating method. Of course, you want more than just a simple popularity rating, so you look deeper and create a sort of mathematical web where ML can determine various features - are the people who rate you high doing so your subordinates? Are they high-trust or low-trust themselves? How does this combine with other things like your industry, your school performance, your credit rating, and many other knowable things about you?

If you combine those, you can get insight into WHY people are rating you in certain ways, and that can further help categorize you into certain behavioral buckets, rather like FB does for ad targeting, and then this can further inform how ratings are used: e.g. mitigate the effects of ratings of “angry conservatives” on “moderate intellectual progressives”, because that’s almost certainly a “retaliation” rating. Someone cohabitating suddenly has a new address and down-rates the person at the old address, along with some of their close friend network? Down-rank the effects there too, because that’s probably a breakup retaliation. And so on.

34

I would say this is more of a Totalitarian thing than a Socialist one. Socialism is more about social ownership and democratic control of the means of production. China may claim to be a socialist country but thats just lip service to its past.

11 Likes
35

It is? Uh oh. I think maybe those weren’t tic-tacs I swallowed this morning.

5 Likes
36

Fixed! Thanks!

4 Likes
37

Speak for yourself. I don’t consider myself to be an off-the-grid libertarian loon, but I absolutely reject the idea that the credit score system in its current incarnation is remotely “acceptable,” especially when used by the government or really for any purpose other than determining whether a credit card company should offer you a card at a good interest rate.

Right now in the US you can be denied a job or a government-backed home loan based on a score that’s generated by private, for-profit companies using secret proprietary algorithms and no effective appeals process. And to even get a score at all, you need to borrow money, even if you’re in a position where there’s no other reason to. I know this firsthand:

  • A few years back I wasn’t able to qualify for a FHA-backed home loan (the only type many banks offer) even though I had a good job and large cash downpayment saved up, because my lack of credit card usage (I prefer to use debit) meant that there was not enough info to generate a credit score.

  • Some years before that, I had a job interview that I thought went pretty well, and they told me they’d bring me back for a 2nd interview after running a credit check on me. I never heard back from them, which may or may not have been related to my lack of a score, but the fact that it’s even plausible that I missed out on a cool job for that reason is maddening.

12 Likes
38

There was also a similar episode from The Orville (“Majority Rule”), that covered the same concept, specifically about how you can easily end up in a downward spiral of bad ratings once you start down that path. Of course in that episode “untrustworthy” people were mindwiped. Lets hope that doesn’t come to pass.

4 Likes
39

How would China (the ruling government) itself be rated if it was subjected to Citizen Scores?

(Perhaps this can be done informally by a local columnist, Dave Barry style.)

40

That is truly frightening. Is it a shoop or real?

1 Like