Can anti-transgender bias in media be measured?

Originally published at:

New data visualization project to reveal bias in media coverage on transgender topics could use your support.


Shared - and recommended. Cool project!


I don’t want to be that guy, but given a large population of which 1/160 people are X, a group of 300 people selected fairly and randomly has (1 - 1/160)^300 = 15% chance of having zero X. That’s not low enough to reject.


But is that a static number - or does it reflect say 20 years of of zero hires when the number of hires may have been say 3,000 different writers?


In this one example, yes. The problem is it keeps happening again and again. The real world chance of any group of 300 that is supposedly not selected for being cisgender having zero trans people seems to be far higher than 15%.


Up to that last part you are doing math, in the last part you are making a value judgement. You are deciding that pointing out possible bias in an organization is considerably worse than an organization actually being biased. I wouldn’t send someone to prison on an 85% chance, but if I’m not willing to be suspicious of racial, gender or trans bias on 85% then that’s going to mean just giving a pass to it almost every time. (ETA: For clarity, the math does not actually mean there is an 85% chance they are biased, calculating the odds of them being biased is not possible, I’m just pointing out that rather than asking ourselves if it’s really fair to accuse someone of bias if there is a 15% chance things happened at random, we could ask if it’s fair to trans people to say there is no bias if there’s only a 15% chance things happened at random)


What percentage of transfolk want to be journalists, and of that percentage who of them would want to work for an org that doesn’t really respect them?

Fair enough. That’s my own bias of using p = 0.05 as the reasonable standard.

I’m not claiming the Atlantic is unbiased, I’m not even claiming it is wrong to accuse them of it. But for a post advocating a data-driven approach, I find it severely off-putting to start with an argument based on flimsy math.


Most likely approximately the same percentage as “cis-folk” who want to be journalists, making the question moot.


Okay what about the second part?

Edit: The point I’m attempting to make is that trying to use stats to prove an unfriendly institution is unfriendly seems kind of insane to me. A place that publishes trans critical journalism is both less likely to hire trans people and is also less likely to be sought out as a place of employment by trans people and stats doesn’t seem to be the solution. But the whole point of this project is quantifying something I think is unjustifiable so whatever.

Not quite true. We usually use a p=0.05 as a cutoff for “statistical significance.” Any study with a p value over 0.05 is considered too likely to be a result of random chance to draw definitive conclusions. I would also defer to someone who knows stats far better than I to analyze in detail, since p value is often not an appropriately applicable measure of reliability and quite a few researchers have got an in trouble for using the wrong tools for the wrong studies. @Lexicat, you got a moment?


What about it?

Are you bringing it up because that might affect the math?
If so, i’d remind you that there are vanishingly few long standing and well known publications that have history of respecting transgender people, considering our society’s history. So options were not abundant. (I.e., while perhaps most transgender people likely didn’t want to write for The Atlantic, many would have taken a job anyway due to a paucity of options.)

Are you bringing it up as a tangent, merely stating that it would be …
And you’ve edited your comment so that my first point speaks to it.


And that’s where the value judgement is. It may be a (nearly) universally reasonable judgement, but it is worth acknowledging that it is a choice based on no more than ‘this seems right’.


We have absolutely no reason to guess that people who are transgender aren’t equally inclined to be journalists as people who are cisgender.

Probably lower than the number would want to work for an organization that does respect them. That’s one of the ways that a biased organization can keep people it is biased against out: by making the environment hostile to them. It doesn’t have to be overtly throwing out resumes.

It seemed like a sort of harmless sleight of hand directed at people who aren’t going to understand a decent account of the probability anyway. I mean, as the author says, in an unbiased industry you should expect one or two transgender employees at an organization with 300 employees. However, in a biased industry you should probably expect that many organizations will have 0. Assuming being transgender or cisgender has no impact on your qualifications to be a journalist, there will be a number of very qualified candidates who are unable to find work because of bias, and an organization that avoids this bias will hire up these great unemployed people, meaning they’ll have considerably more than 2 transgender employees. This dynamic plays out with all kinds of bias.

Imagine a simple model where five 300 person organizations hire from a pool of 1600 people, four of those organizations don’t hire transgender people, and organizations are pretty good at selecting the best candidate (this assumptions is shady but barely makes a difference anyway). Now four organizations have 0 transgender employees and the last one has all transgender people whose skills are in the top 15/16. The expected number of transgender employees is 9 or 10, not 1 or 2. The chance of them having 0 is is the chance of there randomly being no transgender people in 1500 people, which is around 0.008%.

We aren’t trying to figure out whether we live in a world with or without bias, we are trying to figure out whether a company is biased within a world where we know bias exists. The bias may not be as stark as my simple example, or maybe it is that bad. Either way having zero trans employees is a much bigger sign you are in the biased group than the unbiased group than the simple calculation you did suggests. That’s a lot of explain to someone who doesn’t have a great grasp of probability when the intuition that zero when it should be 1 or 2 is a suggestion of bias.

I think the calculation was making unfair assumptions. Like say you know that 10% of $20 loans aren’t paid back. You lend your friend $20 and they don’t pay you back. The next time they need to borrow $20 do you say it’s unreasonable to refuse because 10% is bigger than p=0.05? I think the “zero hires suggests bias” uses some intuition that is probably correct, since as I showed above, if we assume bias exists, that 15% can go down a lot.

(ETA: I realize, looking this over, that I’m basically making an entirely different point than the one I made above. They feel intimately connected to me, but I can’t explain how.)


Wow, what a great comment thread. Intelligent people respectfully and confidently discussing the fine points of a thoughtful, well written article. Compare this with any comment section on YouTube. I think I might be spending the rest of the morning at the Kahn Academy site trying to learn about stats.


Anyhoo, we’ve missed you @AndreaJames. It’s great to hear from you and good luck on your project!


The obverse is true - that not hiring any group of people predisposes a place to publish biased journalism about that group. This has been seen with pretty much every group.

It’s also illegal in your big media cities to have that discriminatory hiring practice.


What the hell is that, even?

It just sounds like a weaselly, sugar-coated euphemism for bigotry and prejudice to me; much like the term “gender critical.”


Well - you’re going to get some of that wherever you go. And given the unemployment rate for trans people - you’ll take many a problematic job in your lifetime.

Many Latinas work in jobs where they’re not particularly respected - most cisgender women have had that lovely experience - etc, etc, etc

A girl’s got to eat.


I have freelanced for The Atlantic online, which is why I use them as an example. They, like much of the legacy media, make a sharp distinction between print and online. I am fine working for a biased organization if it will make it easier for the next trans person. I have been doing that my whole career.