The real Turing Test crashes headlong into issues of gender identity


#1

[Permalink]


#2

Maggie, Why do all the links in your posts direct to the discussion board rather than the source material? I see the correct links when clicking "Show Full Post" from the forum, but not on the main page.


#3

I've come across this too, it's very annoying.


#4

There's a glitch with one of our posting categories, but it's a known problem and I'm not using that category anymore. It's not being used here. And, as far as I can tell, this post is not doing that. At least not from my Chrome browser on a Mac. Are you getting different results?


#5

It got fixed after I posted, I assumed in response. Firefox on Win7, for what it's worth.


#6

aren't there somewhat reliable algorithms for detecting the gender of authors?

Though in this case, the man is required to imagine what it would be like to be a woman, and construct an identity based on that cognitive model, while the computer can be preprogrammed to respond like a lady-- no cognitive model of "the other sex" is required.


#7

How is it a glitch? Isn't that where "discuss" (instead of "read the rest") implies it will go?


#8

It just occurred to me to wonder why Turing didn't generalize more. What if a human and a computer pretend to be computers? What if a woman tried to sound like a man?

I'm willing to bet the results of actual experiments would be very interesting.


#9

The way I hear it, Turing believed that a truly masculine man, such as himself, had no need for women, and so imitating women was akin to a parlor game, rather than a serious understanding of the human psyche,


#10

So that's what the internet in the early 00s was about. Just a bunch of guys practicing for the Turing Test.


#11

I think the claim that Turing's original proposal was that "A man and a computer compete to see who is better at pretending to be a woman" is a misreading. If you look at the original paper, Turing first presents a scenario where you have two people “a man (A), a woman (B)”, both trying to convince an interrogator (C) that they are woman. It seems to me that this example is just brought up to illustrate the idea of someone trying to convince an interrogator that they are something they are not, in a conventional scenario involving two humans (and gender is one of the more obvious ways in which humans can differ). Turing then says:

We now ask the question, “What will happen when a machine takes the part of A in this game?” Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman?

I feel fairly certain that Turing was merely making an analogy to the original imitation game, in effect saying “what if the same sort of game was played, except in this case instead of A being a man falsely pretending to be a woman, A is a machine falsely pretending to be a human?” (I think we're all familiar with the looseness of how analogical phrases like “now imagine the same thing, except…” are used in everyday speech.) I don’t think the intention was that it was meant to be exactly the same game, including the fact that the machine was trying to convince the interrogator it was a woman. Just look at the questions Turing imagines asking A, which have nothing to do with gender and instead have to do with tasks we conventionally imagine humans being better at than machines or vice versa, like writing sonnets and quickly figuring out winning chess moves. Likewise notice where Turing talks about covering the machine with realistic artificial skin, and says even supposing this invention available we should feel there was little point in trying to make a “thinking machine” more human by dressing it up in such artificial flesh–he says “more human” (indicating that the machine is trying to trick the interrogator into thinking it’s a human), not “more like a woman”.


#12

Not that link - the link in the body of the post that should (and now does) refer to the source article.


#13

I can imagine that pretending to be a woman might have felt emotionally significant to Turing, considering his struggles as a gay man living in a culture that doubted his masculinity, and eventually murdered him by forcing him to take female hormones. I don't suppose he thought about this consciously, but it seems meaningful that he chose that particular challenge for his test.


#14

Why would Turing pretend to be a woman?


#15

That depends on what you mean by “somewhat reliable”. Last I checked they existed but were pretty unimpressive. For classifiers like that there is a rather sunstantial range of performances where they work better than chance allows, but they are still useless for anything practical. Some systems are also awfully sensitive to any kind of imbalance in the source material (domain, text length, register…)


#16

I guess they're reliable in the sense that whenever I plug my own work into them, I'm invariably rated as male. I was often told that I "type like a man", and apparently that is true. Even a short story I wrote about a woman reminiscing about her childhood, with entirely female pronouns, came back as "strongly male".

What I'm saying is that I am a ladyperson and I don't think they work.


#17

There are two ways of solving the problem:
Speak like a human + speak like a woman

I think we're quite close to solving the second par of the problem. The problem is getting the program to converse as if it was human. Once you do that it should be a simple matter of biasing the subject matter and vocabulary enough to produce the desired effect.

Speak like a human + understand what makes women different then men in order to mimic a women

And that understanding might be closer to our conception of intelligence than the first strategy, or the conventional Turing Test,


#18

I get the same results; every email, blog post or short story I feed in there comes out as 'male' (or in a couple of cases 'Weak Male' whatever that's supposed to mean).

I think it's still largely based on stereotyping and cultural constructs of how women and men typically communicate, not yet another proof that males and females are oh so innately different. Then again, any tool, survey or study yielding the results that women/men responded stereotypically above chance elicit the belief that male/females are fundamentally different, even though a large number of people did not respond stereotypically (but are usually ignored or explained away). From the gender guessing program itself, I find several clarifications annoying:

The content, knowledge of the material, age of the author, nationality, experience, occupation, and education level can all impact writing styles.For example, a woman who has spent 20 years working in a male-dominated field may write like her co-workers. Similarly, professional female writers (and experienced hobbyists) frequently use male writing styles.Gender Guesser does not take any of these factors into account.

So a woman hanging out with and accepted by men ends up having writings that masquerade as a male's. It is not that females can innately write in any form, genre or style whatsoever based on their own interests and personality (or vice-versa for males). It's also not that it's silly to assign male/femaleness to genres and subjects in the first place beyond already-established stereotypes and tradition.

Also (emphasis added):

While Gender Guesser may be 60% - 70% accurate, it is not 100% accurate.This is better than random guessing (50%), but should not be interpreted as "fact". In particular, men should not be offended if it says you write like a girl.

In order to reassure the unfortunate men who may be compared to a 'girl', they basically admit that their doodad- and its results- is actually, probably flawed and will often guess wrong. Because who wants to turn out writing like a female, or (worse/better?) a Weak Male!

With this in mind, such efforts don't say anything about innate gender differences or writing but a lot about deeply ingrained, cultural and traditional stereotypes.


#19

With this in mind, such efforts don't say anything about innate gender differences or writing but a lot about deeply ingrained, cultural and traditional stereotypes.

And the imitation game doesn't?


#20

You said all of that WAY better than I did. I didn't spend 20 years hanging out in a male-dominated field, that's just how I write. How I've always written. I read everything and I'm often influenced by the style of whoever I was reading last, but I don't always even know the gender of the author (I didn't realize James Tiptree Jr. was a woman until about 5 years ago).

I was just talking about the same thing on another forum, and I plugged the post I made about the gender guesser into it. Yup, male again. This post gave me a formal writing score of male, and an informal of weakly female. When I added that sentence about my score, it changed entirely to male.