Secret service developing a sarcasm detector. Oh great


#1

[Permalink]


#2

I'd like to note the full bullet is "Ability to detect sarcasm and false positives."

If someone knows what a false positive is, they ought to know that designing a system to detect them is exactly the same thing as simple always getting everything right. It's also literally impossible. Like, literally, literally. Obviously and provably outside the bounds of reality.

A sufficiently advanced computer may be able to detect sarcasm on the internet, but this specification is actually asking for fairy dust.


#3

Sure, but how will it do with irony?


#4

I read the 'detect ... false positives' as just a bit of sarcasm. Maybe I missed a false positive.


#5

It's just an RFP. They're not "developing" anything, just asking for it. I don't see the harm in that. They want something that matches that spec (wouldn't you?) and they're telling the world they want it.

They won't get it of course, because it doesn't exist. But maybe they'll get parts of it, or maybe it will spur research. Worst case, they get bogus proposals and don't have anyone realize they're bogus.


#6

Ohhh, a SARCASM detector, well that's a REAL useful invention!


#7

Thing is, it is possible. Disclaimer is that I have studied speech and language recognition but I'm nowhere near an expert. I've worked on some NLP projects. With sarcasm, irony, and all the rest, you could n-gram typical patterns of words, and with a fair cross-section of human testers, you could verify a set of sarcastic/ironic patterns, and record the differences in inflection in the spoken words. There must be differences in the physical sounds, as well as contextual clues and outright word choice. If there were no distinguishing differences between sarcasm, irony and straight-up regular speech, then we couldn't detect them as people. Now I'll freely admit that messages are sometimes hard to receive. So there of course would be error. Maybe they just want to be able to skim off the top layer to reduce their level of hassle - they just want to be able to quickly put into bins all the tweets of someone saying they wanna do such-and-such to you-know-who and take an educated guess if the person could be serious about it. That IS firmly in the realm of possibility.


#8

Yeah, like I said, I figure a sufficiently advanced system could presumably detect sarcasm. Detecting "false positives" is nonsense from someone who thinks "false positive" is a buzz word and not a meaningful term.

Although I notice that on the internet human beings aren't very good at detecting sarcasm either. I think sarcasm in print would be very tough.

An RFP means they are developing it. When they get the proposals back they are pretty much obligated to choose the one that best matches your criteria unless none of them can meet criteria they deemed essential. Companies will put in bids, one of them will win, and whoever wins will be at the end of the money funnel while they produce something that doesn't really work.


#9

I would neeeeeeeeeeeeeever dream of counterfeiting money


#10

This is exactly right. They may or may not have money sitting in a fund. Doesn't matter. Typically they will grab from another budget item somewhere and throw money at a good proposal. Good proposal being defined as my cronies put this thing together at Raytheon and need 250k for the first year and then 600k per year thereafter for it. That kind of "good proposal." I've submitted dozens of these things with a colleague, and occasionally they do fund a little guy... Not like they fund their friends, though.


#11

Thing is, you'd need to get sarcastic people to admit their covert aggression, which is difficult even when they're aware of it.


#12

That certainly is a nice shirt you're wearing. I'd hate to see it get messed up.


#13

You want to come to my dogs birthday party? wink

We don't really need a sarcasm detector. But a bot that closed our sarcasm tags for us would probably be a huge leap forward.


#14

Sure, I'll bring my five little friends. A bot that prevented anyone from posting anything would be fantastic as well.


#15

Hopefully better than a certain Canadian songstress.


#16

I hope it also detects the incorrect use of "literal" and "ironic".


#17

I'm plenty jaded about RFPs generally and gubmint RFPs specifically, but it is very normal to set out an RFP asking for an unreachable platonic ideal in order to spur competition in reaching partway to that ideal.

If it's the Secret Service's job to watch for, e.g., political assassination threats on Twitter, and if they already have some NLP system scanning the stream of all public tweets looking for those threats, then it makes good sense for them to cull the list of reported threats based on an NLP-backed diagnosis of "sarcasm" or "unseriousness" or whatever.

I'm actually fine with this RFP, at least in concept. I would be pissed if it looked like they were trying to do a better job of surveilling my private communications; it looks like they are actually trying to do a better job of NOT dragnetting people who make a threatening-looking statement in jest, on public fora.

I could be wrong, of course.


#18

I think we must be misinterpreting this -- I'm assuming that "detects false positives" is domain-specific jargon in this case.

I don't think they are asking for a system that both detects sarcasm in people's IM messages, and detects false-positives in, say, medical experiments. Such a spec would be akin to asking for a program that detects spam-messages and guides ICBMs at the same time.

Note: I'm reading the spec literally as Ability to detect sarcasm ∗and∗ false positives. If it were Ability to detect sarcasm ∗without∗ false positives that would be an entirely different thing -- and one solvable by simple outputting 'not sarcasm!' each time...

Note #2: you may already know what they mean by "false positives" in this case and maybe it's only me who doesn't.


#19

I guess the issue is I can't imagine anything they could mean by "false positives" that would make asking for that make sense. System A can possibly detect false positives in System B but A will have it's own false positives, so there isn't much point.

I realize that what they want is something that will detect people's attitudes so they can find people who are violent or hostile or whatever and for the system to minimize the number of false positives it gives. But we all want every test we conduct to have an absolute minimum of incorrect results. It just really reads to me like it was written by someone who doesn't know what they are asking for, and that's both troublesome and half-expected.


#20

HLDT: Hodgman Literary Tone Detector

[http://youtu.be/2wdV7Wmv6wk][1]

If you require a device the inserts electricity into your brain via a thin wire.