Turing Test #N-1: detect sarcasm:
“Sarcasm, also called verbal irony, is the name given to speech bearing a semantic interpretation exactly opposite to its literal meaning.” With that in mind, they then focussed on 131 occurrences of the phrase“yeah right” in the ‘Switchboard’ and ‘Fisher’ recorded telephone conversation databases. Human listeners who sifted the data found that roughly 23% of the “yeah right”s which occurred were used in a recognisably sarcastic way. The lab’s computer algorithms were then ‘trained’ with two five-state Hidden Markov Models (HMM) and set to analyse the data – and the programmes performed relatively well, successfully flagging some 80% of the sarky “yeah right”s.
That’s pretty good, but I’ll wait around for the computers to pass the Nth and ultimate Turing Test: compose a joke that is actually funny.
Honestly if we had to rank tests of similarity to human interaction, I believe that composing original humor is probably the very last one computers will solve. (Restricting attention to the usual thought experiment where the subject you are interacting with is in another room and you have to judge whether it is a human or a computer just on the basis of text-based interaction.)
Cheap Talk 
7 comments
Comments feed for this article
April 5, 2011 at 3:21 pm
hbi
not sure how I would perform on that test … at least judging from the blank stares I’m getting in class …
April 5, 2011 at 3:27 pm
jeff
right, an issue with a test like this is that by comparing with human subjects we are really testing whether the computer is just as bad as the average person (or average MBA in your case) at detecting irony. that might or might not be the goal.
by the way, those data are very interesting. real telephone conversations.
April 5, 2011 at 3:23 pm
Donald A. Coffin
Yeah, right. (Sorry; I couldn’t resist.)
The problem is that what a computer t hinks is funny (assuming a computer can think) and what an American thinks is funny (assuming an American can think) are likely to be as different as what a German thinks is funny and what an Ethopian thinks is funny…
April 5, 2011 at 6:07 pm
Yonatan
I don’t think it should be overly complex for a computer to construct a decent knock-knock joke. Similarly sounding words and phrases are easy to identify and while 95% of the jokes aren’t going to be funny, 5% probably will (inasmuch as one finds knock-knock jokes funny…).
April 5, 2011 at 8:13 pm
Scott
I think just the belief that the joke was constructed by a computer will somehow make it less funny, or not funny at all, for any intended audience.
And I think that it will be equally difficult to instruct a computer to laugh at the right moments.
April 5, 2011 at 11:40 pm
Ryan
The report is about how computers analyze voice (right?), but your recommendation to evaluate how computers perform with humor is restricting to text-based interaction, right?
Are you imagining something like coming up with a witty tweet or more like writing a script for Stephen Colbert?
I think Twitter would be a good arena for computers to get started in trying to compete, given their advantage in being able to access huge amounts of data might afford them a halfway decent chance at it. But then again it’s not a fair competition because humans already start with computers best conclusions (e.g. which tweets are trending). Yet I think this is a fundamental problem in trying to compare humans and computers… humans are already using the best computers can offer as a starting point. it only makes sense that the human can win.
April 8, 2011 at 9:48 pm
LRGII
You nailed it!!! Right? No, what you fail to realize that the quantum leap in computing (and our understanding of what/how we test) will come the moment a scientist gets brave enough to attach a babies brain to a chip (pluged in as it were) and allow it to “grow” as it were. It will then not only get humor, but it will be soooo funny and ironic, that YOU wont get it….Funny huh…