Can a machine detect sarcasm? Yeah, right

Applying analytics to social media? Good luck -- not all words can be taken at face value. Natural language processing helps, but it's no panacea

Natural language is a wonder. Like the best poetry, everyday speech often expresses overlapping meanings in tightly wrapped verbal packages.

At the most fundamental level, the linguistic mechanisms for coding, decoding, and distinguishing these packed meanings are built into our brains. But they're also calibrated on the fly by the cultures in which we live. This mental processing spans the range from conscious to unconscious, with most of us barely aware of all the verbal vibrations we're putting out and absorbing as we move through the day.

[ Learn how to use new analysis tools and abundant processing power to gain insights from the massive volumes of corporate and external data in InfoWorld's Big Data Analytics Deep Dive. | Discover what's new in business applications with InfoWorld's Technology: Applications newsletter. ]

Sarcasm is a multilayered, semi-conscious linguistic phenomenon par excellence. Essentially, it's any double-edged utterance -- in other words, the speaker's deep intention is diametrically opposed to the surface meaning of his or her words. For example, anytime someone says, "I wonder what genius thought of that?" this is not a suggestion to nominate that person for a Nobel Prize.

When a customer (actual or potential) directs sarcasm at your company, that's almost never a good thing. What that means is they don't respect you enough to state exactly what's on their mind, whether it be good or bad. Not only that, they're probably saying nasty things about you behind your back. In the process, they may be signaling that they're about to churn away from you or, if they've never been your customer or have recently churned, they may be urging others to stop doing business with you. That's definitely an issue you need to address proactively if you wish to remain in business.

Distinguishing sarcasm from sincerity can be quite tricky if you're not in the same demographic segment as the speaker or, even if you are, if you're entirely unaware of the situational context in which something was said. Even if you try to remain current on the latest lingo of the group you're monitoring, the culture moves so fast and in so many mysterious ways that you may be caught off guard when a long-familiar phrase suddenly acquires air quotes among the terminally hip. It's not always easy to know how many grains of salt to sprinkle on an otherwise innocuous figure of speech.

For those reasons and more, sarcasm analytics most certainly can not be automated 100 percent. When focused on the statements of specific individuals in particular circumstances, it will always demand some degree of human judgment and a fair amount of uncertainty in your assessments. The best way to ascertain any individual's true intentions is to probe them through conversation. Of course, there's nothing to stop the person in question from lying, bluffing, or hedging their statements to keep their true intentions a closely held secret. But you can at least try to drill through with all the sarcasm-whacking tools at your disposal.

If you're trying to gauge broad shifts in customer sentiment, as evidenced by increasingly barbed statements, sarcasm analytics technologies might be exactly what you need. A recent article discusses how this application of NLP (natural language processing) and machine learning can support brand management initiatives. According to author Erin Carson, sarcasm analytics "can provide a chance to learn about things like product issues, service issues, unexpected uses of a product, areas where the brand is spending too much time -- or not enough -- or general feelings from customers and potential customers."

Perfection is too much to ask from automated sarcasm analytics. The article cites an industry analyst who says keyword analysis is typically 60 to 65 percent accurate, while NLP raises that to 80 to 85 percent. But the toughest 15 percent of cases require human judgment to distinguish sincerity from snark. Plus, some humans' judgment can't be trusted; they may be utterly clueless in reading the intentions of others.

If you ask me, perfection is irrelevant when you're trying to gauge somebody's deeper intentions from their overt verbalizations. If 85 percent of the time you've determined that what they're putting forth is sincere, you can safely assume the vaguer, grayer areas in words conceal no daggers.

After all, most people are not clever (or diabolical) enough to double-edge their words 15 percent of the time, while otherwise remaining totally straightforward and transparent.

Sarcasm is not the same as treachery. More often than not, what it expresses is a mere blip of momentary irritation in an otherwise satisfying relationship.

At least that's the way it works in my marriage.

This story, "Can a machine detect sarcasm? Yeah, right," was originally published at InfoWorld.com. Read more of Extreme Analytics and follow the latest developments in big data at InfoWorld.com. For the latest developments in business technology news, follow InfoWorld.com on Twitter.

Copyright © 2014 IDG Communications, Inc.