How to defeat the ignorance of the crowd

Steve McGrath, July 2015

The wisdom of the crowd turns out to be more of an oxymoron than we thought. Not only can vendors game online “star” ratings to deceptively promote their books, restaurants or hotels; positive ratings may be dramatically inflated even when consumers bestow them in good faith.

ignoranceofthecrowdSpecifically, positive ratings seem to trigger more positive ratings until the thing being rated is…well…wildly overrated.

So suggests new research by scholars from MIT, Hebrew University and New York University. They collaborated with an unidentified news website and focused on the reader comment sections associated with each article. Each reader comment receives a numerical rating based on up and down votes that other readers have given that comment. Over five months, the researchers arbitrarily gave newly posted comments initial up or down votes, or left the comment alone, then watched what happened:

The first [real] person reading the comment was 32 percent more likely to give it an up vote if it had been already given a fake positive score…. Over time, the comments with the artificial initial up vote ended with scores 25 percent higher than those in the control group. “That is a significant change,” [co-author] Dr. [Sinan K.] Aral said. “We saw how these very small signals of social influence snowballed into behaviors like herding.” (New York Times)

Remember this the next time you visit a restaurant that Yelp promises will be a five-star experience. And if you’re a vendor, don’t assume your online reputation is as pristine as your star ratings. You may be getting absolutely ravaged elsewhere on the Internet.

Look deeper

A far better way to understand your own brand and market is to look at what people are actually saying, and do it methodically. For example, we recently analyzed online conversation around the hotel industry and went far beyond the numbers.

With our partner MavenMagnet, we looked at more than 18,000 online hotel-related conversations between May 2012 and October 2012 across social networks, profiles, forums, news websites and blogs. We examined:

  • Buzz volume (how much conversation there was about each brand)
  • Positivity of that buzz (positive/negative ratio)
  • Impact of that buzz (e.g., the likes, links, mentions, retweets and conversation volume a comment attracted).

Then we dug deeper, separating conversations of leisure travelers from business travelers. Then we drilled even deeper into leisure travelers, separating the comments of those traveling with children from those without.

When we looked at all of these conversations, we analyzed not only practical considerations like cost and location but also guests’ comments around their senses, values and social needs, which are the other dimensions of Brodeur’s relevance model. (On the sensory side, for example, we discovered that that water pressure in the hotel shower actually eclipses even bed comfort in online attention.)

We ultimately discovered that Hilton, Marriott and Four Seasons rated the highest in what we call Conversational Relevance™.

You can’t game 18,000 online conversations. And when you work this hard and smart to understand where a brand stands, you don’t get fooled. Here’s our report.

Do you ever wonder what people really think of your business?