For many online shoppers, Amazon (NSDQ: AMZN)
is their starting point for researching products they want to buy—and
Amazon’s customer reviews, in particular, play a key role in those
purchasing decisions.
The author of the study, Cornell professor Trevor Pinch,
says the fundamental problem is that people reading the reviews probably
naturally assume that the Amazon reviewers are regular shoppers just
like them—when, in fact, their relationship to the products they review
can be a little more complicated. “The issue of the ‘customers’ not
really being customers needs to be addressed,” says Pinch, who surveyed
166 of Amazon’s top 1,000 reviewers for his study.
SEE ALSO: Who Are Amazon's Citizen Reviewers?
But there is some new evidence suggesting that Amazon’s customer
reviewers—particularly the top 1,000 reviewers—do not always make
independent decisions about which books and other products they write
about. According to a new Cornell study that we previewed
last week, the reviewers in many cases acknowledge that in order to
maintain their high rankings and continue to receive free products (one
of the perks of being a top reviewer), they have to make surprisingly
calculated decisions about what to review and what to say about those
products. We reached out to Amazon to talk about its product-review system, and to get the company’s response to some of Pinch’s claims. But the company didn’t respond.
History
In Amazon’s early days, in the late 1990s, the “Editorial Reviews” that appeared on book pages were written by Amazon employees—especially editors, but “anyone who worked for the company, including warehouse staff, were asked to write as many as 10 reviews a week.” Amazon later made deals with book review publications like Booklist, Publishers Weekly, Kirkus, The Library Journal and the New York Times Book Review, to copy their reviews of newly published books. Over time, the literary editors hired to write reviews in those early years have either left or moved to other positions in the company, and customers themselves have become the main source of reviews on the site—though the Editorial Reviews remain.
For the most part, though, Amazon has outsourced the job of writing reviews to thousands of unpaid citizen reviewers. Seventy percent of the top reviewers are male, their median age is 51-60, and more than half hold a graduate degree. About 14 percent of those reviewers are professional writers. Why do they write the reviews for free? Respondents to Pinch’s survey overwhelmingly mentioned “self-expression” and “enjoyment” as their motivations. Many respondents also cited altruistic reasons for reviewing—“hope to help others decide whether to buy,” “wanting to share what I have liked with others,” etc. And some said they write reviews to help them keep track of the books they’ve read and the movies they’ve watched.
But in the interviews with Pinch, respondents talk about some other motivations that might interest readers of these reviews. The study, for example, found that 85 percent of respondents had received free products from publishers, agents, authors, and others. Why is that an issue? Professional critics—at a publication like the New York Times—also receive free books to review, of course. But those critics are paid by the publications they write for, and their job is to review these books objectively. For Amazon’s unpaid customer reviewers, the only tangible benefit of their “job”—and the study indeed found that for top reviewers reviewing is akin to a second “career,” a “crossover occupation”—is any free books and products they receive. The way to keep those freebies flowing is to pump out glowing book reviews. (Amazon explicitly tells reviewers to “please clearly and conspicuously disclose that that you received the product free of charge,” but Pinch says reviewers don’t always adhere to that directive.)
Some 88 percent of respondents reported that most or all of the reviews they wrote were positive. “I don’t want to make waves, and I don’t want to offend the author,” one said. “I’m in the midst of writing a book myself, and I’m thinking it might be prudent not to be TOO overly critical of books that go through the traditional publishing process.”
The Ranking System
One reason reviewers care about being “prudent” is the way that Amazon’s reviewer ranking system works. In 2008, Amazon made broad changes to the system, causing many longtime reviewers to lose their top rankings. These changes have been a subject of great debate on Amazon forums. While Amazon has been secretive about the algorithms for its reviewer rankings, it says that under the new ranking system rank is determined by the “overall helpfulness” of all of reviewer’s reviews (as rated by customer votes), the number of reviews the person has written, and the recentness of the review. Recent reviews get more weight.
Sixty-seven percent of the respondents to Pinch’s survey disliked the new ranking system: “It rewards the newbies at the expense of long-term reviewers who have worked for years on the site,” one top reviewer wrote. Confusingly, Amazon still includes both reviewer ranking systems on its website, listing reviewers by both New Reviewer Rank and Classic Reviewer Rank. The top-10 lists are different, and nobody who is a top-10 reviewer also holds a top-10 spot in the classic rankings.
“Not Helpful”
Some reviewers told Pinch that they steer clear of books on controversial topics like politics and religion—because reviewing those books can increase their number of “not helpful” ratings. One said: “A positive review of a conservative politics title is sure to attract a great number of ‘not helpful’ votes by those who don’t like the author’s politics.” Wrote another: “Since some people mark reviews as ‘unhelpful’ simply because they disagree with them, this means a top reviewer is most likely to be someone who only gives the ‘correct’ review of a book, rather than a more nuanced and balanced review, or critical one. The new system discriminates against minority opinions and seeks homogeneous reviews and fans of those reviews.” And a third respondent said, “A reviewer can either be willing to address a controversy OR simply go for a higher Amazon ranking. He cannot do both as Amazon has made them mutually exclusive!”
“Helpful” votes lead not only to higher rankings, but also to more free books. Pinch says it appears that publishing companies and agents start to offer free review copies to Amazon reviewers when they hit the top 1,000. Once they made it into the top 100 or top 50 reviewers, they got many more offers. Some respondents mentioned that if they didn’t like a book they received, they would give its sender the choice of whether or not they should post the review. Not surprisingly, the answer was “invariably” no.
To be sure, Amazon isn’t the only site that has critics who question the soundness of its reviews. Yelp, one of the top review sites by traffic, has its share of detractors. In May, a group of small-business owners filed a lawsuit against the site, accusing it of offering to bury bad reviews if the business bought ads. Yelp has rejected the claims, and that case is currently in court.
Who’s to blame for apparent flaws with Amazon’s reviews? There don’t appear to be any obvious villains here. There’s no evidence that Amazon is secretly pulling the strings behind the scenes to keep all the reviews upbeat. And it certainly doesn’t seem as if the citizen reviewers have some innate desire to avoid important but politically charged topics.
That said, reading Pinch’s interviews with reviewers, you get a sense of how hard it is maintain the integrity of a process that is dependent on a virtual army of unpaid but still presumably capitalist-minded laborers. If they’re not paid, they are going to find other incentives and motivations—which may in some cases work at cross purposes with their primary mandate, to produce honest and independent-minded reviews.
No comments:
Post a Comment