High bias found in Amazon reviews of low-cost or free samples


Originally published at: http://boingboing.net/2016/09/20/high-bias-found-in-amazon-revi.html


I ran into this brick wall when I went shopping for an LED light that I could stick on the interior of my mini-fridge. Every search result seemed to be a shilled product. After half an hour, I just gave up.


More and more, products I look for on Amazon seem to have two sets of reviews: five-star reviews from people who were provided the product in exchange for a review, and low ratings from actual customers saying “I’ve been using this for years, but the one I got from Amazon was in a fake looking box with the label coming off, and the product was all messed up, pretty sure it’s a knockoff.”


Wait… that’s the wrong Fry “shocked” GIF…


File this under “No fucking shit?”.


Everyone knows that the one and two star reviews are the real place to go for good info; if the only negative things people can say about a product are related to shipping and/or dumb user-error (or misconceptions about what a product actually does), then it is probably okay.


This is why Steam just changed their review system.


I admit, I did these for books for a while. The e-mails always said an honest review was what mattered, and I always tried to review the book for the intended audience, as opposed to just me (a lot of the books I was sent were way, way outside the genre list I provided by the review service, even though there was superficial overlap if you just read the back-of-book blurb).

Besides the expected issues with thin-skinned authors, there was also “star inflation” to deal with. My standard “if it’s your kind of thing it’s worth reading” wound up being four stars, with five reserved for work I was actually impressed by and three (I didn’t dare go lower) for books that didn’t achieve what I thought they set out to do. All of those three star reviews got a terse “we appreciate your honesty” message from the review service… and then no further books for a long time.

I stopped reviewing because a) I kept getting pushed to read books I wouldn’t enjoy and b) even though the reviews were supposed to be “honest”, there was too much pressure to assign those four or five stars. It does make me a bit sad, because those few five-star reviews were for books that were real discoveries for me, books I thought were amazing but would otherwise not have read.

Having said all that, I’ve also noticed a flipside: books which are indie often get slammed with “needs editing” even when the form is obviously experimental – or even quite professional, thank-you-very-much.

As usual, a full disclosure of bias in the interests of providing context goes a lot further than a sliding scale of stars.

In the meantime, for Amazon, I recommend reading the three-star reviews. That’s where you’ll find the reviews that weighed the pros and cons.


Nailed it. Three-star reviews are where it’s at.


I use this system for my own reviews:

  • Five stars: Perfect, would definitely recommend
  • Four stars: Good, but with some minor drawbacks.
  • Three stars: Not as advertised. Might fit your needs, might not.
  • Two stars: Bad.
  • One star: Unredeemably bad, in ripoff and scam territory.

In practice, I see this:

  • Five stars: Vacuous gushing praise
  • 3 +/- 1 stars: Some honest thought went into the review
  • One star: Usually unwarranted hate

I like the three star reviews for this reason. They typically go into more depth describing the product and its differences between what’s advertised or what’s expected. It’s far more informative to read reviews from someone who knows what they want and can articulate that.


I will admit that I’ve posted a review for a free sample. Completely out of nowhere, I was contacted through Amazon by a company offering me a free bidet seat for my toilet. I suspected a scam but said sure, and a few days later got a bidet in the mail. They then emailed to ask if anyone else in my house wanted one, as well, so a second one arrived. I liked it a lot, and posted a four-star review, explaining what I did and didn’t like. The company emailed to thank me for my review, and as a reward, sent me yet another bidet.


Man, you got scammed hard.

Do you need that third bidet, by the way? :wink:


My house now has three toilets with three bidets, which to me is wonderful, but to houseguests is surprisingly bizarre. Americans are not into the idea of washing their butts.


“Why did you put these leprechaun bath tubs in every bathroom?”



bidets are a miracle of modern sanitation science


Hey, they wash their butts just not with a special butt water fountain.

If I had a bidet I’d put a “For Colaculas Only” sign to confuse people.

ETA - more people would probably use them, but they do take up a lot of space and are an extra expense.


The shocking thing about this is how small the bias is.


The ones I have just attach to your existing toilet seat and add a little spray nozzle to it. They use no electricity – just water pressure – and cost about $30.

I would dearly love one of those fancy Japanese Toto washlets with warm-water spray and air drying and all of that, but honestly, for thirty bucks, a cool water spritz is totally fine.


Does it plug into the hose leading into a tank? This means the water is cold, right? That will wake you up in the morning!

So you really have a bidet attachment, not a true bidet. And I thought you were rich or something…

I got a spray hose for cleaning cloth diapers that attached to the side of the toilet and OMG that thing was also super useful for cleaning the bathroom in general.


Well, in my defense, I did refer to it as a “bidet seat for my toilet”. And it was free :slight_smile:

It does get its water directly from the tank (it takes maybe five minutes to install these things) so the water is cool, but tolerable. It’s definitely a puckering wakeup in the winter! shiver