Almost 1500 people largely agree: If you want to monitor your house with a camera, the Instar IN-6014HD is a good choice. Almost three quarters of Amazon reviewers award five stars, and only five percent penalize the product with one or two stars. This results in an average rating of 4.5 stars – and thus a clear purchase recommendation.
The Stiftung Warentest sees it very differently. In 2017, testers compared 16 indoor and outdoor surveillance cameras. Both Instar devices ended up in last place with a rating of “poor”. The cameras are vulnerable to hackers, contacted Internet servers in China and put the user name and password on the Internet without encryption. There is nothing of this in the customer reviews on Amazon: Most of the negative reports are based on defects or technical problems.
4.5 stars on Amazon, a school grade of 5.0 at Stiftung Warentest: what sounds like an outlier often happens. Scientists from the Technical University of Dortmund have analyzed the ratings of hobby reviewers and professional testers. Their conclusion: Amazon ratings are unsuitable for many products in order to make a purchase decision. The results of the product testers often differ drastically from the online reviews.
For their study, the researchers brought together almost 2,500 electronic products that Stiftung Warentest evaluated between 2014 and 2017. Around half of them have been reviewed at least five times on Amazon. In less than a third of the cases, the test winner from Stiftung Warentest received the best customer ratings. Overall, the correlation is extremely low with a value of 0.18 (0 stands for no agreement at all, 1 for congruent results). The longer the test is in the past, the greater the deviation. In some product groups, the values were even negative: what is liked on Amazon falls through on the foundation – like the Instar camera.
Online reviewers are often emotional, the Stiftung Warentest is objective
To a certain extent, the sometimes large discrepancy can be explained. Anyone who submits an online review usually has a clear opinion: the product will bring great joy or great annoyance. Balanced three-star ratings are rare on Amazon, with the majority of reviewers either praising or scolding. In the laboratory, the distribution is exactly the opposite: few products are perfect or insufficient, but there is a broad middle field.
In addition, the professional testers record many criteria that normal buyers cannot check at all. They cut open headphones and examine them for carcinogenic substances, hire external engineers to assess the electrical safety of hair straighteners, or analyze data streams from surveillance cameras.
In addition, the Stiftung Warentest does carry out scientific tests, but the final score has only limited significance. Sometimes the testers devalue products because the declaration on the label is wrong or limit values for certain pollutants are just exceeded. For some users it may be important whether the smartphone survives the drop test in the metal drum unscathed. Others put their cell phone in a case and attach more importance to the quality of the camera, which accounts for only 15 percent of the overall mark at Stiftung Warentest. It therefore makes sense to take a differentiated look at the test reports: A good or satisfactory device can be better suited to your own requirements than a product with a very good overall rating.
Manufacturers try to buy positive fake reviews
But there are other reasons for the sometimes striking differences of opinion between Amazon buyers and foundation testers. They have less to do with different standards than with criminal activity. Almost all potential buyers read online reviews and use them as a guide. That attracts scammers. Some manufacturers refund the purchase price after five-star ratings, sometimes they pay money for purchase recommendations.
Amazon emphasizes that it invests “significant sums of money” “to protect the credibility of the reviews on our website”. There are clear eligibility requirements for reviewers, controlled by human review teams and automated systems. But despite these measures, consumer advocates continue to uncover systematic fraud, and Amazon deletes tens of thousands of fake ratings.
This is another reason why Amazon has developed its own model to calculate the overall rating of a product. The number of stars does not correspond to the average of all reviews. Rather, Amazon relies on machine learning and takes various factors into account. This includes the age of the rating and the assessment of other users: If many customers find a review useful, the corresponding criticism is weighted higher. Of course, the fraudsters also know this and try to manipulate the system anyway.
How to use online reviews correctly
Still, customer reviews can be helpful – if you know how to use them properly. In principle, caution is advised when it comes to celebrations. Sometimes competing companies pay for slating, but five-star ratings are bought far more often. If several users describe the same flaws, this indicates a real vulnerability. Here it helps to filter the reviews. For this, Amazon offers the selection option “All critical” after clicking on “All reviews”. If you are interested in certain properties, you can also search the comments in a targeted manner.
To assess the credibility of a review, it can help to look at the reviewer’s profile. Anyone who evaluates dozens of surveillance cameras either has an unusual hobby or financial motives. Sometimes it makes sense to search the web for individual sentences from a review. If the wording appears word for word on other portals, that should make you suspicious.
The site Reviewmeta.com is useful. The service analyzes Amazon customer reviews and weights them according to its own criteria. It takes into account more factors than Amazon, including different reviewer ratings, identical passages, suspicious repetitions, and comment length. Based on this, Reviewmeta ranks reviews according to credibility and assigns a new overall score. This system is not perfect either: the Instar camera falls from 4.5 to 4.1 stars – the Stiftung Warentest sees it much more critically.
