Page 1 of 2 12 LastLast
Results 1 to 15 of 16

Thread: Some reviews are hard to believe

  1. #1
    Join Date
    Apr 2007
    Location
    New Jersey
    Posts
    1,347
    Blog Entries
    1

    Some reviews are hard to believe

    I saw a review of scroll saws by Woodsmith.
    They rated the Wen as the best, better than the Dewalt and the Delta.

    I find this surprising and ,to be honest, hard to believe.
    Dennis

  2. #2
    I have a wen scroll saw and find it to be pretty decent.

  3. #3
    Join Date
    Apr 2007
    Location
    New Jersey
    Posts
    1,347
    Blog Entries
    1
    Quote Originally Posted by John Redford View Post
    I have a wen scroll saw and find it to be pretty decent.
    I don't doubt that the Wen is pretty decent, I have a Wen sander and an air cleaner and they are good tools. I just don't believe the Wen scroll saw is better than a Dewalt or Delta.
    Dennis

  4. #4
    Join Date
    Mar 2006
    Location
    SoCal
    Posts
    22,494
    Blog Entries
    1
    You have to look at the qualifiers. If price is one of the qualifiers cheap tools tend to rate higher than one might expect.
    "A hen is only an egg's way of making another egg".


    – Samuel Butler

  5. #5
    Join Date
    Apr 2018
    Location
    Cambridge Vermont
    Posts
    2,282
    I don't know how you can open up a box, take a tool out, and rate it. My best tools are ones that function just as they did the day I bought them years (or even decades later). In fact I have had cheap tools with features that seem better than more expensive tools but quickly failed. Even the so called long term test of cars where they keep them for 40k miles is mostly meaningless. Check back after 140k miles and see which car is the best.

  6. #6
    Well, just because someone reviews something and posts it, or puts it into a magazine or website or video, doesn't mean the actually know what they are talking about. Same if they are unbiased or an engineer (even woodworkers in woodworking magazines sometimes), they still might not know what they are talking about (even if they are truly unbiased; that doesn't offset being unknowledgeable).

    Consumer Reports sometimes is the worst. They do their tests, and the tests might be done correctly, but they often don't matter to the actual usage of the product, or what a professional user of the item would be looking for. My favorite Consumer Reports one is where they differently rate two brands that are in fact identical (e.g. Ford Tempo/Mercury Topaz), and then defend it by saying "that is what the data said". Well, maybe you don't understand your data then. I recently saw a digital piano action comparison on a music website that showed different downstrike weights on different keyboards . . . with the exact same action (sometimes even scientific oriented reviewers don't understand the concepts of tolerance and variance).

    The other thing is that the review criteria might not be relevant or consciously/unconsciously manipulated. Another of my favorite Consumer Reports examples was a ranking of vehicles that lasted the longest. They did a some analysis of vehicle registration milages and found out what vehicles had the most miles on them. The list was exactly what you would expect: Ford F-150 pickups, Chevy Suburbans, Dodge Durangos, Jeep Cherokees, etc, basically the vehicles that run forever and that people fix and drive forever. This apparently surprised them, because none of them were vehicles they rated highly. So, rather than reexamining their rating criteria, they changed the criteria of the ranking and made it so Hondas, Toyotas and Lexi came out on top.

    Even fairly knowledgable people can be off the mark sometimes. There is a YouTube guy that normally does pretty good reviews, but he did one on what wood glue was the strongest. He then did a bunch of tests that would make most woodworkers cringe, as they were non real-world woodworking situations. The main problem with the test though, is that as most woodworkers know, any standard wood glue is stronger than the wood itself, so the relative strengths between the glues really aren't relevant, what matters are the other characteristics (open time, initial tack, cure time, water resistance, etc). Despite having an unbiased review from a relatively qualified person, the tests themselves were flawed and the entire test not relevant.
    Last edited by Andrew Seemann; 10-21-2021 at 11:25 PM.

  7. #7
    PLUS ONE ANDREW. An excellent presentation of why we all should apply the "old saw" Caveat Emptor. Classic apples and oranges.
    Dave Anderson

    Chester, NH

  8. #8
    Join Date
    Jan 2004
    Location
    Lewiston, Idaho
    Posts
    28,504
    When I read reviews I try to find several different sources that reviewed a product to get different views.
    Ken

    So much to learn, so little time.....

  9. #9
    Join Date
    Apr 2013
    Location
    Kansas City
    Posts
    2,652
    I disagree. I am a long time fan of Consumer Reports, and they base much of their recommendations on reliability as reported in surveys of owners, and owners willingness to buy again. Major remodels of vehicles score low when there isn't much history of the newer models, but they always show the owner reported history over several years.
    < insert spurious quote here >

  10. #10
    Join Date
    Apr 2007
    Location
    New Jersey
    Posts
    1,347
    Blog Entries
    1
    I often look at "Project Farm" . He does some incredible testing and reviews, things such as glues, tools , etc
    Dennis

  11. #11
    Join Date
    May 2009
    Location
    Wenatchee. Wa
    Posts
    769
    Quote Originally Posted by Stan Calow View Post
    I disagree. I am a long time fan of Consumer Reports, and they base much of their recommendations on reliability as reported in surveys of owners, and owners willingness to buy again. Major remodels of vehicles score low when there isn't much history of the newer models, but they always show the owner reported history over several years.
    Thank you Stan for defending CR. Using the information from their test reports has saved me many dollars and time over the years.

  12. #12
    Join Date
    Mar 2010
    Location
    Somewhere in the Land of Lincoln
    Posts
    2,546
    Quote Originally Posted by Bernie Kopfer View Post
    Thank you Stan for defending CR. Using the information from their test reports has saved me many dollars and time over the years.
    If one takes their (CR) data as gospel there would be Ford F-Series, Chevrolet Silverado's, and Ram pickups littering the roads broken down because they are so unreliable. The reality is there are more pickups with extremely high miles than most passenger cars. I know of several in all three brands with 3 and 4 hundred thousand miles and the only shop time was for wear and tear maintenance. Brakes, shock's, and scheduled maintenance. Sure some people have trouble but while there are indeed some lemons out there some people also have trouble with everything so there might be a pattern there. I'm not dinging Toyota or Honda because we know their reputations for usually getting it right. Every dealership has a service department and I'm certain you will find mechanics working on vehicles at everyone of them. I just view CR skeptically. I've bought my share of new vehicles over the years and I was never contacted for my opinion by them on any one of them. So where does their data come from? Their subscriber base? I don't subscribe and never have. I've looked at their reviews at different times but I bought what I felt fit my needs and not what they recommended. The rule of thumb is shy away from brand new models for the first year until it's known if they have issues. I just see CR as another source of info but they aren't the authority to go by exclusively.

  13. #13
    Join Date
    Apr 2013
    Location
    Kansas City
    Posts
    2,652
    Yes, of course they can only survey their membership base. They ask about other products too, not just vehicles. And they never said those trucks are unreliable. Just point out if and when they have above average number of reported problems over time. They report on new cars based on clearly stated performance measures, and long-term durability is not the primary or only factor. Their goal is not to sell cars (unlike other testing platforms) but to inform people shopping for cars. And they dont rank each individual vehicle like some kind of contest, just give a rating on performance and value. Sure its all just information and data that no one takes as gospel. If there is another independent source that you think is more reliable please share.
    < insert spurious quote here >

  14. #14
    Join Date
    Sep 2009
    Location
    Medina Ohio
    Posts
    4,516
    I rate CR, Better Homes and Garden, JD Powers all in the same boat. you need to ask them to compare and you have to pay them

  15. #15
    Join Date
    May 2014
    Location
    Alberta
    Posts
    2,162
    If you really want to know what is good and what isn't about a vehicle you are looking at check out Phil Edmonstons "Lemon Aid" series. Truly unbiased and shows no mercy exposing all recalls ,issues and warts of cars and trucks. Unfortunately he has now retired and discontinued his life work of publishing these books.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •