By Marc Phillips
Just a few weeks ago I received an email from a retail customer who wanted to purchase one of the products I represent. He contacted the appropriate dealer, arranged for an audition, took the product home and within a couple of days discovered that it brought great joy to him and his loved ones, enough to warrant the writing of a check. But before he made his final decision, he needed to email me with an all-too-common request:
“Could you provide me with links to reviews?” he wrote. “That will help me to decide.”
In the privacy of my office I huffed and puffed and rolled my eyes. But I acquiesced and checked my own website for an appropriate review, and lo and behold I had one to send back. Unfortunately it was a horrible review, one of my least favorite, and not because it trashed my product. The reviewer, in fact, had praised the product to the skies and even wound up purchasing it for his own reference system. The problem, however, was in execution: the writing was that of an amateur, the editing seemed to be non-existent and even the review website itself was sloppy, garish and downright annoying to view.
I emailed that review out, and the dealer got the sale. But I kept asking myself, why was obtaining a review the last step of the buying decision—instead of one of the first? In my equipment-reviewing days we held fast to the notion that a review was always the first step of a well-informed, intelligent buying process, that audio equipment reviews were designed to spotlight certain products that the reader might find interesting. The next steps, of course, were going to a dealer, listening to the product, taking it home to see how the product fared in a familiar environment, and finally whipping out the checkbook or credit card.
The real question I wanted to ask the customer was this: Do you really know this reviewer well enough to blindly trust his opinion? Do you know who he is, or what his qualifications are to review equipment? Or did you just need an insurance policy for when your audiophile buddies come over and listen to your new component, something to throw back at them when they ask, “Why did you buy this? Why didn’t you buy the [insert positively reviewed component] instead?”
In this current Age of Information, the preponderance of “experts” can be overwhelming, especially for consumers. In the old days we had people like J. Gordon Holt, Harry Pearson and many other lighting the way for the hobby, making observations based upon technical experience, rigorous testing protocols and, hopefully, some semblance of an entertaining writing style. Most of these people had personalities to go along with their opinions, so it was fairly easy to identify with their tastes and preferences and then apply those judiciously to our own sensibilities.
But these days, it’s much more difficult to ascertain those credentials. Many among the current crop of Internet audio writers seem to be made up of audiophiles who grew up reading equipment reviews from the likes of Stereophile and TAS, so they know what a proper review looks like. But who are these people, and why should we listen to them?
I’ll give you three instances, right off the top of my head, of why you shouldn’t listen to an audio reviewer just because they’re an audio reviewer:
- A couple of years ago, I attended one of the smaller regional trade shows for high-end audio. In one room, I noticed a man sitting in the sweet spot and scribbling copious notes on a yellow legal pad. The exhibitor whispered to me that this gentleman was an “important reviewer” for one of the online mags. The next day, I decided to check out this fellow’s website and some of his past reviews. What I found on the home page was his “World’s First Review!!!” on the very product he was listening to in that room just the day before. In other words, this guy was performing his in-depth “reviews” while attending trade shows.
- I asked an online website a specific question about one of their writers, and found out that this writer’s name was merely a pseudonym used by the publisher/editor to give the appearance that he had a “staff of writers” working for him—when in reality, it was all basically a one-man show.
- A manufacturer at a trade show recently told me that they had to stop sending a certain well-known reviewer products to review because those products kept shipping back to the manufacturer with the factory tape and seals still intact. Needless to say, that didn’t prevent the reviewer from publishing a rave review, nor hitting up the manufacturer afterward for a full-page ad.
I’ll even offer a personal anecdote based on my own experience as a reviewer. Do you know what a “pull quote” is? That’s when an advertisement contains a quote from a positive review as a selling point for the product. In all my years of reviewing, I only had one pull quote ever used in a print advertisement. I would have celebrated that accomplishment, but my enthusiasm was tempered by the fact that the quote used in the ad did not belong to me. It was written by an editor who wanted to make my review more positive in order to please the manufacturer. In other words, my only personal endorsement in the pages of a print magazine was complete and utter bullshit. I’m less than happy about that.
So am I saying that you shouldn’t trust reviewers? Of course not! There are plenty of excellent, trustworthy audio scribes out there who provide a genuine and valuable service to audiophiles. Generally speaking, the guys who have been doing it forever and a day are the ones who have resisted temptation and have managed to hold onto their integrity—even in a marketplace consumed by unmeasurable social media campaigns and questionable Google hits. There’s certainly a lot of talk among audiophiles—especially on audio discussion forums—about the more unsavory tactics employed by reviewers and their magazines such as swapping favorable reviews for advertising dollars and free equipment, and then dumping the products on AudiogoN for a little extra money on the side. But in a vast majority of cases, these shenanigans are imagined—and for the simple reason that reviewers and publications can’t hope to have any longevity in this industry if they aren’t being as honest as possible about what they really do for a living.
The first step in using equipment reviews as part of an informed buying decision, of course, is to find a reviewer who has the same tastes as you. This won’t happen overnight. I started subscribing to Stereophile in 1985, and I didn’t apply the knowledge I’d gained from their reviewers until well into the ‘90s. By then I had learned that I shared the same love for British hi-fi as Sam Tellig, the same musical wild streaks as Corey Greenberg and the same curiosity for exotic yet old-fashioned designs as Art Dudley (someone I discovered a few years earlier, in his days of helming Listener). If they loved a product, I sought it out for myself to see if my opinions coincided with theirs. Yes, I have made some unwise buying decisions over the years, but not because I blindly trusted a reviewer.
In fact, most of the first high-end audio purchases I made—Snell Type J and Spendor S20 loudspeakers, for example—were based purely on my listening experiences and not reviews. I remember the Spendors receiving a luke-warm review from Stereophile a good year after I purchased them, but I didn’t mind. I knew they made me happy, and that was all that counted.
Today, it seems to be a different story. We’re all so busy, busy, busy. We don’t have time to audition everything out there (which seems to run counter to my belief that audiophiles have to listen to absolutely everything before they make a buying decision), so it’s just easier if we ask some random strangers on an Internet forum if one speaker is “better” than another. It’s easier to do a Google search on whatever product is getting buzz at that particular moment. It’s easier to trust a stranger than our own ears. That’s because we audiophiles secretly dread inviting our buddies over to hear the latest upgrade. There’s nothing worse than spending a lot of money and then being told you made a mistake. So finding a trustworthy and knowledgeable guide is one way to avoid a potential criticism.
The next step after finding a reviewer you can trust is to find more reviewers you can trust—because one guy can occasionally screw up and get it wrong. The industry is full of stories of reviewers who mated inefficient speakers with low-powered tube amps and weren’t that thrilled with the sound—resulting in a less-than-enthusiastic review. Occasionally a product suffers a minor mishap during shipping that results in damage to a minor part—fuses, resistors, etc.—that causes the product to operate somewhat normally, albeit with compromised sound quality, and the review still goes forward. (My biggest headache usually comes from ribbon cables inside the unit that somehow become unplugged as they travel halfway around the world.) So going with one review from one guy is not the smartest thing you can do in this hobby.
When it comes to personal preferences, you really need a consensus before you can gather truly relevant data. One of my favorite websites in the world is Metacritic.com, where movies, music releases, video games and TV programs are all assigned a weighted score from 1 to 100 based upon all the reviews that exist, as opposed to just one. I know that if a film scores above 80, for example, the likelihood that I will enjoy it is high. Reading a single film review in the local paper, as you might imagine, is a far less reliable way to predict quality. What if that film critic hates Adam Sandler, and I love absolutely everything he does? If so, then that review is simply not useful to me. It’s the same for audio reviews. Why would you take a tube amp review seriously when the reviewer goes out of his way to say he prefers solid-state? I’ve seen it happen. It’s not right. And you shouldn’t take that reviewer’s advice.
Once you assemble a list of writers you trust, then you can use their opinions as a tool to find the right high-end audio component for you. Contact dealers, make appointments, bring your own music and then ask if you can take it home once you’ve determined that the purchase is a true possibility. A review is a starting point. This hobby is a long, long journey. If you cut corners—and buying high-end audio components based merely on a single good review is cutting corners—then chances are you’ll never be truly happy pursuing a hobby that’s ultimately about finding happiness.
About the Author
Marc Phillips has been writing about turntables and LPs under the Vinyl Anachronist moniker since 1998. Since then he has written for such publications as Perfect Sound Forever, Ultimate Audio, TONEAudio, Positive Feedback Online and much more. Since 2011 he has partnered with Colleen Cardas to form Colleen Cardas Imports, the US distributor and importer for Unison Research, Opera, PureAudio and Axis.
He currently lives in Western Colorado with Colleen and their dog/CCI mascot Lucy, where he hikes, bikes and constantly complains about the paucity of good cigar stores and record stores within a 300-mile radius.
I have a little evil rule about reviewers…If they review audio cables subjectively I won’t trust them. This goes for sites also.
I mostly browse websites for news about gear, so I trust few reviewers. Its all about the money so I take all websites and reviewers that make a living from the hobby with a grain of salt. In fact I might actually care more about the opinion of a fellow enthusiast than a reviewer.
Everyone needs a metric!
Look, it may be that you feel everyone is out to rob you. I find this a little extreme, but if that’s the mental block-and-tackle maneuver you need to make in order to keep your money where it belongs, then more power to you.
For me, I don’t trust reviewers either. Especially me. I’m a total liar.
Scratch that.
I trust most reviewers. I do! I generally believe that they believe that the thing they’re writing about is good, is interesting, is worthwhile. That does not mean I’ll agree with their take, however. For while I believe them and their enthusiastic fervor, I know that my preferences and predilections are probably going to be quite different. Because they’re not me.
I guess that’s almost the same as the “grain of salt” thing. “Trust, but verify” might be a better way to capture my approach, though. Over time, you may (or may not) find an reviewer who’s preferences and predilections line up quite smartly. If you’re inclined to make purchases that require a leap of faith of some kind (perhaps you can’t try before you buy, say), then that sort of sympatico is rather helpful — assuming your favored reviewer has in fact explored the products most relevant for you.
If you don’t “believe in” audio cables, and the sonic impact they might have on your audio system, then you’re a lucky man — you get to keep that money, along with that smug belief that you’ll never be tricked out of it by a lying reviewer. I hope that belief endures. It’ll certainly be cheaper for you.
But using the “cable thing” as a barometer of trust is kind of a silly metric. Sure, you’re entitled to disagree with a review — reviews are all, by definition, op-ed pieces — but calling the reviewer “untrustworthy” because they choose to do something in a way other than what you would do is bigoted, isn’t it?
I actually respect people who objectively try to test cables, like archimago for example.
One of my favorite audiophile writers is Mr. Roger Skoff…I’m sure you know what he did for a life and what’s his expertise. I respect him because he writes about cables but doesn’t review them at least not that I know it.
Btw, don’t take my comment as a personal attack, it’s just my opinion.
I understand and respect your opinion but eh! I’m one of those hard headed objective kind of guys that has too many engineers around him. XD
Best wishes to you Mr.Scot.
Ps: If I knew you reviewed cables around here I wouldn’t post my comment, it was not written as a provocation, I leave that to my personal place. I usually respect the owner of the house (site).
My apologies.
Roger is a marvelous writer. We were very happy to publish his work here!
As for me, “testing cables” (or any gear) in a “objective” way means an investment in testing gear, and a significant investment in education to both learn to not only use them effectively, but to interpret the results they spit out in both a meaningful and a relevant context. That’s expensive.
I can assure you, I — and the vast majority of my peers — do not have the scratch to pull that off.
It’s also not entirely clear such information would actually inform. For that to work, the readers would also need at least some of that education ….
But we digress.
I go back before Stereophile to Stereo Review. I’ve been active in audio hobby since early seventies. I like John Atkinson’s follow-up to Stereophile component reviews. With PTA I like Mr. Grandberg’s component reviews. For music reviews I usually read Gramophone and Downbeat.
When I first got involved in this hobby I relished the experience of attending trade shows. In the course of a day or two you can survey a vast amount of equipment and sort out what kind of sound is most appealing. Learning one’s own preferences is key to making successful purchases–opinions of your friends be damned! Late at night when you need the music to move you, they’re not there anyways. So pack a bag, hit the road or take a plane and seriously indulge yourself in any of the shows across the country. When you get there–talk to people! Distributors, retailers, even the manufacturers themselves! You’d be surprised what you can learn from the people that actually build this stuff. Or, in the absence of that possibility, scour the web and print publications for show reports to learn what excited the reviewers. The cream usually rises to the top.
Excellent article Marc. Although not the case with the person who contacted you for a list of reviews, I feel the decline of the specialty retailer has helped elevate the status of product reviews. This likely applies to most merchandise. My town used to have several good audio specialists, with a large selection of components to demonstrate. Now we have but one, providing limited choices, time and inclination to commit to demonstrations. This is not intended to adversely indict the dealer, but is simply the reality of the marketplace and the competitive landscape. Our only alternatives are 200-300 mile road-trips or buying online. If consumers are limited in their ability to audition products, reviews, poor as they may be, become one of the few sources of additional information we have at our disposal.
I wanted to include a paragraph on audiophiles who must trust reviews because they live too far away from dealers, but I’ve written about it a couple of times already (in my Perfect Sound Forever columns). I feel for the guy who lives in the middle of nowhere, but I have to assume that if you’re a serious audiophile or music lover, you probably have the means to travel a few hundred miles to a dealer or, even better, a high-end audio show. You can also find out if there’s a local audiophile society or club where you can hear other people’s systems. Ultimately, it comes down to the cost of traveling to the closest dealer vs. spending a lot of money based on reviews and not getting what you wanted.
I’ve lived in the middle of nowhere before, but I still managed to get out and hear a lot of gear.
One of my favorite reviewers was a guy who called himself the Vinyl Anachronist, yes Marc himself. The common thread was a link back to dealer extraordinaire, Gene Rubin; a more sensible dealer one is not likely to meet. From there Corey Greenberg helped through the early modest system days, along with “Cheapskate”, Sam Tellig. Then the most thoughtful and common sense of them all, is for me still Art Dudley. Before Listener, with Art, was Hi Fi Heretic whose back issues are still on my bookshelf and often referred to. Great article Marc, as always and thank you.
Thank you for the kind words, Mike! I’m a-blushin’.
Well said Marc. I keep files on reviewers and have since being in the business. My ‘files’ are not meant to judge as good or bad. Those that are adequate in their process, can generally be identified by careful reading…is this an announcement or a review? The files are to assess camps or taste.
Just as you have different schools of thought in wine reviewing (robert parker = bordeaux, clive coates = burgundy) so there are different schools within audio. None less deserving than the other.
In terms of ‘reviews’, by folks really just jotting down some thoughts, well, they’re all to often taken seriously, until buyers risk their hard earned and come up with different results. We are approached for reviews all the time. Rarely is the solicitor adequate in room and equipment to warrant a review loaner. That technical aspect being taken separately from the issue of whether or not they can convey ideas and thoughts via the pen. (or computer) Upon being questioned about their set up and room, you can hear ‘reviewers’ squishing around in their chairs. Being a good reviewer is really a multi faceted and skilled affair that requires both left and right brain prowess. And it’s not always the most thankful position.
All to say, you should consider reviewing again Marc. We’d love to have you review some things we carry, in fact, I’ve already worked out some of the quotes for you.
FC
Agreed on all counts, Fred. I miss reviewing, but what I’m doing now is far more fun!
..one should only trust reviews penned by Jon Valin…or the late Julian Hirsch
“…one should only trust reviews penned by Jon Valin…or the late Julian Hirsch”
By this do you mean ‘trust’ to be not believable?