Good Reviews and Bad Decisions







By Marc Phillips

Just a few weeks ago I received an email from a retail customer who wanted to purchase one of the products I represent. He contacted the appropriate dealer, arranged for an audition, took the product home and within a couple of days discovered that it brought great joy to him and his loved ones, enough to warrant the writing of a check. But before he made his final decision, he needed to email me with an all-too-common request:

“Could you provide me with links to reviews?” he wrote. “That will help me to decide.”

In the privacy of my office I huffed and puffed and rolled my eyes. But I acquiesced and checked my own website for an appropriate review, and lo and behold I had one to send back. Unfortunately it was a horrible review, one of my least favorite, and not because it trashed my product. The reviewer, in fact, had praised the product to the skies and even wound up purchasing it for his own reference system. The problem, however, was in execution: the writing was that of an amateur, the editing seemed to be non-existent and even the review website itself was sloppy, garish and downright annoying to view.

I emailed that review out, and the dealer got the sale. But I kept asking myself, why was obtaining a review the last step of the buying decision—instead of one of the first? In my equipment-reviewing days we held fast to the notion that a review was always the first step of a well-informed, intelligent buying process, that audio equipment reviews were designed to spotlight certain products that the reader might find interesting. The next steps, of course, were going to a dealer, listening to the product, taking it home to see how the product fared in a familiar environment, and finally whipping out the checkbook or credit card.

The real question I wanted to ask the customer was this: Do you really know this reviewer well enough to blindly trust his opinion? Do you know who he is, or what his qualifications are to review equipment? Or did you just need an insurance policy for when your audiophile buddies come over and listen to your new component, something to throw back at them when they ask, “Why did you buy this? Why didn’t you buy the [insert positively reviewed component] instead?”

In this current Age of Information, the preponderance of “experts” can be overwhelming, especially for consumers. In the old days we had people like J. Gordon Holt, Harry Pearson and many other lighting the way for the hobby, making observations based upon technical experience, rigorous testing protocols and, hopefully, some semblance of an entertaining writing style. Most of these people had personalities to go along with their opinions, so it was fairly easy to identify with their tastes and preferences and then apply those judiciously to our own sensibilities.

But these days, it’s much more difficult to ascertain those credentials. Many among the current crop of Internet audio writers seem to be made up of audiophiles who grew up reading equipment reviews from the likes of Stereophile and TAS, so they know what a proper review looks like. But who are these people, and why should we listen to them?

I’ll give you three instances, right off the top of my head, of why you shouldn’t listen to an audio reviewer just because they’re an audio reviewer:

  • A couple of years ago, I attended one of the smaller regional trade shows for high-end audio. In one room, I noticed a man sitting in the sweet spot and scribbling copious notes on a yellow legal pad. The exhibitor whispered to me that this gentleman was an “important reviewer” for one of the online mags. The next day, I decided to check out this fellow’s website and some of his past reviews. What I found on the home page was his “World’s First Review!!!” on the very product he was listening to in that room just the day before. In other words, this guy was performing his in-depth “reviews” while attending trade shows.
  • I asked an online website a specific question about one of their writers, and found out that this writer’s name was merely a pseudonym used by the publisher/editor to give the appearance that he had a “staff of writers” working for him—when in reality, it was all basically a one-man show.
  • A manufacturer at a trade show recently told me that they had to stop sending a certain well-known reviewer products to review because those products kept shipping back to the manufacturer with the factory tape and seals still intact. Needless to say, that didn’t prevent the reviewer from publishing a rave review, nor hitting up the manufacturer afterward for a full-page ad.

I’ll even offer a personal anecdote based on my own experience as a reviewer. Do you know what a “pull quote” is? That’s when an advertisement contains a quote from a positive review as a selling point for the product. In all my years of reviewing, I only had one pull quote ever used in a print advertisement. I would have celebrated that accomplishment, but my enthusiasm was tempered by the fact that the quote used in the ad did not belong to me. It was written by an editor who wanted to make my review more positive in order to please the manufacturer. In other words, my only personal endorsement in the pages of a print magazine was complete and utter bullshit. I’m less than happy about that.

So am I saying that you shouldn’t trust reviewers? Of course not! There are plenty of excellent, trustworthy audio scribes out there who provide a genuine and valuable service to audiophiles. Generally speaking, the guys who have been doing it forever and a day are the ones who have resisted temptation and have managed to hold onto their integrity—even in a marketplace consumed by unmeasurable social media campaigns and questionable Google hits. There’s certainly a lot of talk among audiophiles—especially on audio discussion forums—about the more unsavory tactics employed by reviewers and their magazines such as swapping favorable reviews for advertising dollars and free equipment, and then dumping the products on AudiogoN for a little extra money on the side. But in a vast majority of cases, these shenanigans are imagined—and for the simple reason that reviewers and publications can’t hope to have any longevity in this industry if they aren’t being as honest as possible about what they really do for a living.

The first step in using equipment reviews as part of an informed buying decision, of course, is to find a reviewer who has the same tastes as you. This won’t happen overnight. I started subscribing to Stereophile in 1985, and I didn’t apply the knowledge I’d gained from their reviewers until well into the ‘90s. By then I had learned that I shared the same love for British hi-fi as Sam Tellig, the same musical wild streaks as Corey Greenberg and the same curiosity for exotic yet old-fashioned designs as Art Dudley (someone I discovered a few years earlier, in his days of helming Listener). If they loved a product, I sought it out for myself to see if my opinions coincided with theirs. Yes, I have made some unwise buying decisions over the years, but not because I blindly trusted a reviewer.

In fact, most of the first high-end audio purchases I made—Snell Type J and Spendor S20 loudspeakers, for example—were based purely on my listening experiences and not reviews. I remember the Spendors receiving a luke-warm review from Stereophile a good year after I purchased them, but I didn’t mind. I knew they made me happy, and that was all that counted.

Today, it seems to be a different story. We’re all so busy, busy, busy. We don’t have time to audition everything out there (which seems to run counter to my belief that audiophiles have to listen to absolutely everything before they make a buying decision), so it’s just easier if we ask some random strangers on an Internet forum if one speaker is “better” than another. It’s easier to do a Google search on whatever product is getting buzz at that particular moment. It’s easier to trust a stranger than our own ears. That’s because we audiophiles secretly dread inviting our buddies over to hear the latest upgrade. There’s nothing worse than spending a lot of money and then being told you made a mistake. So finding a trustworthy and knowledgeable guide is one way to avoid a potential criticism.

The next step after finding a reviewer you can trust is to find more reviewers you can trust—because one guy can occasionally screw up and get it wrong. The industry is full of stories of reviewers who mated inefficient speakers with low-powered tube amps and weren’t that thrilled with the sound—resulting in a less-than-enthusiastic review. Occasionally a product suffers a minor mishap during shipping that results in damage to a minor part—fuses, resistors, etc.—that causes the product to operate somewhat normally, albeit with compromised sound quality, and the review still goes forward. (My biggest headache usually comes from ribbon cables inside the unit that somehow become unplugged as they travel halfway around the world.) So going with one review from one guy is not the smartest thing you can do in this hobby.

When it comes to personal preferences, you really need a consensus before you can gather truly relevant data. One of my favorite websites in the world is Metacritic.com, where movies, music releases, video games and TV programs are all assigned a weighted score from 1 to 100 based upon all the reviews that exist, as opposed to just one. I know that if a film scores above 80, for example, the likelihood that I will enjoy it is high. Reading a single film review in the local paper, as you might imagine, is a far less reliable way to predict quality. What if that film critic hates Adam Sandler, and I love absolutely everything he does? If so, then that review is simply not useful to me. It’s the same for audio reviews. Why would you take a tube amp review seriously when the reviewer goes out of his way to say he prefers solid-state? I’ve seen it happen. It’s not right. And you shouldn’t take that reviewer’s advice.

Once you assemble a list of writers you trust, then you can use their opinions as a tool to find the right high-end audio component for you. Contact dealers, make appointments, bring your own music and then ask if you can take it home once you’ve determined that the purchase is a true possibility. A review is a starting point. This hobby is a long, long journey. If you cut corners—and buying high-end audio components based merely on a single good review is cutting corners—then chances are you’ll never be truly happy pursuing a hobby that’s ultimately about finding happiness.

About the Author

Marc Phillips has been writing about turntables and LPs under the Vinyl Anachronist moniker since 1998. Since then he has written for such publications as Perfect Sound Forever, Ultimate Audio, TONEAudio, Positive Feedback Online and much more. Since 2011 he has partnered with Colleen Cardas to form Colleen Cardas Imports, the US distributor and importer for Unison Research, Opera, PureAudio and Axis.

He currently lives in Western Colorado with Colleen and their dog/CCI mascot Lucy, where he hikes, bikes and constantly complains about the paucity of good cigar stores and record stores within a 300-mile radius.