Latest News

On Experts, Reviews, and Drive By Shootings: Part 3

Recently, Roger Skoff offered up a caricature of an argument around the so-called "expert" and skewered some of the challenges that appertain thereto. Heh heh. In short, Skoff argues that experts, generally speaking, aren't -- and that's okay because we're all still going to live. Good to know, right? Right. And your mother was a hamster and your father smelt of elderberries! Ha ha!

Ethics

In the first part of this" exploration", I wandered through a philosophical thicket of what a review is. Call that Part 1. We then took an extended detour around some of the problems of what one has. That was Part 2.1 and Part 2.2. So, let's take a moment and talk about where things go off the rails. There's this notion of "value" and corruption and ... stuff. For lack of a better frame, let's hang the shingle "Ethics" on it. That should let us shine a light on some unsavory places, if just for grins. Got your hip boots on?

Negative Value

Anton_EgoI’ve already mentioned that most reviews are pretty much ad copy. The sheer fact that someone is out there, writing about a product or its manufacturer, helps to promote that product or manufacturer. It’s “buzz”, at the very least. A review creates it, sustains it, and grows it. Creating and/or stealing “mind share” in a heavily populated and wildly competitive landscape like high-end audio can only help.

I can almost hear your hand shoot up and say, “but what about a negative review?” Excellent point, and there are a couple of things to point out here. Yes, a negative review can be damaging to the brand or the sales of a particular product. In fact, if you’re a high-profile reviewer like Art Dudley, you need to be rather careful lest you inadvertently drive someone out of the business. Happily, reviewers operating in such rarified air are cautious and their editors are even more so. Well, except when they’re not. (“Oops. Sorry about that retirement, hope for a vacation, operation, or plan to get your kids braces. Better luck next … life.”)

But even then, in this “very dire circumstance”, the (very) clever manufacturer has a couple of options. One, they can lift out the one good line and plaster that on a billboard. Two, they can immediately rev the product — totally possible when the “company” is tiny (i.e., one dude) and the products are all hand-made — and then broadly announce that the new version was made with inspiration by, in consultation with, and to directly address the concerns of … Anton Ego. Three, they can hope to average out the noise with more, positive, reviews. Four, they can sue the reviewer and his publisher. Yeah. Anyway. With that last set aside as an exception, negative reviews (while soul crushing) don’t have to mean shuttering.

Happily, truly negative reviews are rare as most products in the market that are actually products, and not some half-assed attempt at bilking an unsuspecting audience, are usually better-than-decent. That doesn’t necessarily mean that they’re great, or that a given reviewer is able to uncover that greatness even if it’s there (see Part 2.1 for more).

About that latter bit: most mixed-results reviews I’ve read online are replete with compatibility issues and setup decisions that are head-scratchingly difficult to justify, and leave me with a “well, no wonder you weren’t impressed” thought-bubble floating over my head and the words “WTF” and a “moron” popping out of my mouth. Bah. If nothing works, it may well be you that’s the problem. But I digress.

Who’s getting paid

So, back to ad copy. I should note that most reviewers are not paid by the manufacturers they review. At least, not directly. Sure, manufacturers and resellers choose to advertise in the same places that reviews appear because that’s where the relevant eyeballs are. But most organizations (other than the very small outfits) tend to keep some kind of wall up between the reviewer and the reviewed — there’s usually an intervening pair of hands between them, at least.

But make no mistake, a review isn’t there (wherever “there” is) for fun. It has one purpose: someone wants to sell something. Usually, that “something” is advertising (and for those periodicals that do not accept advertising, the ask is more direct — the content is there to sell subscriptions). The more content and/or higher the quality of that content, the more eyeballs, and more eyeballs means that more ads that can be sold and sold at a higher cost. Not to put too fine a point on it, but this is the ultimate point of a review. Sure, there’s value to the reader. Sure, there’s some value to “the public”, generically defined. But the review wasn’t created spontaneously, for its own sake, it was created to be sold. And it was bought because the buyer believed it could be used to draw eyeballs. It’s a product, first and foremost. It’s a lure to sell ads. Yes, even if the venue you found it in is Consumer Reports.

Sure, there are lots of places where you can find a review that was written just for the fun (or prestige) of it. Some, like this very site, were founded not as potential revenue generators, but as engines to force a lazy would-be writer to actually practice that skill. Some, unlike this very site, stay that way forever. Some make the jump and some end up jumping the shark …. But I digress again.

So, back to ad copy (again), and with all the above laid out, I’ll stick by this: A review is meant to sell something. I’ve linked this directly to ads sold by the publisher, but that’s hardly the end of it. Or, to put it more correctly, hardly the beginning of it: a manufacturer would never condone the practice of reviews if they felt that there wasn’t something they themselves will gain from it. That is, reviews also sell product — usually the one discussed in the ad. Duh. But again, it’s worth keeping it in mind. There are no innocents here.

It’s almost a miracle, then, that they actually still have value to anyone else.

But the fact is, they do. A well written review (1) sells ads, (2) sells product, and (3) provides something to the reader. Something they didn’t have before. Something they might need or want. Something of sufficient value that the reader actually returns for more. That last bit is absolutely key — and is what separates a good commercial publishing venture from a shabby one. A well written review, if it’s entertaining and informative, will actually bring returning eyeballs, which in turn sells more ads and more products. It’s in the interest of a publishing house, therefore, to seek out the best writers and present the highest quality content they can — for just those reasons. Which means walking a rather fine line: a review has to be valuable to the reader, to the manufacturer, and to the magazine in order to be profitable.

Which is why reviews, even though they’re pretty much equivalent to a 15 minute mid-day informercial TV spot, are still worth reading. Because if they weren’t providing infotainment, the publishers know they’d be selling less (subscriptions or ads) and the manufacturers know they’d be selling less, too. It’s in everyone’s interest that the review be useful to us, the consumer. Which is handy because, as Skoff told us, no one has time to sort through all this shit on their own.

Cheating and decision making

There is a complex formula a buyer may use to describe the arc between want and ownership, and I think most buyers would admit to the fact that reviews, both positive and negative, can and do influence their behavior. And rightfully so — and yes, I say that without a trace of smug self-satisfaction. The problem, at least as I see it, is that there is an incredible level of effort that one would have to exert in order to make an informed decision about what is most likely an extraordinarily expensive (relatively speaking) purchase. In fact, that level of effort hearkens directly back to Skoff’s description of why collecting exhaustive experience is pure futility. Which is why we cheat.

Cheating is pretty common, and I’m sure you’ll agree that it’s a fine thing to encourage. Given the alternative Skoff lays out, it’s not like we have a choice. Yes, I would go so far as to venture that cheating is a biological imperative and one of the (no doubt many) reasons there are so very few Neanderthals wandering about these days. In the fields of Science and Engineering, we tend to use the very fancy term “heuristic” to describe this sort of strategy, but after all the hand-waving, it’s just a cheat. A short cut. A ad hoc guide filled with “rules of thumb”. In the hunt for audio’s high-end, such an approach is just pure self-preservation, and the reviewer’s judgment is one of the mechanisms by which consumers zero-in on exactly how to divest themselves of the unwanted burden presented by excess cash.

I’ll simply stipulate that this happens, and refrain from a judgment here as anything I have to offer would be horribly biased. Ahem. But I will note that reviews aren’t the only mechanism deployed in the cheat. Forums, anecdotes, and Common Knowledge are all similarly deployed. The problem with them is tied to the problem of the expert, and this is why I bring them up — because, as a set, they tend to suck. 

Again, as Skoff mentions, there’s something interesting about the qualifications of an expert that make them different from JoeBob123 and HiFiKing1987. To wit: experience. Not saying that the JoeBob123 and HiFiKing87 don’t have experience, but just that it’s unclear if they do and whether or not that experience is helpful or nonsensical. We have unchecked and unverifiable claims to personal expertise, while a “pro” reviewer may (probably) has been vetted, at least to some degree (even if it isn’t to Gladwell’s threshold), or at the very least, has publicly shown himself to be reliable and opened himself up to vilification if he is not. This is that “bad habit of most audiophiles” that I referred to early on — there’re plenty of sources for information; the problem is that most of them are terrible and audiophile consumers tend to binge on all of them. All sources are not created equal. Some are better. Some are worse. And yes, that includes The Review. And yes, the problem is, precisely, a lack of expertise. I feel like we’re tracing circles here.

The easy answer is that you, Gentle Reader, really ought to trust those sources with demonstrable expertise (and probably only those). The problem with easy answers is that “easy” is usually short for “too easy” and here that’s true as well. There really aren’t that many Reviewers with expertise in all the facets of high-end audio that they’d need to have in order to write authoritatively about anything, much less, everything.

You see this in Stereophile, too. Art Dudley tends to not do reviews of solid-state gear. Michael Fremer tends not to do reviews of tube gear. Why? It’s not that they cannot appreciate those other domains, nor that they don’t have pretty clear opinions about what they like, it’s just that these categories of product may lie outside of their zone of expertise. The division is quite respectable, really. And quite frankly, I’m not sure I’d trust Dudley to give a fair accounting of modern high-power amps; or, more precisely, that I’d trust his opinion more than I’d trust Fremer about that category (and vice versa). And now we’ve reached yet another problem with experts — there just aren’t enough of them for any given domain in high-end audio critiquery. Which makes “consensus” — something that would be extremely helpful in making informed purchasing decisions about products we’ve never heard something of a joke.

Okay, so where does that leave us consumers? Square one? Well, not quite. If a review has value as a data point plotted out in our trajectory toward a future purchase, and if forum comments, anecdotes and Common Wisdom do also impact decisions (though, hopefully, to a radically lesser extent), our own personal purchasing calculus can get pretty hairy because as Skoff says, you can’t experience everything yourself.

I do want to throw out another variable, and one that gets overlooked these days. More so than in years past, especially. And that’s your friendly neighborhood dealer. Assume that they’re out to sell you something and move past that — what makes an audio dealer a  successful businessman is customer service. Providing excellent service is the only way they’ve managed stay in business in the age of Internet consumption. While they may not have experience with everything in the market either, they will most likely have experience with products in your prospective hunting class that tend to work well with others, and this kind of experience may well be at that “mastery” level. Do not discount this! This is what these professionals do, and I’ve never met a dealer yet that wasn’t able to totally school me about some aspect of system matching. I’m not saying you have to trust them, or that you have to love them, or that some of them aren’t snake oil salesmen or total slime balls or willing to eat their own young. But I know most of them are not. And if you’re looking for more data points to use in plotting out that arc toward audio triumph, ignoring this input is foolish.

Experts by the pound

Forbes columnist Allen St John sent me an article by Tom Nichols in The Federalist, called The Death of Expertise. It’s an interesting read:

I fear we are witnessing the “death of expertise”: a Google-fueled, Wikipedia-based, blog-sodden collapse of any division between professionals and laymen, students and teachers, knowers and wonderers – in other words, between those of any achievement in an area and those with none at all.

This mirrors what I’ve just been talking about, so that’s not a surprise. I call it out, specifically, because these conversation can go in a slightly different direction; we’ve discussed how reviews are valued, but I want to take a moment to talk about how experts are valued, because while they’re related, the topics are quite different.

To wit, experts tend to not be valued. Not as such.

Take the audio dealer, who in almost every case is the closest thing you’ve ever had to an actual, approachable, expert anywhere near your back yard. The audio dealer, that breed of entrepreneur who’s been sliding into the long, slow death of obsolescence due to [insert a 21st century version of the “Who Moved My Cheese” argument, here]. I saw a recent article by columnist and audio-expert, Steve Guttenberg, on brick-and-mortar headphone stores and how awesome they’d be. Honestly, I agree with him — I think they’d be a lot of fun and would help solve a ton of the en-pauper-izing swap/trade behavior that happens in headphone audio today. You could, in theory, sit in a comfortable environment and try out headphones and headphone paraphernalia to your heart’s content and end up at the end of the experience with a solution that’s tailored to your tastes. You could try options. Explore variations. Play with permutations. Discuss those options and variations and permutations at length with an actual expert. You’d learn something! And then you would, if you were the typical 21st century consumer, wander home and buy all that shit right off of Amazon, or from some out-of-state slime ball competitor willing to undercut the local hardworking shop on price and sell into their territory. And you’d probably have the balls to ask for a deep discount to boot — and you’d probably get one. Not to put too fine a point on it, but this is precisely what many now ex-dealers cite as reasons for why they’re ex-dealers.

We, the average consumer, simply don’t value the time and expertise found in the local shop to reward that by paying “premiums” (aka, MSRP). Which sucks — for us and for the experts we’ve put out of business. And this explains quite a bit of the pickle that the high-end audio industry is finding itself in today.

I want to believe that this is part of a larger anti-intellectual trend and general suspicion of “experts”, but there’s also the just-discussed fuzzy lines that lay between review and advertisement and the general belief that reviewers (and dealers, for some reason) are all irredeemably corrupt. Insert yet more shock and horror here.

You’ve probably heard something like this before: manufacturers over charge. By a lot. The MSRP of a given product is two, three, four — ten times what the product actually cost to make. It’s all a racket! Someone is “getting rich” … and it ain’t you.

Then there’s the viewers getting all those fat cash kickbacks or some kind of special considerations. Considerations like free gear, whether for indefinite periods “for review” or just personal use, crazy non-standard pricing (up to, and including, free), or massive insider access. As for me, I’m still waiting on my bags of cash, but whatever. As to the other stuff, well, some of that happens. It’s on the editor to sort that kind of thing out, if possible, and on the reviewer to not be a total schmuck. But people are people, and that means there will be idiots, morons, and a-holes mucking it up for everyone else — even among reviewers.

I’ve talked about the business of pricing products elsewhere, so I won’t rehash it here. All I can say is, yes, there’s gouging. That also happens. But for every instance of an “over priced product”, I can name at least one that’s under priced. Just remember that everyone needs to get paid and sometimes the “value chain” is a long, complicated affair. Don’t like it? Well, there’s always DIY.

I should also offer that given the levels of direct renumeration offered to reviewers (and being one for a major outlet does tend to give me a bit of an insider view here), allow me to say with extreme confidence that there are no reviewers getting rich off their skill with a keyboard. In fact, most cannot even make a living doing this as a full time job — which is why they don’t. TAS actually employs only a handful of people and most of them are administrative. Ditto Stereophile. The next tier of players in this space tend to be one-man-shows — and if they make a living, it’s a pretty thin one. As for me, there’s a very specific reason as to why the title of this site includes the words “Part-Time” in it.

As to the specialty pricing, kickbacks, the buckets of cocaine and the chorus line of bendy hookers, well, it sucks to be you. And yes, I’m kidding. Maybe. But the specialty pricing thing is totally real. Most respected audio reviewers can and probably do get offered special accommodation with respect to pricing. Not all the time. Some manufacturers simply don’t bother to do it. Some manufacturers don’t bother with anyone but the top-shelf in high-end audio publications. But for those that do offer it, it’s generally the case that there’s some discount for “industry insiders”, something usually along the lines of “dealer pricing”.

This isn’t all that weird. If the reviewer is above-board and willing to treat with the manufacturer directly (which they or their publisher is supposed to, in order to not get sued, and also to solicit information on the product, get guidance on proper setup and/or system matching, and to get access to the designer or the their on-staff experts, for just a few examples), it’s quite possible that the manufacturer will offer to sell that unit directly to the reviewer at the same price they would have sold it to their distributor or any particular dealer. The fact that the reviewer gets dealer pricing makes sense — the manufacturer isn’t getting any more or less on this transaction as contrasted with any other. The fact that the reviewer gets a good deal in pretty much accidental — in most cases, the review unit cannot be resold at the original asking price. It’s now b-stock, at best. In fact, a lot of the “review gear” I see has made a few stops before me on the reviewer circuit and the condition … varies. In any case, it’s in the manufacturer’s interest to unload these demos without having to restock, refurb, or pay anything more in shipping. No one is trying to screw or reward anyone; this is just part of the cost of doing business. If that price happens to be less than what a non-reviewer might pay for it in a shop, well, remember that this stuff never went to a shop, so that part of the financial equation was removed.

Another thing to note is that in every case there’s also a contract agreement that goes along with this one-off sale that explicitly prohibits the buyer from then selling the unit for some period of time, usually a year or more. Not exactly foolproof, but the reviewer that violates such agreements and then proceeds to pump and dump is not likely to remain a reviewer for long. Most editorial policies explicitly forbid such moves. Folks in the industry share this kind of story fairly freely, and yes, we all talk. But yes, fine — it is possible for an unethical reviewer to do it anyway. Shit happens.

And yes, it’s also true that reviewers are just like audiophiles — they tend to trade gear around just as freely as everyone else. So at the end of an agreement, I wouldn’t be surprised to see a reviewer sell unused, unneeded, or just plain excess gear on AudiogoN or wherever … for pretty much what they paid for it, or less, depending on the market for that item at the point of sale. Everyone can suffer buyer’s remorse — even reviewers. Things change. Other gear comes in and out. And what you have on hand may need to make way for what it is that you need for the current iteration of The System.

But that said, I have never seen the “insane sweetheart deal” (like 95% off, or for-free gear) that the rumor mills says that reviewers are supposed to get as a matter of course. Now, I’m not talking about T-shirts, promo bits, or demos of products that cost about $100 or less to produce. That happens, sure. But the $200,000 turntable for $5,000? Or a $50,000 preamp “just because I’m nice”? No. Not happening. Like many, I’ve heard piles of anecdotes greasy with sly innuendo, but I think it’s mostly jealous snipery that accounts for that, not any general practice. Dunno. Maybe I just don’t play at that level.

Audio Jesus

Anyway, all this goes to undermine the value placed on a reviewer, for sure. What I think readers would like, ideally, would be someone that’s “above all that”. Some friendly and generically attractive neighbor of theirs that will rise up miraculously, freely, and vociferously offering helpful, entertaining and informed opinions based on true, undeniable expertise in all relevant domains, and be entirely unconflicted in any of their biases. And their words will be universal. True. Wise. Funny. But most importantly, judgmental. And maybe a little mean. But still totally scientific, a devotee of the double-blind test, a Houdini-like figure that routinely exposes the expensive as equivalent to snake oil. And one that takes an ethical stance against payment of any kind because money is the root of evil, but is still willing to spend countless hours away from their family and friends and daytime job in order to sort out, once and for all, which $25 footer sounds best.

You know. Some kind of Audio Jesus.

Pardon me while I roll my eyes, but I get it. The only opinion worth listening to is the Average Joe. Right? This is that beer thing, again. Apparently, there is an unnatural and unfortunately deep trend in our society that says you have to be able to relate in a least-common-denominator sort of way before you’re able to give respect. And Our Joe really ought not to be all cluttered up with too much learnin’ or he’ll make us feel stupid and uncomfortable, which would be the End of All because really, at the end of the day, it’s all about whether or not we feel secure in our ignorance. Or something. Yeah. Squirrel!

There are two things tangled up in there that are worth separating. First is the sense that the only good review is a negative one. We’ve already touched on this, but I want to call out another particularly egregious habit audio nerds tend to indulge in, and that’s the roadside tragedy fetish. You know what I mean — the tendency many have to slow way down just to catch a glimpse of the wreckage at the side of the road, even when the highway in front of you is entirely clear. You know who you are, so pardon me while I smack you in the head. This kind of thing is just irritating, and I wish there was no crossover in reviewing. But there is! I call this the Crusader Approach, where the would-be reviewer “puts it all on the line” to call out a particular product or brand for some slight, real or imagined, and wrap themselves in the righteous flag of “public service”. This is the review that draws praise from the most vocal trolls, earns the most reposts, wins the most links and the most comments on social media. Grab the torches! Tear ’em down! Stomp their face! Kick ’em in their dirty parts! The rotten snake oil sons-of-bastards — I knew they was trying to cheat me!

Bah, humbug and bullshit.

I’ve already offered a couple of caveats to the whole negative review thing that makes “being negative” problematic for the reviewer, but even given that the relevant expertise is present, I firmly believe that there are other ways of presenting a judgment that doesn’t require sharp knives. Remember: a review is an aesthetic judgment, not truth. So getting all high and mighty is bit like saying there’s only one way to have sex with your spouse. Which is asinine. And totally fails to disguise the Crusader as anything other than a narcissistic jackanape. Anyway, what I want to say is this — a good reviewer can maintain their integrity and still provide value without being an self-righteous a-hole. You’ll have to trust me on this.

The last thing to mention here is that reviews, and the reviewers that write them, are a matter of taste. We’ll get to that in the Part 4.

Addendum

[Added April 23rd, 2014]

Given that this is the “Ethics Post”, I thought it’d be worth revisiting to mention a few developments in the last 10 days since this post went live.

First, I realized that Part-Time Audiophile did not, in point of fact, have an official or stated editorial policy. This has been addressed.

Second, in an effort to stave off the seemingly endless stream of doubters, naysayers, and general haters, I’ve attempted to go back through the catalog and call out particular cases where there might have been the appearance of impropriety, and then addressed those alleged infractions directly. I am satisfied that no policy was (even if retroactively) broken or bent. I can’t say “it won’t happen”, because … Life, The Universe and Everything. Anyway, if it does come about that there are issues of impropriety, or some conflict of interest, these issues will be called out and actions will be taken.

Finally, it should be noted that no one is killing baby seals here or talking about taking a trip where that’s even a possibility. Go. Grab an LP. Put it on the turntable. Sit down. Sip that bourbon. And know, with that deep, warm glow of satisfaction that is Truth, that it’s all going to be okay.

Get your Occasional now
About Scot Hull (979 Articles)

Founder, Editor and Publisher at Part-Time Audiophile and The Occasional Magazine.

10 Comments on On Experts, Reviews, and Drive By Shootings: Part 3

  1. Perry Wharfedale // April 25, 2014 at 4:24 AM //

    An example of some fake expertise taken from this web site http://www.sa-cd.net/faq

    “A possibly more important difference between CD’s 44.1 kHz 16-bit PCM sound quality and SA-CD’s 2.8 MHz 1-bit DSD sound quality is the accuracy in the time domain. As it turns out, the human ear is extremely sensitive to minute timing differences. In fact, of the various cues our brain uses to determine the direction of sound sources, probably the most important cue is the difference in time it takes for a sound to reach our left ear versus the right ear. With a sampling frequency of CD, 44,100 times per second it’s very difficult to reproduce a good ‘sound stage’ which is why you may find that the sound of a CD often ‘sticks to the speakers’: you’ll hear it coming from the left speaker and from the right speaker but there’s nothing in between – the proverbial ‘hole in the middle’. This is an area in which DSD excels.”

    This is factually wrong: the sample rate has no impact on accuracy or resolution of timing. And yet the writer sounds so plausible. He supplements his sagacity with real world experience, appealing to the reader to confirm it with his own unfortunate experience of hearing the “proverbial” CD hole in the middle – classic emperor’s new clothes stuff.

    At least this particular ‘expertise’ can be shown to be wrong mathematically. But what of the other, entirely subjective but plausible, audio expertise out there?

  2. Charlie Mathews // April 18, 2014 at 5:54 PM //

    I need to digest all this “stuff” before I can make an intelligent response from an ethics perspective. BUT I want to point out that Bound for Sound always (I know a stupid logical error) seemed to call it like it was and the staff reviewers were not afraid to “diss” a product (even if the product was from a heavy hitter manufacture). I just don’t see any of that kind of critical reporting/evaluation online anymore.

    Best wishes

    charlie

    • Part-Time Audiophile // April 18, 2014 at 10:50 PM //

      Yeah. That doesn’t happen much because … laws. That is, SCOTUS doesn’t afford First Amendment protections to online publications. Whoops.

      • I’ve heard you argue this before but haven’t really seem much evidence to support it.

        One reason for this may be that litigating is expensive and there would have to be clear evidence of false statements with the intent to slander & cause harm to the company for litigating to possibly make sense, particularly in the “audiophile world” against an online publication that likely consists of a few, possibly even just one, person, and who’s influence in the greater market is likely very limited. Small “boutique” high end audio companies are not deep pocketed as it is and pursuing litigation needs to make business sense.

  3. Perry Wharfedale // April 14, 2014 at 2:02 PM //

    Yes, enjoying the articles. Thanks.

    And I enjoyed the article linked to on The Federalist web site. But all the time while reading it I was thinking of the previously-respected area of expertise that, since 2007, has been revealed to be complete and utter rubbish: economics. This subject is, to my mind, the perfect example of why self-appointed experts are not to be trusted at face value.

    I have thought long and hard about whether I am an expert in anything, but I don’t think I can claim to be, despite having designed and built my own audio and music equipment since the 70s, and made a living as an electronics engineer for almost 30 years. I have lots of experience, and a few paper qualifications and publications, but I think I am too much of a generalist to ever sit down and write a meaningful text book, or be called upon to give evidence to a government committee. But what I do have, I think, is a healthy scepticism and awareness of a ‘hierarchy of expertise’. That is, an awareness of whether expertise is fundamental, or simply the result of specialised vocational training or on-the-job experience without any reference back to fundamentals.

    The ‘fundamental expert’ might start as a physics graduate and specialise in some area such as precision instrumentation (for example), or start off in psychology and end up developing haptic user interfaces or some such, but their specialised expertise would be against a backdrop of real science and the scientific method. The economist, on the other hand, I would immediately think of as the product of nothing more than glorified vocational training. Able to spout a few rules of thumb, and do some elementary statistics on data that doesn’t really qualify for statistics, they have no real idea of how the economy actually works. If a physicist spent a few hours studying the economy, I would accept his view on it over the career-economist who had spent years rehearsing the dodgy assertions in economics text books. Fundamental science and education trumps vocational training.

    Which brings me to audio ‘expertise’. Is audio expertise based on fundamental science (physics, psychology) or is it derived from vocational on-the-job experience? Of course it’s mainly the latter. As in economics, an ‘expert’ can spout rules of thumb that can never be proved incorrect, construct narratives to fit pre-conceived ideas and prejudices and abuse data in sighted ‘tests’ to his heart’s content.

  4. Scot, thank you for this entire series of articles, which are enlightening and a very interesting read. We’re looking forward to Part 4! Best regards, Mike and Lynn Bettinger

  5. Bromo Ivory // April 13, 2014 at 9:19 AM //

    Great (if a bit long) analysis – and I have to say we happen to agree (both me and my wife who I read this to). I do think your definition of “cheating” is so broad that it would apply to most things of human endeavour (for instance we have “discovered” the Law of Gravity so we don’t have ot test every object in the known universe vs every other object – might be considered “cheating” in your book. And then the entire history of the human race, even the use of expert, would be a giant cheat. And while entertaining to think about, and correct to a degree, has a broad enough definition that it might lose some utility, and the underlying assumption that “cheating is something you shouldn’t do.”)

    And while not a reviewer, I was always a bit skeptical about the conspiracy theories, but also know the marketing angle to it all. While a reviewer needs a “reference rig” and a prominent one may not be able to afford one of sufficient resolution, so gets gear “on loan” from manufactueres who get bragging rights, and the reviews might be more insightful. To me, this is closer to the University that gets “donated” gear that is used for precisely the same purpose (company gets bragging rights, and the university gets to accomplish their mission of education and research easier than before). So I can’t get too worked up about it. Because the gear is given so the reviewer does reviews – if a “civilian” audiophile were to try to cut the same deal – they would have to figure out how to become one of these reviewers, and then convince someone to loan them the gear, and then they would have to WORK to keep it all in place. And even so, I’d imagine that it isn’t as “easy” as people would theorize it might be anyway. Might be easier to just earn the money to buy what you can, so then you don’t HAVE TO do anything to keep the rig intact?

    Loving this whole series!

    • Part-Time Audiophile // April 13, 2014 at 9:28 AM //

      Thanks.

      That bit about the reference rig is spot on, by the way. Given that most reviewers aren’t wealthy, there are plenty of ways for a manufacturer to get free press, and donating is a fine way to go about it. I think this was far more prevalent in the past, but I wouldn’t be surprised to see that this goes on now. Manufacturers can adopt a reviewer, donate a pair of speakers or an amp or a turntable or something, and as a quid pro quo, that gear gets mentioned as “reference gear” in every review that reviewer writes. Corruption could arise where the reviewer feels grateful for the support and fails to note anything negative about that loaner gear — or anything negative about any gear from that manufacturer. I suppose this doesn’t have to be a straight up donation, either. It could be deeper discounts. Or a “long term” loan. I’ll talk a bit about this in Part 4, but more about familiarity and appreciation than about “how experts get undermined”.

      • Gavin Hadley // April 13, 2014 at 10:10 AM //

        “Long term loan”…a practice with which JV is intimately familiar or “accommodation pricing” a practice from which nearly all reviewers benefit. If you don’t think this crap impacts how they review said product I’ve got an arsenal of Machina Dynamica tweaks I’d sell you at msrp..

      • Part-Time Audiophile // April 13, 2014 at 10:34 AM //

        No idea about JV, or any other reviewer (well-known or not), so I can’t and won’t gossip.

        But, you’re right. It does impact a reviewer to have long term, positive, exposure to a brand or company. But it might not be quite what you think. I’ll talk a bit about that in Part 4.

        But (times two), I do have to note that without some “accommodation”, there might be no useful reviews at all. If you think reviewers (well known or not), are getting wealthy writing about audio … yeah. Ain’t happening. Taking myself as an example, if I were to leverage only my income from writing as the basis for acquiring a review system, and I was required to buy all of that at full MSRP, I’d probably have some really sweet bookshelf speakers from Pioneer and maybe a Marantz receiver. Yep. A perfect basis from which to review anything from, say, Onkyo. Nothing wrong with Onkyo, Marantz or Pioneer. But I think it’s safe to say that there’s a bit more to audio’s high-end.

        So, if you’re content with wealth being the only criteria for criticism, then we’re good to go. Banish the ethically questionable bits! Of course, this might also mean that manufacturers are no longer able to support formal reviews by qualified reviewers because indulging the entire practice makes it impossible to even break even. But that’s fine, too — the available content will be much easier to keep up with. There just won’t be much of it.

        The problem is that this isn’t easy to sort out. You’ve got symbiotic business models here, where both survive (if not thrive) because of the other. Kick out even a few of the supports (like accommodation pricing and equipment loans as arbitrary examples), and the relative stability of both is suspect. I’m oversimplifying here, but then, no more than the criticism was.

        But (times three), the point you’re making is only sharp in the context of outrage — “why are prices so high?” Unravel that, and the air leaves the balloon. Prices are high. Yes. Economics are a bitch. And business models only stay in use because they’re reliably helpful in extracting profit from markets. There are businesses looking to “go direct” and cut out the “middle men” (distributors and dealers), which will allow them to offer products at radically reduced prices. But that entails dramatic countervailing investments in marketing and outreach. Turns out that there’s no free lunch. Shocker.

        You don’t have to like it. High-prices suck. I wish I was rich, too. But being angry about it is silly. And shining a light, hopefully, leads us all toward understanding and better behavior.

1 Trackback / Pingback

  1. On Experts, Reviews, and Drive By Shootings: Part 4 | Confessions of a Part-Time Audiophile

Comments are closed.

%d bloggers like this: