On Experts, Reviews, and Drive By Shootings: Part 2.2

Part 2.2: Epistemology and some Problems with Knowing

Recently, Roger Skoff offered up a caricature of an argument around the so-called “expert” and skewered some of the challenges that appertain thereto. I then took it upon myself to elaborate. Um. Yeah.

Let me offer up again that apology I made in advance. Okay? Okay.

So, this whole “knower and knowing” section took a lot more words than I thought at first blush, so I chopped it in half. Just trying to help!

To review: in the last section, we poked a stick at reviewers. In this section, we’re going to turn that pointy bit around and talk some trash about the Knowing, calling on some challenges we all seem to face with getting there. Wherever “there” is.

On what you (don’t) know

Opinions are like assholes, everyone has them. Sharing them is as natural as breathing, and if you’re a Gen X’er (or younger), you’ve probably overheard some version of this platitudinously bullshit line:

Well, you’re entitled to your opinion and I’m entitled to mine.

This is usually tendered as a game-ender, a more or less polite way to terminate a conversation that seems to have run out of constructive or logical argument. It’s a way of saying, “I know you think you’re right, but I’m just not going to believe it or you so you may as well stop talking,” or the more familiar, politically sensitive, version “Let’s just agree to disagree.”

school-prayerAs you can imagine, this kind of move make me reach for my demon-killing knife. As if anyone has some kind of right to believe … well, whatever it is they choose to believe. As if beliefs are like socks — let’s have a drawer full, because you just never know what we’ll need!

But are beliefs justified by volition alone? Are you somehow entitled to a belief? Any belief at all?

Me? I believe in magic. Sorry about your cat! (I’m kidding).

I think the problem is that we were coddled as kids. My whole generation, probably. “You can be anything you want” and “It doesn’t matter if you win or lose, it’s how you play the game” and “everything happens for a reason” — all are poisonous little bromides that usually are only helpful because they’re delivered in a soothing, positive and hopeful tone of voice. All three are patently false, and like the bit about opinions, it’s namby-pamby feel-good nonsense that the weak-minded use to wrap themselves up in, like some kind of magic security blanket, a kind of fingers-in-the-ear along with the chant: “La la la! I’m not listening! La la la!”

And no, just for the record, there’s no such thing as “your truth” you can retreat to. Words have meanings, else language isn’t shareable. You want to go that route, you and your colorless green ideas can sleep furiously. Because Language. Yep, it’s what’s for dinner. And in language, particularly this one, terms like “truth” really do have bounds. Apologies in advance about your belief systems, but you’ve been led astray.

strong_opinionsAnyway, opinions are durable. Unfortunately, I think it’s fair to say that we are rather pointedly obdurate about them. This is something that we’re seeing quite a bit of in studies of political partisanship. We cling and cling and nothing can shake our faith. Because faith is the most important thing in the face of a strange and terrifying world. Faith not in God, no, but in the belief that we have not been wasting all our time being stupidly, pointlessly, and publicly, wrong.

Guess what? You’re wrong. Pretty regularly, too. Don’t panic! It’s okay — shit happens. But, that said, you’re not done — throwing your hand up and saying “oh well” is not on the menu of acceptable responses. Because Truth, that’s why. No, you do not have the right to be wrong. The fact that it’s your opinion does not in any way clothe that belief in some kind of special underwear. If and when you’re ever faced with evidence inconsistent with your beliefs, you are pretty much compelled to address that inconsistency. If it turns out that an adjustment is required, you must do so forthwith or else be justifiably mocked and preferably ostracized. This is doubly true if you’re in public office, but whatever.

There is nothing privileged about your beliefs. I don’t care if they’re about God, your Dearly Departed Mother, or about audio cables. If you have evidence that appears to directly contradict your beliefs and you do not accommodate that evidence in some meaningful way, then you’re a tool and cannot be taken seriously.

Now, this doesn’t mean that all evidence is created equal. This is, quite frankly, where experts tend to come in. In the equation of knowledge versus belief, there’s only one thing that really matters: justification. This is a philosophical road fairly well-traveled and speaks to the reasons for holding the belief in question: (1) what is your evidence, (2) where did it come from, (3) and how did you come by it. Logic, experience and expert testimony are usually the (only) accepted bases for justification. For what it’s worth, this is precisely where clever little heuristics like “parsimony” and intricate verification structures like “Science” come into play. And this is why experts carry so much more weight, too — they’ve seen more, done more, been around more, [insert x] more than the average Joe. So, their testimony counts more.

It’s like a weighting scale. If my buddy tells me the local Tex Mex place is “great”, I’m curious. But if that buddy happens to be a professional chef at a competing restaurant, I’m going. Remember that bit about Robert Parker in Part 1? Yeah, it’s like that.

Side note: Like many things, this bit about weighting is not really subject to debate and your opinion here isn’t really relevant. This is epistemology, and like Science, it holds whether you choose to believe in it or not. Again, what we’re engaged in (like it or lump it) is how we discriminate knowledge from beliefs. And in that calculus, experience just counts more heavily. In fact, in many instances, it’s the only currency worth accepting.

Here’s the short form about Science and Faith, and no, they’re not the same. The difference is fairly simple and comes down to this: “What would it take to prove [x] wrong?” If the answer to that question is “nothing”, you’ve now moved past argument into poetry. Have fun. And yes, that’s pretty much it.

Stop hating on your experts. They’re the only ones that can save you from yourself.


A bit on “how they know it”

So. Experts. We hate them even though that’s self-defeating. Complicated creatures are we.

But let’s talk a bit about knowledge. Informal-like. No need for algorithms or formula. Let’s just talk about inputs. Stuff goes in. In where? Let’s keep it simple and say “your head”. Stuff goes into your head and you … well … you do stuff to it. Process. Parse. Analyze. Match. Whatever. That stuff going in? Experience. Usually, we take sensory data as input, but we could stretch things and say that “thought experiments” ought to count too. You know. Logical exercises. These are different from things we directly experience, but sometimes a “eureka” moment doesn’t actually require an apple bouncing off your noggin’ before you see a bit deeper into the fabric of the Universe. And that’s interesting.

thought-experimentThat kind of “thought experiment” that leads to insight, understanding, or something, is something we can label. Typically (historically), that label is a priori, a nifty Latin phrase that means “from before” where “experience” is the thing they’re referring to. This gets contrasted with a posteriori, which “from after” — again, we’re referring to “experience”. What gets labeled a priori or a posteriori? Simple. It’s knowledge. Ha ha. That’s a joke. There’s nothing simple about knowledge. Happily, we can put that aside for now and focus on process and what that means for you and your expert.

You’ve probably waded through your share of online debates and saw them devolve to the point where someone barfs, swings an axe, or reaches for the flamethrower. The attempted take-down may include something like the following: ProfessorXY2007 is ranting about [insert victim here], going on about their latest crazy-talk claims, insisting that there’s “no way” that what [victim] says is even possibly true because [invoke some mathematical, electrical, chemical, or physical principle] and [victim] is therefore an ignoramus and almost definitely a shill. End of screed.

Deductive logic is very interesting stuff. If you start from true premises, adhere to simple rules, the result must be true. I mean, that’s how logic works. The trick, obviously, is getting your premises right. Especially if you’re going to invoke “natural law” to do it. Because, as it turns out, Natural Laws are always always ALWAYS false. Whoops. You also tend to sound like a pedantic douche, so that’s a bonus. But from the perspective of logic, you’re going to have to do more work — there’s no automatic freebie on offer, there.

Getting back to knowledge, what we have here is also an a priori argument. Nothing wrong with a priori arguments, per se, but if a counter-argument is relying on logic alone, we should be concerned. Why? Well, logic (much like Natural Law) has a nasty tendency of oversimplifying and thereby missing shit.

It’s true, not everything can be subject to a direct, personal experience. Logic has a role. But, one could easily make the argument that the whole 20th century was a rejection of the primacy of pure reason. Experience matters. And if the relevant experience is trivial to acquire, a priori arguments are kinda pointless. At the risk of piling on, if someone actually has done the experiments, undergone the training, lived with the costs and benefits, their testimony almost always carries more weight than even the best a priori arguments. Sorry about that.

I recently saw this in a forum argument: “Well, my system doesn’t show it, therefore it must not be real.” I like the appeal to experience (good show), but this is a good case of where logic takes the wheels off the bus. What the claimant appeared to be saying was that his experience with his one system was somehow universal, and that his conclusions based on his experiences with his system were valid, translatable, and relevant to all systems, everywhere. Yeah. Well, about that. Proving the negative is, erhm, problematic. But making universal generalizations from specifics is equally so.

Here’s another thing to keep in mind: a priori arguments are really only limited by imagination — and the lack of imagination isn’t an indicator of much of anything. Maybe you’re just boring. Likewise, a lack of experience doesn’t entail or imply anything about existence. Maybe you haven’t been lucky, or tried hard enough, or don’t have the right tools.

This was a problem for the Higgs Boson. No one had ever seen one, so it was assumed to be a useful but fictional construct by many physicists. And then the tools got better. Whoops. Guess what? Anyway, arguing from your own failures is never a convincing argument.

The cherry on the pie is the double-blind. If all else fails, says the naysayer, then prove it. “In fact, this is my falsification principle,” they chant. “This is the thing you have to do in order to provide me with the motivation to change my opinion.”

First, about the burden of proof. As a matter of course, rejecting an explanatory theory is a fine thing to do, but you need two things to do it. First, you need a counter example. Second, and most important, you need a replacement theory. You can’t kick the legs out without offering a prop — that’s bad form and, more seriously, explicitly disallowed under that other part of the Scientific Method. You know. The one used by scientists. Ahem.

Anyway, when you’ve got a claim that you’re evaluating, that claim usually has stated or unstated supports that hold it up. If you want to dispute that claim, that’s totally fine — but it’s on you to “do the work” to disprove it. Just saying “no, you’re wrong” is a party foul, and the appropriate response is “you are drunk”. The “burden of proof” is always carried by the doubter, and you exercise that by responding with “no, you are wrong — and here is why.

Next, lets talk method for a second. Claims of existence are not typically subject to double-blind experiments. Sorry about that. Usually, the double-blind is a tool used for efficacy, a rather different category of problem. Claims of existence are supported or obliterated by evidence and explanatory power. That’s why no one had to double-blind the Higgs.

double-blindYou see, a double-blind is a methodology, one of many, and not a universal can opener. This is another one of those delightful cases of “a little knowledge is a dangerous thing”. For the record, Science as a whole proceeds very handily without mandating them or waiting on them. Just sayin’. And then there’s the matter of analysis — the result of a double-blind experiment is not a negative indicator for existence.

It may be an indicator for average effectiveness, however. Take sensory detection trials (heard|not heard, for example). If your sample pool is of sufficient size and sufficiently random complexity in makeup, you might be able to conclude that a particular effect is either profound or negligible or somewhere in between. There’s a kicker, however, in the form of the unstated appended clause: “… for the average person.” Which is entirely irrelevant for existence claims. The question wasn’t “Will I hear it?” That’s a bait-and-switch. The question, at least as it bears on most audiophile debates around sonic attributes, is better stated as: “Can it be heard?” And for that, we’ll need to do things a bit differently.

I’ve talked about this before, but this is worth pulling apart again, so here goes: In any statistical sample, you have outliers. Folks that beat the average. In a pool of 10 subjects, there may well be one that “gets it right” every time. Similarly, there may be one that “gets it wrong” every time. This is interesting. Very interesting. But in a double-blind experiment, this is also a problem.

Handily, statistics gives us a clever way to normalize our data to keep “noise” like this from cluttering the results and fucking up our graphs: we discard them! The assumption is that there was something flawed in the execution of the methodology. In short, they cheated. So — toss ’em! That’s what you’re supposed to do. What we’re looking for is the distribution patterns …. And if you’re looking for efficacy, this makes sense. You want to know what’ll happen to Average Joe. But … what if the outliers didn’t cheat?

If you’re looking for existence, discarding the outliers means you’ve just tossed your detectors. It’s like saying: “We’re looking for an extremely rare and hard-to-see subatomic particle. Here are your binoculars! Now, go look at random collisions out there in the Universe. Let me suggest those kids banging tennis balls against that brick wall! Excellent — tell me what you find.” More seriously, the folks hunting the Higgs didn’t use “just any” gear in their search. They went after, and built, a crazy-ass special-purpose, one-of-a-kind detector. This is what you do when you want to justify existence claims — you don’t shoot for random tools, you look to the weird stuff.

So, in sensory trials, you need to round them up. All that you can uncover. There’s absolutely no reason to believe that they’re common, so there may be some work to find a large enough sample pool. But it is precisely these folks that you want to ABX to death. Who are they? Experts. Well. Maybe! But I’d start there. Want to get fancy? Have those “reliable detectors” attempt to train others to do what they seem able to do. That would be interesting.

The point is that this entire line of argument, wherever it crops up, is usually bogus. A random appeal to the Scientific Method is not going to “save you” from having your opinion swamped by an expert. Constructing an a priori argument is fine and all, but when actual experience is directly and readily available, it’s a little obtuse. The best that an appeal to methodology will do is count toward undermining the credibility of the expert, but even then, it won’t necessarily overturn it. Not by itself. Why? Because a thought experiment does not, in itself, yield truth. It’s a pointer. A suggestive anecdote. By submitting it, all you’ve done is moved the problem … all the while shifting the burden of proof squarely to your own shoulders.

Have fun storming the castle.

Respect and beer

When it comes down to brass tacks, I really don’t know or understand why it is that experts are so routinely put down, sidelined, or otherwise marginalized. Yes, there could be a million reasons, but it seems more pervasive and more reflexive than anything I’ve outlined so far. This concerns me, because these are the exact people that we ought to be relying on to help us not mis-spend our money.

Fine, if you’re Mitt Romney, you can just go buy whatever AudioNerd1949 recommends as the flavor of the week and try it out yourself. In fact, this isn’t a bad plan — if you’re Mitt Romney. But given that the bottoms of bank accounts for the rest of us tend not to enjoy such depths, you’d think that a little guidance from the very people most likely to help us hold on to our clams would be a welcome and sought-after thing. But no. It’s like we can’t help it.

Smart people are scary, complex topics are abhorrent, special knowledge is suspect, and there is nothing under the sun that my neighbor, that a good ole boy with a lick o’ sense and his head screwed on straight, can’t figger out. Which pretty much explains why most policy decisions made in the US today are stupid: stupid does as stupid is.

I want to say that the problem here has to do with something simple, but I can’t. If you buy me a beer at RMAF some year, I’ll be happy to share my Dumbing Down Of America theories. But there’s a well understood meme in politics that came out in a recent election that sheds some rather sickly light on the subject, and that’s beer. “Which candidate would you rather have a beer with?” is a bizarre way to select anything other than plans for Friday night. When you’re picking a Brigade Commander, a CEO, a plumber, or a firefighter, approachability isn’t really high on the list of job requirements. A pal? Sure. But a President? Baffling. Pardon me while I shake my head in utter disbelief. In what possible universe is that a relevant criteria? The fact that it is says something fundamental about our psyche, I’m sure, but it’s above my pay grade to sort out.

In the end, it’s your money. If your vision of “being an audiophile” means engaging in spirited debate over rabbit holes and minutiae, then welcome to the club. If you are looking for a way to spend the least amount of money in the most effective way possible, then there’s another way. There’s a difference between “stupid” and “ignorant”. You can’t fix stupid, they say. But the other one? Well, there’s a path for them. Part of that path is listening to those that have already been down that road before you, and if you find yourself in unfamiliar territory, from learning from those that have been down similar paths.

As Skoff suggests, you’re really best served by trying it out yourself. Might as well get to it. And that, I think, is where Skoff — a real, honest-to-God expert in his own right — wanted you to get to all along.