latest Stereophile.....

TroyD

There is an article in this months Stereophile where Jon Iverson bemoans double blind / ABX testing....the crux of it is he says that it doesn't accurately capture the differences between gear, only the person's ability to hear the differences so therefore it's not a valid test of gear.

Well, FREAKIN' DUH! I still can't believe that little ephiphany passed the editors. The way that I, and I'm sure that many who read Stereophile, interpret it is if a reviewer can't distinguish between two pieces of gear, doesn't that cast a shadow of doubt over the credibility of the reviews these guys are peddling??

What world do these guys live in??

Oh, btw, there is a pretty glowing review over the B&W 603S3, seems like a neat little floorstander for 1K.

BDT

Early B.

Does this fella offer an alternative?

How can you determine the differences in gear if you don't listen to them?

My own A/B tests and listening abilities validate my reasons for purchasing certain gear. No other validation is required, especially from reviewers who are paid to always say nice things about audio gear.

ND13

Originally posted by Early B.

My own A/B tests and listening abilities validate my reasons for purchasing certain gear. No other validation is required, especially from reviewers who are paid to always say nice things about audio gear.

Exactamundo, I'll listen to other's reviews and take them for what there worth, but in the end, it all comes down to what sounds good/right to me.

TroyD

My point is that if a reviewer can't reliably tell, under controlled conditions, the difference between two different items, it makes me question the validity of thier claims in the reviews that they write. What irks me is the attitude that how dare we, the great unwashed, expect any sort of credibility check on these hacks?

These guys, for the most part, have never met a piece of gear that isn't the latest and greatest, until the next review. However, in cases, like the savagery of Carver's gear......these guys wield a lot of power over much of the audio buying public and to expect no iota of accountability is rediculous.

BDT

polkatese

Originally posted by Early B.
especially from reviewers who are paid to always say nice things about audio gear.

that is also a point that several of letters to editor clearly annoyed by the reviews (in August's edition). Yep, it's a sell out to the advertisers, the end statement of any reviews had always ended with: we recommend.....

PolkThug

Originally posted by TroyD

What world do these guys live in??

The live in the Audioquest Theory World, where double-blind tests are embarassing, therefore invalid.

ND13

Originally posted by TroyD
My point is that if a reviewer can't reliably tell, under controlled conditions, the difference between two different items, it makes me question the validity of thier claims in the reviews that they write. What irks me is the attitude that how dare we, the great unwashed, expect any sort of credibility check on these hacks?

These guys, for the most part, have never met a piece of gear that isn't the latest and greatest, until the next review. However, in cases, like the savagery of Carver's gear......these guys wield a lot of power over much of the audio buying public and to expect no iota of accountability is rediculous.

BDT

I've read more eloquently written, real world, and unbiased reviews here at CP than most of the ones in the rags. This is the only forum(s) I've ever been involved with, much less read. We have some highly articulate and qualified reviewers of our own so I don't have to go elsewhere, unless it's about something noone here knows about(which is very rare).

Early B.

Professional reviews suck! Here's why:

-- they are completely biased

-- they sometimes use a controlled environment that has no resemblance whatsoever to real world appilcations

-- they use meaningless words that none of them have come together to agree on their meaning

-- they have not developed standards for their listening tests

-- it is only one person's opinion

-- they never use any gear that you own

-- reviews are often too technical; the only thing that matters is how it sounds

-- they sometimes compare one speaker to another as though that means anything to anyone (unless, of course, you were in the listening room, too)

-- every single item reviewed is recommended and is always one of the best at its price point

-- they rarely take stuff into consideration that means a lot to the consumer like WAF, street price, used price, lower priced alternatives, etc.

-- they have never met a component they didn't like

For these reasons, I stick primarily with consumer reviews. Yeah, there's lots of issues here, too, but at least you're listening to a real person with real world issues who are not being paid to be kind to everything they review.

Early B.

I've read more eloquently written, real world, and unbiased reviews her at CP than most of the ones in the rags. This is the only forum(s) I'm ever been involved with, much less read. We have some highly articulate and qualified reviewers of our own so I don't have to go elsewhere, unless it's about something noone here knows about(which is very rare).

Precisely. That's why I stick around.

Shizelbs

Thats a real nice summary there Early. Couldn't agree with you more. Matter of fact, I am struggling to think of anything you missed.

Sound and Vision is a great example of piss poor reviews. They love everything. Everything! And a great deal of it is stuff we all know to be junk, well, at least not as good as they claim it to be.

Actually, their video game reviews are half-way decent. I doubt they are getting paid for those however.

reeltrouble1

Troy,

I read that last night, the writer stated he only got things "right" some of the times, I took it to mean he "picked" the more expensive or supposed better piece of gear only some of time, called ones who never picked it "cloth ears" and ones who always do golden ears or some other high felutin term.

I dunno seemed pretentious for him to castigate an entire group of people based on the fact they reported what they heard to JA or whomever.

We all hear differently and I know I am well past my prime hearing time, I still enjoy nuances between gear. I buy the fact you need to train yourself for how to listen but the dont like to listen to "gear" much rather listen to the music if that makes any sense.

BTW what the WTF can Wilson do to a pair of speaks to make them worth 45 grand, man now there are some golden effin ears and a gold lined pocket book.

RT1

madmax

No, no, no...

I think what he is saying is that there are definitely differences in equipment. All the ABX testing does is to check the hearing of the individual doing the test, not whether there is a difference or not. ?? And to take it a step further, you cannot make a decision on a given piece of music. You must live with the setup and carefully engage with the music to "get it". You are never going to be able to judge given a snippit of one and a snippit of the other. For example, we all know the louder one will win. It goes further though. The fuller one will win too. The one with the highest treble will win. The one with the stronger bass will win. There are a lot of variables and depending on the given piece of music one will win over the other and this will change depending upon the music.

I personally believe in their ratings.

madmax

MacLeod

My humble opinion is I think people mistakenly think an ABX test is trying to prove there are NO differences between 2 certain products. Thats not the case really.

Take a $100 Jensen car amp and a $600 Xtant amp. The differences will be in build quality most of all. The Xtant will be built like a tank using materials thatll last forever and not fail where the Jensen will be using lids from an old tin can.

Also, the features on the Xtant, bass boost, crossovers, EQs and such will also be of a much superior grade.

All the ABX test proves is if you take both amps, and set them at the same power level, playing thru the same speakers, one wont be "warmer" or have "tighter bass" than the other.

So you still need to buy equipment of the highest quality possible but make sure you do it for the right reasons.

steveinaz

I think the point the guy is trying to make is that personal preferences play a large part in blind testing, rather than empiracal evidence. If tester "a" likes sweet highs, but tester "b" prefers a more laid back sound, then you've got a conflict.

You have to know how to read audio reviews. I look for stand-out information, or things that are outside what is considered the norm. I pay no attention to a reviewer's subjective opinion.

Spawndn72

AB/X testing has nothing to do with wether A sounds better than B or vice-versa. In an AB/X the person listens to A then listens to B then listens to X. Then they try and decided wether X is really A or B. All the test does is see if the person can tell the difference between A and B.

HBombToo

I'll have to do a refresh reading of that article tonight to make sure my impression was correct the first time and only time through.
BUT,...
my impression of the article is that the lack of familiarity of a certain piece of gear can slant the test. ie, To be thrown in a room with a bunch of other people and have to listen to this, then what about that, then, what do you think, can really add a layer of error in the mix. I believe that was the intent of the article.

I will re-read that article.

HBomb

Emlyn

I only read it for the pictures.

HBombToo

Originally posted by Emlyn
I only read it for the pictures.

whats that... the great pair add pic comes to mind;)

markmarc

My biggest gripe with reviews is the same as many of you, the reviewer uses equipment and a room that only 1% of customers can have themselves. We all have various pieces of furniture, odd shaped rooms, combinations of floor coverings, etc. Then, no matter what the price of speakers they review, the author hooks them up to 10K amps.

I will say that I do appreciate that some of the Stereophile reviewers use the same music samples (Robert Riena for example). However, no a/b comparisons, shootouts between several models are offered.

Maybe it's done on purpose, so that we all go visit a quality dealer and audition for ourselves. If that is the case, it is a noble, but flawed one. Why, because their are so few independent dealers left.