Are those blind ABX tests valid??

CoolJazz

Every time the cable debate comes up, it doesn't take more than a few responses to get the "Well, blind tests don't show any difference".

Do you know when has an ABX ever shown a difference is audible??

I ran across this proud example on the web, of a face off involving some decently high quality equipment versus simplistic, all but junk stuff. Yet the outcome was no difference between the systems.

http://matrixhifi.com/ENG_ppec.htm

There is also the case out there, that I have no link to, that showed something like a Blose system sounding just the same as a real system in a blind test. And there are many more stories like that out there. The evidence would seem to be mounting that differences don't seem apparent in the test situation that we'd never expect to be true in normal use.

So I ask...can we really expect the blind test to prove differences do exist?

Here is the conclusion of the testers in this article I posted to, for those that can't take the time to read a fairly short article.

How can it be possible that a basic system with such a price difference against the "reference" one, poorly placed, using the cheapest signal cables found, couldn't be distinguished from the more expensive one?

And, most of it all, how come the cheap system was chosen by so many people as the best sounding of the two?

Shouldn't the differences be so evident that it'd be a child's game to pick the best?

Well, we think that each can reach to its own conclussion...

I ask this question in all seriousness and would prefer it to not go nuclear. I'm asking in all seriousness...does the blind ABX ever prove to be a valid way to judge audio equipment?

What a great question they ask..."Shouldn't the differences be so evident..."

CoolJazz

tonyb

With all due respect, Why on earth are you starting this B.S. again?

CoolJazz

Because I've never seen it asked why the one and only tool that seems to satify, also can't show any difference anywhere. Or does it? Why insist if it can't be shown effective?

I think it's a great question.

hearingimpared

Here we go . . .

treitz3

I'll get the popcorn. Anyone for a beer?

Should have made this a poll, man! Anyhoo, I'm just razzin' ya'. I do not concur with the findings personally. I just trust my ears, they never lie. Blindfolded or not.

tonyb

CoolJazz wrote: »

Because I've never seen it asked why the one and only tool that seems to satify, also can't show any difference anywhere. Or does it? Why insist if it can't be shown effective?

I think it's a great question.

If it's such a great question, why are you a member of an audio board to begin with ? Please, do tell.

hearingimpared

Thanks but keep the beer Tom, but I will take some popcorn. I agree with you about the findings and trust my ears also as long as they aren't filled with wax.:D;)

CoolJazz

Again Tony, because I've never seen it asked. No need to be guard dog and go after a chunk of my **** is there? I enjoy audio all day everyday. Talk of audio issues is much more interesting to me than the weather.

There is a thread on another forum that's over 10,000 responses strong and 6.5 years deep. It goes on and on about testing procedure like that it biases the test for the person running it to even no which is which. It makes me wonder why insist that it's the only way to look at something. Do scientists use the blind ABX for everything? Is that where this comes from?

I really would like to see some thought out responses. I can filter through the ones that don't put anything into it. Does the ABX ever show differences? Doesn't seem like a question to make the world stop.

CoolJazz

Face

What do you expect when the speakers use $12.00 tweeters: http://www.madisound.com/catalog/product_info.php?cPath=45_229_324&products_id=8264

hearingimpared

Here's my take CoolJazz. In order to properly evaluate a piece of gear including the IC and speaker cable components, you have to live with it for awhile let's say a week, take notes and then compare the "other" piece of gear taking notes for a week. Then go back to the original and compare notes. This all takes time.

ABX tests don't give you that opportunity thus in my book are invalid, at least in the subjective audio world.

CoolJazz

That's a good catch Face. I don't know anything about those particular speakers they used. I don't think they showed those at any of the various high end shows I've been to. Or any friends with the brand either.

But as I've linked to here before, I've set through the Audioquest boombox demo many years ago, that it wasn't hard at all for anybody I saw to be able to hear differences in speaker cables. So I don't know that a $12 each driver makes it impossible to hear the differences. Note that they were comparing a whole system up to the speaker and the spkr cables.

Hearingimpared, I kind of agree with you about the long term exposure thing. I do think there is some validity to the first impression differences, but it's dangerous to use only those. I think your system would tend to go to the "jump out and grab you" sound quality if you only use short, quick impressions. I like to do the short comparisons, then settle in and really use something for a couple of weeks and get used to it's sound signature. See what the impressions are...of what seems to work good and maybe not so good. When you've gotten used to it, then go back and see if you feel differently about a change back.

In some of the ABX testing push though, they are so confident that no differences can be heard that they are willing to offer "use as long as necessary" and even "use as good and as bad cables as you want". Doesn't this mean that difference simply go unnoticed when you go into the formal testing procedure? Or is it something else?

CoolJazz

megasat16

CoolJazz wrote: »

I don't know that a $12 each driver makes it impossible to hear the differences. Note that they were comparing a whole system up to the speaker and the spkr cables.
CoolJazz

Whether one can hear a difference is based on many things but these two come out at the last of the chain so they both shouldn't be bottleneck for the sound reproduced from the earlier electronics in the chain.

1. Ears - without a good pair, you don't hear any difference from mp3 to live music.

2. Speakers - without the ones that can truly flatly reproduce from 20Hz-20KHz both on-axis and off axis (coz it dictates the size and width of the sweet spot and acoustics reflection from the room).

In the ABX testing review, these speakers are a joke and I wouldn't use them for PA system.

megasat16

^^^ And let's don't forget the medium! The thin air or the thick air? It makes a difference how a speaker sounds and how an ear hears!

I'll be selling Audiophile Grade Air Treatment for optimal air density in the ready to spray canisters. Anyone interested? I guarantee it smells like audiophile ****. J/K!

sucks2beme

Lets say I do all this and hear a difference, and then post it.
Do you think any of the cable "flat-earthers" would be swayed?
it proves that somebody on the internet can pull your strings.
That's about they only conclusion to be had with the double blind.

WilliamM2

Do you know when has an ABX ever shown a difference is audible??

Yes, there have been many positive DBT tests, even for things like .1db level differeces and upgraded capacitors. Just seems to be cables that elude a positive DBT.

http://www.provide.net/~djcarlst/abx_phca.htm
http://www.provide.net/~djcarlst/abx_pwr.htm
http://www.provide.net/~djcarlst/abx_cd.htm
http://www.provide.net/~djcarlst/abx_lvl.htm
http://www.provide.net/~djcarlst/abx_caps.htm
http://www.provide.net/~djcarlst/abx_spk.htm
http://www.provide.net/~djcarlst/abx_tapg.htm
http://www.provide.net/~djcarlst/abx_tape.htm
http://www.provide.net/~djcarlst/abx_f1.htm

ben62670

Some recordings sound better on a boom box than a decent stereo. ABX tests are flawed. Play some of your favorite music with the burned in gear of your choice. If as a whole you don't hear a noticeable difference save your money. I feel the placebo effect can be real depending on your expectations. You need more than a weak test like an ABX test. It takes time. Again buy used,or barrow gear when possible.
Ben

CoolJazz

Thanks for the links William. It's interesting to see that at least a few of the tests show differences were found. Make sure you notice the count of people involved in each test. I was surprised to see that it was a smaller count that often get together for audio listening functions.

Here is more info on the 'ABX' company....

The ABX Company designed, manufactured and sold the ABX Double Blind Comparator System, which made possible scientifically valid self run subjective comparisons of audio components. The information on these pages is historical. This web site documents the ABX Comparator System and lists the subjective test results obtained by the company and the Southeastern Michigan Woofer and Tweeter Marching Society, SMWTMS. I was one of six partners who formed and ran the ABX Company from 1980 to 1987.
May 7, 1977 SMWTMS did the first ever audio double blind subjective listening tests. An argument over the audibility of differences between amplifiers at a club meeting in November 1976 resulted in an agreement that a double blind test could settle the question. Just six months later, Arny Krueger gave a lecture on his design of a double blind comparator and the first three double blind tests were done. The results include the first three listed in the Power Amplifier Comparison Table in the data. Thus we credit Arny Krueger and his opponent in the argument, Bern Muller, as the inventors of the ABX Comparator. The agreement to create a company to manufacture comparators was informally made the following summer.

I would love to follow up many years later with people that relied on the results to see if they remained pleased or not. Did they continue to use the information from ABX tests or turn to Stereo Review or what? I ask because I've honestly never been around someone that didn't just trust their own ears for what they wanted.

CoolJazz

CoolJazz

ben62670 wrote: »

Again buy used,or barrow gear when possible.
Ben

Thanks Ben! I'm a huge believer in what we get to sample through friends and get togethers.

Best of all is hearing it in your own room. I find it pretty tough to really get the "feel" for something the same somewhere else...other than the place that I'm so used to!

Hope you're doing ok!!!

CJ

zingo

I have speakers I like and cables I like; that's good enough for me. Is your equipment good enough for you?

treitz3

Yes.

CoolJazz

treitz3 wrote: »

Yes.

So your "done" Tom?

Sparky is gone. New perfection has arrived. Nothing but music to buy now?

WilliamM2

CoolJazz wrote: »

I would love to follow up many years later with people that relied on the results to see if they remained pleased or not. Did they continue to use the information from ABX tests or turn to Stereo Review or what? I ask because I've honestly never been around someone that didn't just trust their own ears for what they wanted.

Arny Krueger is a regular poster over at AVS, I suppose you could ask him.

treitz3

No. This hobby never ends. Sparky just sings better and better as the years go by....;)

WilliamM2

I feel the placebo effect can be real depending on your expectations.

Are you serious?

hearingimpared

WilliamM2 wrote: »

Are you serious?

:p:p:p:p:p:p:p:p:p:p:p:p:p

Those aren't just tongues sticking out they are the Bronx Cheer!:p:D

ben62670

WilliamM2 wrote: »

Are you serious?

There are people who can and can not hear a difference. We all have different taste. Some lemmings will buy expensive cables to "fit in". If you don't hear a difference spend your money elsewhere. I here a difference, and I will continue to use the gear that sounds best to me. You do the same, and others do the same. I will still recommend people try different gear(cables too). Simple enough?

F1nut

Another opinion.....

Now back to the question of the blinded testing. Here is what the now publisher (Robert Harley) of one of the major magazines wrote a few years ago....


Quote:
Blind tests nearly universally appear to indicate that no differences exist between electronics, cables, capacitors, etc. In fact, one infamous test "revealed" that no sonic differences exist between power amplifiers. Mark Levinson, NYAL Futterman OTL tube monoblock, NAD, Hafler, and Counterpoint power amplifiers were all judged to be sonically identical to each other and to a $219 Japanese receiver (footnote 7). This very test, wielded by the objectivists as proof that all amplifiers sound alike, in fact calls into question the entire blind methodology because of the conclusion's absurdity. Who really believes that a pair of Futterman OTL tube amplifiers, a Mark Levinson, and a Japanese receiver are sonically identical? Rather than bolster the objectivist's case, the "all amplifiers sound the same" conclusion of this blind test in fact discredits the very methodology on which hangs the objectivist's entire belief structure.

If differences do exist between components, why don't blind tests conclusively establish the audibility of these differences? I believe that blind listening tests, rather than moving us toward the truth, actually lead us away from reality.

First, the preponderance of blind tests have been conducted by "objectivists" who arrange the tests in such a way that audible differences are more difficult to detect. Rapid switching between components, for example, will always make differences harder to hear. A component's subtleties are not revealed in a few seconds or minutes, but slowly over the course of days or weeks. When reviewing a product, I find that I don't really get to know it until after several weeks of daily listening. Toward the end of the review process, I am still learning aspects of the product's character. Furthermore, the stress of the situation—usually an unfamiliar environment (both music and playback system), adversarial relationship between tester and listener, and the prospect of being ridiculed—imposes an artificiality on the process that reduces one's sensitivity to musical nuances.

Going beyond the nuts and bolts of blind listening tests, I believe they are fundamentally flawed in that they seek to turn an emotional experience—listening to music—into an intellectual exercise. It is well documented that musical perception takes place in the right half of the brain and analytical reasoning in the left half. This process can be observed through PET (Positron-Emission Tomography) scans in which subjects listening to music exhibit increased right-brain metabolism. Those with musical training show activity in both halves of the brain, fluctuating constantly as the music is simultaneously experienced and analyzed. Forcing the brain into an unnatural condition (one that doesn't occur during normal music listening) during blind testing violates a sacrosanct law of science: change only one variable at a time. By introducing another variable—the way the brain processes music—blind listening tests are rendered worthless.

sk1939

There is a reason this topic never ends. There is a flaw in his argument about the blind test with the Japanese receiver vs. the Mark Levinson et al. and that is that it could be that perhaps to the testers they did all sound the same. It could be that the test pulled all average joes off the street who listen to nothing but the HTIB or ghetto blasters, so anything would sound good to them. It could be that they played an unfamiliar body of songs. It could be that after listening to everything they got fatigued. Or maybe they listened to them all in such rapid succession, that they didn't hear each sample well, or that they thought it sounded the same. There are so many if's it's not even funny.

The problem with these tests, forgetting the fact that it is indeed a double blind, is that you listen to one, then to the other. The problem with that is, that the mind and the ear "forget" the differences, and this is even more so after a while listening to the same items. Not to mention as previously mentioned, the unfamiliarity of the recording and the way the components sound.

I think the best way, is to actually run the test with the competing items going at the same time (requires more equipment/money). Thats the only way to actively hear the difference (it's the only way I shop for speakers), and to eliminate the "forgetfulness" of the ear and mind, because what you "thought" you heard may not be what you actually heard. You also have to listen to music your familiar with (right down the the most minute things, and it has to be a recording/version you've heard alot, can't be like you own the Beatles White Album record, and then use the White Album CD as a test base). The person also has to be relaxed, if your blood pressure is increased, the less likely you are to listen to the music.

The problem I have with the above article is where he talks about brain function, and states that musicians use both halves of their brain, but saying that doing so forces the brain into an unnatural condition for anyone else is wrong. There are people who can do both successfully, and Mr. Harley draws too much flawed logic from his closing argument.

Plus, I have to argue here, a major magazine of what? Could it be an audio magazine? Doesn't that mean that there is a writer bias, and that it is a conflict of interest?

sk1939

ben62670 wrote: »

There are people who can and can not hear a difference. We all have different taste. Some lemmings will buy expensive cables to "fit in". If you don't hear a difference spend your money elsewhere. I here a difference, and I will continue to use the gear that sounds best to me. You do the same, and others do the same. I will still recommend people try different gear(cables too). Simple enough?

I agree, and although I believe that the expensive cable business is nonsense (copper is copper), it doesn't mean that I won't try something else, or that I force others to agree.

To me, cables have the placebo effect; you spend $2000 on cables, so you expect to hear a difference, so your mind kicks into gear and voila, you hear a difference.

People spend $1000's on plugs, wires, outlets, etc. only to realize that the copper in your walls is $.10 a foot romex, and the same as everything else in your house. So is that circuit breaker. There are no such things as "audiophile" circuit breakers. But this is all imo of course.

To me, the most bang for your buck upgrade is if you have 1/2 decent speakers (with enough power to power said speakers) a power conditioner that does EMI/RFI and voltage regulation. It will eliminate all kinds of power issues, more so than any power cord.

ben62670

Copper is copper? So water is water? Drink the water here, and drink the water from a mountain stream. There is more to it than that. Dielectric means a lot. Also when testing wire they test for RCL. Seeing audio is more than just a single test frequency your theory does not hold water. Also maple is maple right? Mahogany is mahogany right? If this were true than all wood instruments would sound the same if made to the same dimension.
Sorry sk. I used to believe like you then I used my ears when a Polkie was kind enough to bring over a few sets of wires;)
Take care.
Ben

ben62670

BTW the world is flat, there are only roughly 2,000 stars, and this super high mass thing the size of the tip of a ball point pen blew up billions of years ago and developed into billions of people with different personalities, the grand canyon, millions of species...

Edit: my brain processes taste, smell and hearing the exact same as everyone else's;)