Stupid things you hear from the DBT/Null test crowd....

headrott

tonyb wrote: »

DK has gone above and beyond in referencing his position. He has an infinite amount of patience, more so than me anyway. Yet his responses are exactly why most don't participate anymore in these types of discussions. Learning, in itself is a lost art for some.

That muti-dimensional thing.....in my day, we called it LSD. :cheesygrin:

"Turn off your mind, relax and float downstream...... It is not dying, it is not dying." There is something to that.

Greg

tonyb

What are you boys doing up so late ? Man, here I thought I was going to be the only one up at this hour.

This whole thread deserves a award...or 2 . One for the best explanation and reference of one's position, and another for the best lack of comprehension in a thread.

Systems

I'm thinking we can all disagree in a heated discussion, but in the end, we are all friends.

YOUR HONOR, I would like to call "all the other times we have done this" to the witness stand please!

Syndil

tonyb wrote: »

DK has gone above and beyond in referencing his position. He has an infinite amount of patience, more so than me anyway. Yet his responses are exactly why most don't participate anymore in these types of discussions. Learning, in itself is a lost art for some.

My points have never been addressed. What I am seeing in this thread merely validates my position--people naysaying blind testing because they don't want to do blind testing, in the same way they sometimes hear differences in unblinded testing because they want to hear differences. It's a preference towards bias.

As articulate as DK may sound, and as much as you may all rally behind him, his argument is lacking. None of my key points in post #145 have even been attempted to be addressed. DK made fun of my disagreement with the usage of the terms "multi-dimensional" and "metrics" and posted some mocking comments, but no argument was made. Disagreements in semantics aside, there is a whoooole lot more meat to that post that just went totally ignored. Perhaps he is unable to satisfactorily address my points and knows it.

heiney9

Syndil, you are a real trip.

H9

WilliamM2

DarqueKnight wrote: »

Do you understand the difference between

1. Listening to discern a gross difference between two music samples (TWO SIGNALS), particularly when you know the nature of the potential difference (i.e. contamination with noise) in the signals

and

2. Listening to the entire stereophonic presentation presented by TWO PIECES OF EQUIPMENT and discerning the difference(s)?

1. If injecting .003% THD into the music can be detected, then the "gross difference" of .06% inserted into the signal by swapping to the stock power cord can also be detected using the same method.

2. Your not comparing two pieces of audio equipment, you are comparing two pieces of wire. The "entire stereophonic presentation" wasn't present in Petri-Larmi's test?

DarqueKnight

Everything I Know About Audio I Learned From The "Stupid things you hear from the DBT/Null test crowd...." Thread


1. Stereo sound can't be multi-dimensional because then it would be outside the current realm of existence...in another dimension.

2. The word "dimensional" can only be used within the context of quantum mechanics.

3. When two world renowned experts in the sensory evaluation of food used the term "multi-dimensional" in one of the standard textbooks in the field, they were incorrect, because the word "dimensional" only applies to quantum mechanics.

4. It is not appropriate to continually refer back to the teachings of Dr. Harvey Fletcher and his thuggish Bell Labs colleagues, who invented home stereo systems, because they are not the end-all be-all authority on hi-fi and their basic advice on how to evaluate the home stereo systems is not relevant today...even though the basic design and operation of home stereo is the same.

5. The only way to evaluate home audio equipment with "metrics" is with the use of laboratory analysis equipment.

6. Stereo equipment cannot be properly evaluated by a single person trained and experienced in stereophonic operation. The only proper way to evaluate stereo equipment is to invite a friend over to hide your equipment from you while you listen and determine which piece of equipment draws the biggest "emotional" response.

7. If you are not an "expert" in a field, it is better to keep your ideas to yourself because you can't possibly have anything valuable to contribute. The actress and the composer who co-invented and earned a patent in spread spectrum technology, one of the fundamental technologies in wireless communications, should have remained silent because they had no scientific credentials or expertise.

8. Science journals must not accept and publish papers from non-experts in the field, even though such papers will be reviewed by experts in the field.

9. If you hold a counter opinion to the DBT/Null test crowd and can't explain it in one or two simple sentences, then according to Einstein, you don't understand it well enough yourself. Einstein also said there were perhaps only twelve people in the whole world who understood his theory of relativity. Since Einstein couldn't make more than twelve people in the whole world understand his theory, he obviously didn't understand it well enough himself.

10. If you provide the citation and abstract of a scientific journal paper and someone reads the wrong paper, it is your fault for not providing a direct internet link to the paper.

11. If you summarize someone's comments exactly as they were written, you are mocking them and avoiding constructive dialog.

12. Dr. Harvey Fletcher and his thuggish Bell Labs colleagues should be ashamed of themselves for not applying the blind testing techniques (that they were well trained and experienced in using) to stereo.

13. All forms of audio are alike and any test suitable for one type of audio is suitable for all others.

14. You can't trust your ears because ears, like eyes, can be fooled. Most things that glitter aren't gold and most things that sound good are snake oil.

15. Blind testing (Clark 1987) proved that there was absolutely no audible difference between a $220 45 watt solid state receiver and a $12,000 80 watt tube amp.

16. High end = snake oil.

17. Home stereo systems became commercially available in 1958. Blind testing for home stereo systems was proposed in 1980. The period 1958-1980 is known in DBT/Null test circles as "The Dark Ages".

18. If you answer a question by providing citations to credible peer-reviewed research, you really didn't answer the question because you should have been able to explain it simply in one or two sentences. No one has time to read all that stuff. (See #9)

19. The snobbery and elitism of the early pioneers in stereo was revealed when Harvey and Schroeder, two thuggish Bell Labs scientists, said in 1961 that stereo was only for "serious listeners" who were "sophisticated in the art of sound localization" and who were "accustomed to playing the same records many times thus becoming familiar with the more subtle artistic and technical effects".

20. DBT/Null test cultists don't like to read. Case in point: Why come to a thread titled "Stupid things you hear from the DBT/Null test crowd..." and then express shock, disappointment and amazement that one or two people made a sarcastic comment?

This is a work in progress.

DarqueKnight

WilliamM2 wrote: »

1. If injecting .003% THD into the music can be detected, then the "gross difference" of .06% inserted into the signal by swapping to the stock power cord can also be detected using the same method.

I understand your frustration. As I said before, it is difficult to explain these things to people who are not knowledgeable of the basic principles if stereophonic system evaluation. It is the same principle as me trying to explain a concept to one of my students without them having done the required prior study.

Also, in light of some recent teachings in this thread regarding people writing papers outside of their area of expertise, I'm beginning to doubt the veracity of the Petri-Larmi paper. Margit Petri-Larmi was a medical doctor. She probably shouldn't have been writing about noise detection in stereo systems.

Also, are you aware of any blind tests for stereo in the scientific literature where such tests were applied according to the rules of proper stereophonic reproduction?

WilliamM2 wrote: »

2. Your not comparing two pieces of audio equipment, you are comparing two pieces of wire.

Wire is not equipment? Telecommunications companies and power companies consider the wire that transmits their signals to be "equipment". Similar to this, audiophiles consider cables to be another audio component just like amps and source components.

WilliamM2 wrote: »

The "entire stereophonic presentation" wasn't present in Petri-Larmi's test?

Of course it was. The point is that Petri-Larmi was not testing stereophonic performance. You should read the paper if this is something that really want to pursue further. I would also advise studying the basics of stereophony.

Ultimately what you learn and eventually come to understand is determined by your own investigation, study, experience and reasoning.

colddirt

I stayed up late last night reading this thread and it was very entertaining. I won't/can't get into an arguement with any of you about this subject because I'm neither articulate enough nor do I feel like getting upset. I'm guilty of bias - extream bias. When I grew up, my uncle told me what was good and what was not, and I listened real well. He said Onkyo (we called it Honkyo) was junk, pioneer, and basically everything else was either over priced or junk compared to what he had. He had Hafler, Adcom, and Denon components and he started out with Advents and then moved on to Polks. I've never owned anything other than this equipment. Do you know why? Because I can find equipment that I think sounds great within my limited circle of brands at a good price. I wouldn't bother listening to or think of buying a $20,000 pair of cool looking speakers because I can't fathom anything sounding better than the Polks I have. The same goes for the Denons I'm running - if I can't find an Adcom or Hafler power/pre amp at a decent price I don't need one............the Denons I've got sound great to me.

As for speaker cables, PLEASE.

There is certainly a reason no one wants to do an ABX test on these.............my SDA's have 16 gauge wire inside them - what point is there in running thousands of dollars of speaker wire OUTSIDE the cabinets when lamp cord is INSIDE of them? "Super" listeners can't tell the difference in ABX tests, and make excuses (I.E. - they're not valid for audio - what?) when/if they're forced to participate. If anyone has to make an excuse for justifying spending thousands of dollars on their equipment I don't know what to say. Before I lose any friends I may have made while I was here, I appologize if I offended you. Come over to my house and listen to my "junk" - since you already know what I'm running you already have a bias, now that makes two of us.

I'm going to go outside and play now while the sun is out. It's Portland and it's going to be 80 degrees.

WilliamM2

DarqueKnight wrote: »

I understand your frustration.

What frustration? As usual, you perception is completely wrong, and a product of your own imagination.

markmarc

colddirt wrote: »

I stayed up late last night reading this thread and it was very entertaining. I won't/can't get into an arguement with any of you about this subject because I'm neither articulate enough nor do I feel like getting upset. I'm guilty of bias - extream bias. When I grew up, my uncle told me what was good and what was not, and I listened real well. He said Onkyo (we called it Honkyo) was junk, pioneer, and basically everything else was either over priced or junk compared to what he had. He had Hafler, Adcom, and Denon components and he started out with Advents and then moved on to Polks. I've never owned anything other than this equipment. Do you know why? Because I can find equipment that I think sounds great within my limited circle of brands at a good price. I wouldn't bother listening to or think of buying a $20,000 pair of cool looking speakers because I can't fathom anything sounding better than the Polks I have. The same goes for the Denons I'm running - if I can't find an Adcom or Hafler power/pre amp at a decent price I don't need one............the Denons I've got sound great to me.

As for speaker cables, PLEASE.

There is certainly a reason no one wants to do an ABX test on these.............my SDA's have 16 gauge wire inside them - what point is there in running thousands of dollars of speaker wire OUTSIDE the cabinets when lamp cord is INSIDE of them? "Super" listeners can't tell the difference in ABX tests, and make excuses (I.E. - they're not valid for audio - what?) when/if they're forced to participate. If anyone has to make an excuse for justifying spending thousands of dollars on their equipment I don't know what to say. Before I lose any friends I may have made while I was here, I appologize if I offended you. Come over to my house and listen to my "junk" - since you already know what I'm running you already have a bias, now that makes two of us.

I'm going to go outside and play now while the sun is out. It's Portland and it's going to be 80 degrees.

If you look at the images of DK SDA system Colddirt, you'll find that that DK uses the very fine-sounding Parasound amps. If you regulary visit Echo Audio's website in our fine city of Portland you'll find that brand of amps coming thru from time to time at very reasonable used pricing. I used to own Adcom, and have plenty of listening time with Denon and Hafler. In the budget world I understand your positive experience. But I would suggest that you take advantage of Echo Audio's home demo option on used gear sometime, you'll be amazed at the increased resolution. Way back when I shared your view, but since then thru in-home experiences that there is a obvious differences in quality-built amps as one moves up the audio chain.

Regarding your speaker wire thoughts: Have you ever tried the types of speaker wire you reject? When I started Affordable$$Audio in 2006, in my first review I was writing about my experience with the Linn Keilidhs a respectable minitower that was being sold at Echo Audio for $600 or so used. With typical 14 gauge wire the speakers were dull, flat, and lacked any bass, even with the Adcom 535, the little sledgehammer of an amp. I switched over to some 20 year old Monster Cable at 10 gauge. Everything that was missing came to life immediately. Even my wife, who has zero appreciation for the audio hobby even commented with "Now that sounds nice." Once again you can either stop by Echo HiFi, or order a pair of "budget" speaker cables from say Audio Advisor, and give them a try, if you don't return them for a refund.

I will say that spotting differences in equipment and wires does get easier as you go up the audio chain. I'm NOT in any way saying your rig is bad or anything like that. What I am saying is that you should have some fun and audition gear outside your circle to experience the sheer variety of sound that exists.

DarqueKnight

markmarc wrote: »

Way back when I shared your view, but since then thru in-home experiences that there is a obvious differences in quality-built amps as one moves up the audio chain.

markmarc wrote: »

I will say that spotting differences in equipment and wires does get easier as you go up the audio chain.

I dare you to prove any of that with a blind test.

markmarc wrote: »

What I am saying is that you should have some fun and audition gear outside your circle to experience the sheer variety of sound that exists.

Fun? This is supposed to be fun? You talk just like the elitist snobs at Bell Labs who invented home stereo only for "music lovers" and "serious listeners".

tonyb

colddirt wrote: »

I've never owned anything other than this equipment. Do you know why? Because I can find equipment that I think sounds great within my limited circle of brands at a good price. I wouldn't bother listening to or think of buying a $20,000 pair of cool looking speakers because I can't fathom anything sounding better than the Polks I have. The same goes for the Denons I'm running - if I can't find an Adcom or Hafler power/pre amp at a decent price I don't need one

Thats the whole arguement in a nutshell. If you don't venture outside your comfort zone, you'll never discover anything....and that applies to more than audio.

nooshinjohn

colddirt wrote: »

I won't/can't get into an arguement with any of you about this subject because I'm neither articulate enough nor do I feel like getting upset. I'm guilty of bias - extream bias.
.

A great idea if you have no experience in the matter... wait, what's this???

colddirt wrote: »

As for speaker cables, PLEASE.

Ok... this is where you start to come off the rails...

colddirt wrote: »

There is certainly a reason no one wants to do an ABX test on these.............my SDA's have 16 gauge wire inside them - what point is there in running thousands of dollars of speaker wire OUTSIDE the cabinets when lamp cord is INSIDE of them? "Super" listeners can't tell the difference in ABX tests, and make excuses (I.E. - they're not valid for audio - what?) when/if they're forced to participate. If anyone has to make an excuse for justifying spending thousands of dollars on their equipment I don't know what to say. Before I lose any friends I may have made while I was here, I appologize if I offended you. Come over to my house and listen to my "junk" - since you already know what I'm running you already have a bias, now that makes two of us.

I'm going to go outside and play now while the sun is out. It's Portland and it's going to be 80 degrees

And now you've lost your mind.

Based upon you beginning your post by saying you don't know what you are talking about in this matter, I will dismiss the rest of your post as the rantings of a drunken poster, otherwise I would have to say that you are full of B.S.

nwohlford

DarqueKnight wrote: »

Again, biases are only relevant to the extent that they will prevent a fair and accurate evaluation.

A person does not get to decide which of their biases are irrelevant. No one is very honest with themselves about their biases.

DarqueKnight wrote: »

Some brands market exclusively on snob appeal rather than performance. Then know that knowledgeable buyers will equate high price with high performance. Monster Cable has been very forthcoming in admitting that overpricing their products was part of a strategy of inducing consumers to attribute higher quality to higher prices. Noel Lee says he was surprised that it worked so well...for wire. By the way, I have quite a bit of Monster's products, which I bought at substantial discounts on closeout and on the used market.

However, a true audiophile is not a lover of expensive electronics. A true audiophile is a lover of high quality sound. If I were an audio snob I would not have a house full of Polk Audio loudspeakers

For most people, the biases probably have very little to do with money. It does not make them any less of a concern.

DarqueKnight wrote: »

I agree. I don't know of anyone this forum who does this. Manufacturers engage in this type of marketing hype, but the consumer has to use some common sense and realize that no one thing is going to be to everyone's benefit.

There are definitely people who make sweeping generalization on this forum (and everywhere else). (I am too lazy to insert an ironic emoticon here, but you get the point.)

DarqueKnight wrote: »

I don't agree that you need to be an expert in a field before your ideas and contributions can be taken seriously. This is an attitude of snobbery that can cause the dismissal of valuable contributions. If you consult with the U.S. Patent Office, you will find scores of valuable inventions made by people with no expertise in the field of the invention. One of the most famous examples of this is the invention in 1942 of spread spectrum communications technology by Hedy Lamarr (an actress) and George Antheil (a composer and musician).


Your mindset is the type of thinking that fuels ad hominem attacks: if you can't discredit an idea-discredit the person offering the idea. You may find this shocking, but you actually don't have to be a credentialed expert in a field to make a valid contribution. You do however, have to have some knowledge, understanding and reasoning skills.

I have some knowledge and experience in the area of testing human subjects/observers. I have expertise in the design, performance evaluation and manufacturing of electronic equipment.

I said years of concentrated study/work experience. This learning shows in the work you do. No one needs to check credentials. I have known phd physicists who have made valuable contributions to statistics but they did it after years of working in the field full time. The era of the "gentleman scientists" is finished. Fields have become too specialized. Efforts such as yours really require across specialty collaboration now. This is not snobbery but just a general fact. I have met a few exceptions to this in my life, but these people are truly the extremes. (If the people around you don't think that it is likely that you will win a noble prize one day, then you are probably not in this group.)

DarqueKnight wrote: »

When I submitted a paper on audio equipment evaluation to the world's premier sensory science journal, the review panel did not check my credentials. If you look at the editorial review board of the journal, you will see that it is staffed by people with international reputations in the field of sensory science. Many of them hold or have held leadership positions on national and international standards setting bodies. Fortunately, these people were not narrow-minded and ignorant to the point that they believed that someone outside their field couldn't possibly have something valuable to contribute.

I am curious. Did the peer reviewers include a statistician?

For others on this forum, it worth stressing as you note that being published in a journal just means that the journal viewed the paper as something that could add to the discussion in the field. Being published in a journal is not an endorsement of your ideas, and not a statement that these ideas are the norms within a field.

DarqueKnight

nwohlford wrote: »

A person does not get to decide which of their biases are irrelevant.

What then, determines the relevancy of specific bias to a decision? Please indicate whether your answer is personal opinion or if there is scientific substantiation. If there is scientific substantiation, please point to it.

nwohlford wrote: »

No one is very honest with themselves about their biases.

No one...out of multiple billions? Would you provide citations to some scientific studies that prove this?

I find it strange that you would say this after I specifically mentioned my strong bias against certain aesthetic features in audio equipment.

nwohlford wrote: »

For most people, the biases probably have very little to do with money. It does not make them any less of a concern.

Out of the total world population, what percentage constitutes "most people"? How relevant is that group's bias profile to to the subject of the use of blind testing in evaluating stereophonic audio systems?

nwohlford wrote: »

There are definitely people who make sweeping generalization on this forum (and everywhere else). (I am too lazy to insert an ironic emoticon here, but you get the point.)

Are you including yourself in that number? See your comments above.

nwohlford wrote: »

I said years of concentrated study/work experience. This learning shows in the work you do. No one needs to check credentials. I have known phd physicists who have made valuable contributions to statistics but they did it after years of working in the field full time. The era of the "gentleman scientists" is finished. Fields have become too specialized. Efforts such as yours really require across specialty collaboration now. This is not snobbery but just a general fact. I have met a few exceptions to this in my life, but these people are truly the extremes. (If the people around you don't think that it is likely that you will win a noble prize one day, then you are probably not in this group.)

You specifically said:

nwohlford wrote: »

It is nice to have interests outside your area of expertise, but one must always remember that they are not an expert in that other field.

Then you said what "expertise" required.

nwohlford wrote: »

Expertise in any field such testing with human subjects really requires years of concentrated study/work experience.

I agree with your two statements above. I also claim no expertise in human testing, although I have experience in the area from my work in the telecommunications industry (AT&T and Bell Laboratories). However, I must point out that the subject of my journal paper was not blind testing of subjects. The subject of the paper was descriptive evaluation methodology for stereophonic audio equipment. The human subject trial constituted 3.5 pages out of 15. Of the 3.5 pages, 2.5 pages were text and 1 page was tables and figures. It was there primarily to illustrate the inaccuracies that could occur when applying that type of testing to stereo system evaluation.

The methodology I proposed was something I developed over 25 years as a consumer and as a practicing electrical engineer. Are you insisting that my 25 years of experience in listening evaluation, electronic modification, and electronic and acoustic performance testing of stereo systems does not qualify me to write a cogent paper on an evaluation methodology for such systems?

nwohlford wrote: »

I am curious. Did the peer reviewers include a statistician?

I don't know the identities of the reviewers. Assuming there was a statistician on the panel, what is the implication? Assuming there was not a statistician on the panel, what is the implication?

The acknowledgement section of the paper includes the names of people I consulted with during development. That group includes:

1. A co-author of the standard textbook in the field of sensory evaluation techniques. This person also did some pioneering work in the sensory evaluation of sound.
2. Two Ph.D.'s in psychology who work in academia.
3. A research engineer who had published several studies in the application of sensory evaluation of sound.
4. A Ph.D. in cognitive and perceptual psychology who works for a Silicon Valley software company.

I did not assess the level of expertise in statistics of the five individuals mentioned above. Their publications and their academic credentials indicate that they have some proficiency in the area.

I have some training and experience in statistics, although I don't know if rises to your standards of expertise. My doctoral research in electrical engineering involved the probabilistic modeling of traffic on packet communications networks. I hold one patent in the area of probabilistic modeling for packet networks. For the four years immediately prior to entering academia, I was engaged in research and development at Bell Laboratories where I conducted applied research in the area of traffic modeling and performance analysis of enterprise network switching products.

nwohlford wrote: »

For others on this forum, it worth stressing as you note that being published in a journal just means that the journal viewed the paper as something that could add to the discussion in the field. Being published in a journal is not an endorsement of your ideas, and not a statement that these ideas are the norms within a field.

This is true of every journal in every field? If so, how do you know this?

DarqueKnight

WilliamM2 wrote: »

What frustration? As usual, you perception is completely wrong, and a product of your own imagination.

OK. Back to the original topic.

Did I adequately address your concerns about the audibility of the noise reduction resulting from the switch in power cords?

As I recall, your issue was that fluctuations in incoming power THD would mask any noise reduction seen at the loudspeaker input. I showed that the variation in incoming THD was only +/- 0.2% and that that same amount of THD fluctuation exists at my location today.

Furthermore, the power supply filtration produced another 13.75x reduction in incoming line noise. If we extrapolate the effect of the power supply's filtration on the incoming THD (i.e. reduce by 13.75x), the outbound THD fluctuation becomes +/- 0.015%.

Therefore, I am thinking the line noise reduction at the speaker cable output should be audible, if the detection threshold in the Petri-Larmi AES study is valid.

What do you think?

WilliamM2

DarqueKnight wrote: »

Therefore, I am thinking the line noise reduction at the speaker cable output should be audible, if the detection threshold in the Petri-Larmi AES study is valid.

What do you think?

I think you have a theory, now all you need to do is test that theory, the same way as Petri-Larmi. That's just what I said before, the difference should be clearly audible in ABX testing, since it's 20 times greater than what they detected in their testing.

Now for the long winded excuse as to why ABX testing isn't suitable, even though you use it to arrive at your (unproven) conclusion. Rinse, repeat.

unc2701

Random question for DK, you have have said this previously and I just missed it or forgot:
For the human subject part of the publication, what did you use for the IC... and did the journal ask you about that?

DarqueKnight

WilliamM2 wrote: »

I think you have a theory, now all you need to do is test that theory, the same way as Petri-Larmi. That's just what I said before, the difference should be clearly audible in ABX testing, since it's 20 times greater than what they detected in their testing.

Actually Petri-Larmi used A/B testing, not ABX.

At this point, I think any serious listener has enough information to do their own evaluation with whatever test methodology they choose.

WilliamM2 wrote: »

Now for the long winded excuse as to why ABX testing isn't suitable, even though you use it to arrive at your (unproven) conclusion. Rinse, repeat.

No need for further explanation as the bases have been covered. Thanks for your insights.

DarqueKnight

unc2701 wrote: »

Random question for DK, you have have said this previously and I just missed it or forgot:
For the human subject part of the publication, what did you use for the IC... and did the journal ask you about that?

I recall that you and I had a long discussion a couple of years ago when the paper was published. You might want to review that thread as it is easy to find. What I remember is that you had a very low opinion of the paper's subject matter and a very low opinion of the quality of the journal and its editorial staff.

This is from post #169 in this thread:

DarqueKnight wrote: »

However, I must point out that the subject of my journal paper was not blind testing of subjects. The subject of the paper was descriptive evaluation methodology for stereophonic audio equipment. The human subject trial constituted 3.5 pages out of 15. Of the 3.5 pages, 2.5 pages were text and 1 page was tables and figures. It was there primarily to illustrate the inaccuracies that could occur when applying that type of testing to stereo system evaluation.

The methodology I proposed was something I developed over 25 years as a consumer and as a practicing electrical engineer.

As I recall, the main thing the journal wanted me to do with the human subject trials was take two of them out and summarize the results of the remaining one for the sake of brevity. The original submission included three human subject case studies.

In the future, when I am more dedicated to audio than I am now, I will submit a paper with the other two case studies.

Syndil

I see much more chaff has been posted (wow, nice job on #160!), but post #145 has still not been properly addressed. I was really looking forward to some sort of a rebuttal. Disappointing.

Here it is again, minus the (apparently) very distracting disagreement over semantics (which btw still gives me a chuckle). Anyway:

DarqueKnight wrote: »

1. Blinding in trials with complex sensory stimuli and untrained subjects can lead to confusion, stress and results skewed by guessing (guessing bias).

2. Blinding in trials with complex sensory stimuli and trained subjects is unnecessary and adds an unwarranted level of complexity to experimental procedure. No credible scientist would advocate a standard test methodology that is more complicated than it needs to be.

If a potential for bias exists, then the conclusion that the blinding is unnecessary is not a foregone one. And I would submit that the potential for bias nearly always exists, and it seems our resident statistics expert nwohlford agrees. Frustratingly complex or not, if the goal is to produce results free of bias, the complexity must be dealt with. If guessing is the concern, then the randomness of the guessing is easily discovered by spreading the comparison over numerous trials and looking for a result of 80% or better, or whatever standard you adhere to that would indicate a non-random result.

I was advised by the journal's review panel, which was composed of sensory science experts, that the blind nature of the trial was unnecessary. I was further advised that blinding with minimally trained subjects could induce guessing as the subject might focus on getting the right answer rather than focus on the sensory stimuli presented. More information on that journal publication can be read here.

In this case, the type of trial being conducted has nothing in common with the type of XY comparison an audiophile would conduct in order to rank one piece of equipment versus another. A blind trial may very well be unnecessary for an untrained listener trying to determine localization of images within a sound stage, but dismissing blind testing for XY comparisons based on this is a non sequitur. Apples to oranges--the logic does not follow. [Added: Not all blind testing is ABX testing, and on this point we actually agree, ABX testing is not suitable for ranking audio gear. See post #12 in this thread.]

In conclusion, I will restate the four points offered previously in post #78:

1. Forced-choice discrimination test methods, such as ABX, A/B, etc., are not suitable for multi-dimensional stimuli. They are more suited for simple discrimination tests where the subject's senses are not affected by a number of simultaneous stimuli. A "dimension" in this case being a characteristic or unit of measurement.

2. A stereophonic sound field is multi-dimensional in nature in that it presents a complex set of simultaneous stimuli to the listener: sound stage height, width and depth; the location and character (clarity, weight, detail, dynamics, etc.) of images in the sound stage, and tactile sensation.

We do not need our sense of taste, sight or smell in order to listen to music. One can feel loud and low sound waves as they pass through your body, but this is not always present in audio, and one could argue that it is a secondary characteristic: nice to have, but not something that is necessary for the enjoyment of the music. Some may even find it detrimental to the experience or distracting. The enjoyment of audio is completely subjective, after all.

The perceived location of the origin of the sound as well as its character are all direct products of a single stimuli: sound waves. One does not need to view the woofers moving or the VU meters dancing or the graphic equalizers pulsing or what have you in order to enjoy the sound. Those other stimuli may add or subtract from the experience as a whole depending on your tastes, but when it comes to judging the quality of the audio they have no bearing, and can only serve to distract or perhaps even create bias, i.e. "those VU meters look cool, so I want to like this piece of gear more than the one it is being compared to." And bias is the enemy. [Added: The point here is that neither of your above quoted points offer an explanation as to why blind testing should be inappropriate for "multi-dimensional stimuli"--they merely reiterate the as yet unjustified claim as such. Perhaps points #3 and #4 will offer some justification...

3. The Bell Laboratories scientists who invented home stereo systems were experts in the application of forced-choice discrimination test methods to telephony audio. Such methods, like ABX and A/B, were well suited for discerning simple quality differences in limited bandwidth telephone voice circuits.

4. When these same Bell Laboratories telephony scientists turned their attention to developing home audio equipment, the evaluation methods for such equipment was based on trained listeners who knew how to properly evaluate all the dimensions in stereophonic sound reproduction. Furthermore, peer-reviewed scientific journal literature clearly states that the stereophonic test methods used by Bell Laboratories scientists were intended to be adopted by the consumer.

That's all well and good, but the scientists at Bell Labs are not the end-all be-all authority on hi-fi audio, despite your heavy reliance on their work, as you seem to refer back to them constantly. [Added:] This is an appeal to authority fallacy. Logic does not care who said what; logic cares only for the validity of the reasoning behind what was said. Constantly referring back to these scientists and saying "well, they didn't need it, why should I?" or "blind testing was never intended for stereophonic audio" does not provide us any insight as to why it might be considered inappropriate for stereophonic audio.

And here is my argument in a nutshell:

Regardless, my initial [simple] question has still not been answered. No one can tell me why a blinded test (appropriately conducted over several trials to eliminate randomness from guessing) should be inaccurate. Going back to my original example (trying to keep it simple), why should comparing cable X versus cable Y produce any different results from comparing a Pear Audio cable versus a Monoprice cable, if the only difference between the comparison methods is the knowledge or lack thereof of the specific gear being tested at the time? I submit that it is indeed a forgone conclusion that, unless bias towards one of the brands is present in the unblinded test, the tests should produce exactly the same results. And if that is indeed the case, then it is also logical to conclude that XY testing eliminates potential bias from unblinded testing. And if that is the case, then I am left to questioning the motivation for preference against blind testing, which would seem to me to be an admission of a preference towards bias.

DarqueKnight

One of my most vivid childhood memories is the conversation I had with my parents at the age of 3 concerning where babies came from. My mother was 6 months pregnant with my sister and I casually remarked to my father that "mama is getting fat". My father explained, or he tried to explain, that my mother was not getting fat but that her stomach was getting bigger because a baby was growing inside of her.

Of course, then I wanted to know how the baby got inside my mother and why would a baby do that. My father gave a very straightforward explanation using analogies from our experiences planting vegetables in the garden behind our home. I did not accept father's explanation. I would have accepted that babies come from the hospital, from church, or from a baby store.

I also did not accept the fact that I was once a baby and that I once grew inside my mother's body. When I asked my mother where babies came from, she laughed and said everything my father said was true.

On a visit shortly thereafter to my maternal grandmother's, I asked her where babies came from and she said "from the coming place" and laughed. When I asked where the "coming place" was, she laughed and said "ask your mama". When I asked my mother where the "coming place" was she laughed and said "that's a special place on a mother's body where the baby is planted". At that point I gave up.

Three months later, I felt that a great secret had been revealed when my father and I went to pick up my mother and sister from the hospital. I thought I finally had discovered where babies came from.

Some more years passed before my mind was able to grasp and accept the real truth.

headrott

DarqueKnight wrote: »

One of my most vivid childhood memories is the conversation I had with my parents at the age of 3 concerning where babies came from. My mother was 6 months pregnant with my sister and I casually remarked to my father that "mama is getting fat". My father explained, or he tried to explain, that my mother was not getting fat but that her stomach was getting bigger because a baby was growing inside of her.

Of course, then I wanted to know how the baby got inside my mother and why would a baby do that. My father gave a very straightforward explanation using analogies from our experiences planting vegetables in the garden behind our home. I did not accept father's explanation. I would have accepted that babies come from the hospital, from church, or from a baby store.

I also did not accept the fact that I was once a baby and that I once grew inside my mother's body. When I asked my mother where babies came from, she laughed and said everything my father said was true.

On a visit shortly thereafter to my maternal grandmother's, I asked her where babies came from and she said "from the coming place" and laughed. When I asked where the "coming place" was, she laughed and said "ask your mama". When I asked my mother where the "coming place" was she laughed and said "that's a special place on a mother's body where the baby is planted". At that point I gave up.

Three months later, I felt that a great secret had been revealed when my father and I went to pick up my mother and sister from the hospital. I thought I finally had discovered where babies came from.

Some more years passed before my mind was able to grasp and accept the real truth.

Do you think he'll get the analogy in your story Ray? Let alone the analogy of your story to what's going on in this thread?

Greg

tonyb

headrott wrote: »

Do you think he'll get the analogy in your story Ray? Let alone the analogy of your story to what's going on in this thread?

Greg

Was thinking the same thing....probably not. I was expecting the usual response like....what does babies have to do with anything ?
DK deserves a cookie for his patience anyway.

DarqueKnight

headrott wrote: »

Do you think he'll get the analogy in your story Ray? Let alone the analogy of your story to what's going on in this thread?

No, but that's ok because it wasn't written for him. Those who get it, get it.

headrott

DarqueKnight wrote: »

No, but that's ok because it wasn't written for him. Those who get it, get it.

Just as stereophonic audio. Maybe not all high fidelity audio was intended to be heard by everyone.

Syndil

More of what I expected, really. Oh well. Was fun while it lasted.

evhudsons

As some here I respect and sometimes understand has said, careful and consistent listening will cause your ears to evolve. Its my favorite quote ever on this forum and is completely true. With this debate it's hard for noobs like me to understand, but some folks don't get that quote above and therefor their lack of listening lessens their ability to let their ears evolve.

As far as testing, each person has different aspects they look for and their reactions differ. But when at least two people can look at each other and say wow that was an improvement, you know you are on to something worth testing.

My biggest interest in school was psychological testing. It's an awesome range of a variety of tests to test just about anything. There is certainly a place for psychoacoustics. For instance I minimize my component lights and meters. I don't like silver finish. Both take away from listening to music. I have a clean and artistic comfy setting with warm lights. I prefer this setting but others like meters, flashy components and lots lights.
I like slightly humid and 80 degrees, my wife 72 degrees and desert dry. I like to have no fans or Ac because it's not audio. I wind up sharing Ac and fan noise, fridge noise, wife banter and dog snoring.

If your ears are physically different you receive a different audio experience. Simply move your ears in or out a little and it's a huge difference. What if you had bigger ears flared out or flat small ears? Your experience would be entirely different. So to have a good test the inherent differences would have to be averaged and the settings as consistent as possible, but also either consistency in types of listeners or factor in the various groups. So folks with vast knowledge and experience such as h9 and DK are in a category who can discerne differences that your mother in law listening to englebert Humperdinck could only give blank looks to would be best to handle such tests.
If you put a bunch of southern speaking folks together I could in these sorts of tests tell you not Only what state they are from but sometimes even what city. I spoke with a random stranger the other day and surprised her when I asked her which high school she went to in Montgomery Alabama. My experience has given me this specialized insight ability to tell the subtle but certainly present differences in their speech patterns, inflections and tones. Someone from Russia might only discerne they are from the South based on watching movies or TVs. My experience would be analogous to high end audio equipment with experience to hear stereophonic differences. The Russian might have recieved their analogous experience in a cheap HTIB.
Of course I would have a hard time making such distinctions from just one spoken word. I would have to listen to a conversation. In audio you would have to have a listening session, not just a quick sound bites worth to make a distintion. But some differences are so much faster to make when you are educated about what you are listening for and what may contribute to the difference.

I got two of the lowest end MIT ic cables, and my wife noticed right away despite not knowing I changed a thing. I had been using basic RadioShack and basic monster cable interconnects before. I was surprised at the immediate difference.

One test I did is after reading a snake oil report that electrical wire performed as well as $10,000 speaker wire I tried using some electrical wire I had. It had a bigger gauge than what I was using. It changed immediately but seemed sterile and lacking that nice spatial and warm tone. I upgraded the gauge on speaker wire and was rewarded with music. Wire makes a huge difference. At what point is it worth arguing though if you haven't made the proper equipment change to know what a quality wire will do. For my setup low end MIT is just fine.

I've tried the various amps and just acquired a NAD. My favorite thus far is parasound by far. AdcOm second and nad 3rd. The b&k is nice to. My opinion is that the avr is my weak link. It's not bad but it doesn't have the same nice sound.
But of course each piece is used, varying age, and hooked to various speakers. It would take me quite a bit of time to do these tests. I would put my faith in opinions of people that know how to test, what to test for, and know why the differences occur.

Just speaking as an enthralled, listening, and trying to understand, but grateful for your thoughts noobie.
Without folks like H9, DarqueKnight, Nspindle, and Trey/vrsomething, and many others I would not have the audio joy I have at home. My wife thanks you all too.

evhudsons

I was stuck in jury duty and wrote that long post on my iPhone. The guilty pleaded so I'm free now.

Keep up the debate. I love it! Might have to sticky this one. I also loved the links.