Stupid things you hear from the DBT/Null test crowd....

nwohlford

As a statistician, I find a lot of this pretty funny. It is nice to have interests outside your area of expertise, but one must always remember that they are not an expert in that other field. Maybe DarqueKnight has some expertise in the area of testing with human subjects/observers, but I don't see any claim of that (nor evidence in his opinions). This would be like me reading a few electrical engineering journal articles and claiming some expertise. Expertise in any field such testing with human subjects really requires years of concentrated study/work experience.

The idea of an unbiased observer is a myth. Biases (or preconceptions) are very basic to human decision making. One can try to reduce his/her own biases by being very honest about their biases and in a sense trying to correct for them. I can thinking of training listeners for preforming unblinded studies where a large part of the training would be learning your biases as a listener and learning how to try to minimize your biases.

I almost never use blinded testing in my own audio comparisons, but I am very careful about making any comments comparing two audio components. If I do state a preference for an audio component, I try to state the limitations of any comparisons made.

Realistically, some of the enjoyment we get out of audio has nothing to do with sound. It partly has to do with the look of the components, how owning some brand name makes us feel, and so on. This is true of any hobby, and to trying to remove it from the equation often will remove the fun. I think that everybody should buy what allows them to enjoy the hobby. One though has to be carefully when making sweeping comments about whether a certain component will improve somebody else's experience.

Syndil

heiney9 wrote: »

Sorry, but you have explained nothing. Where did you get a copy of the published Journal?

Another dodge. Since you are unable to answer in your own words what about blind testing skews the results towards inaccuracy, then I must conclude that you have no answer.

heiney9

As a statistician, I find a lot of this pretty funny. It is nice to have interests outside your area of expertise, but one must always remember that they are not an expert in that other field. Maybe DarqueKnight has some expertise in the area of testing with human subjects/observers, but I don't see any claim of that (nor evidence in his opinions). This would be like me reading a few electrical engineering journal articles and claiming some expertise. Expertise in any field such testing with human subjects really requires years of concentrated study/work experience.

.

Apparently you haven't read the peer reviewed and accepted Journal paper either. He may not be an expert (whatever defines that anyway) but to devise a study, present the findings of said study, have it rigorously reviewed by a peer panel and then have it accepted for publication in the field is helluva lot more than any of the naysayers have done. So I'd say it's up to them to refute the issues not the other way around. As the above process has earned more credibility than anyone else arguing on both sides and that includes myself.

Again, I am not saying that after analysis of all the data, etc one has to agree, but so far I have not seen a single person pose any sort of logical methodology coming anywhere close to refuting or even challenging what was presented.

Hell, I can bet most if not all those arguing against have even read the study published in the Journal, yet they say "I've reviewed and processed all the information and it's poppycock"

H9

heiney9

Syndil wrote: »

Another dodge. Since you are unable to answer in your own words what about blind testing skews the results towards inaccuracy, then I must conclude that you have no answer.

I have very carefully fully explained why I won't answer your question, you obviously aren't willing to accept that, so I guess we're done.

H9

Syndil

I am beginning to think that you don't understand this oft-quoted journal paper either, since all you are able to do it point to it and say that the answer lies within it somewhere. If you do understand how this paper proves that blind testing produces inaccurate results, then explain it to me simply.

DarqueKnight

DarqueKnight wrote: »

A naysayer is going to irrationally deny a different viewpoint no matter what. Here is a perfect example of how they do: Reading Is Fundamental

WilliamM2 wrote: »

I didn't deny your evidence. I read the wrong study. Not surprising, when you consider that you didn't link to the actual study directly, but rather to your own opinion posts as usual.

WOWWWWWWWWWW! Let me make sure I have this straight:

1. YOU read the wrong study but it's HIS FAULT because he didn't link to the actual study? In other words, it was not enough for him to list the full citation, as well as the FULL ABSTRACT for the study in his opinion post, he actually needed to sit you in his lap and spoon feed you the information because you are clearly not capable of accessing accurately cited information yourself? INCREDIBLE!!!

2. Anyone with the most basic of reasoning ability should have been tipped off by the differences in title and abstract. However, I understand the thinking: Both papers were written by the same authors and similar to the concept that all amplifiers sound alike, all papers written by the same authors read alike.

3. I would make some statement about this being a textbook case of delusional, pathological naysayerism, but I'm trying to get (back) to Heaven.

WilliamM2 wrote: »

You never did explain how Petri-Larmi conducting DBT's for artificially injected distortion on an audio signal, proves that there is an audible difference in power cables. Your measurements aren't convincing, unless you found some way to keep the THD on the power line constant, which it is not.

Sorry I'm not as easily fooled as BF.

Really? You are not easily fooled? How easy was it to fool you in to making a fool of yourself by attempting to accuse me of hypocrisy and false statements while you obviously did not understand what you had read?: Reading Is Fundamental

I understand your frustration. If it was impossible to distinguish between two papers, with two different titles, and two different abstracts, yet written by the same authors, I can imagine the difficulties my explanation caused. However, rather than making knowledgeable assumptions about my power line quality, it would have been more appropriate to simply ask about the power line THD fluctuations from moment to moment at my location.

You say my measurements are not convincing because the THD on the power line is not constant. It does not need to be constant, it needs to be within an appropriately close range (without wild fluctuations) over the measurement period. The FFT measurements shown here were taken on Saturday afternoon August 16, 2008. At the time of these measurements, the power line THD was consistently in the range of 2.2-2.4% during Monday-Friday business hours, then dropped to around 2% in the evening, then dropped again in the early morning hours to 1.7-1.8%. Therefore, over a one hour measurement/listening period on a typical workweek day, the power line THD would vary up to 2/10ths of one percent (0.002%). On the Saturday afternoon when these measurements were taken, the power line THD ranged from 2-2.2%.

THD was constantly monitored by a PS Audio Power Plant Premier and measured on the spot with an AEMC OX 7102-C portable oscilloscope with harmonic analyzer function.

I may also need to point out that the FFT plots shown here ARE NOT 60 Hz power line signals. They are 60 Hz test signals measured at the output of the speaker cable. Please verify if you were aware of that. The noise amplitude at the output of the speaker cable was consistently 0.11 volt (0.16% of the signal amplitude) with the stock power cable and consistently 0.07 volt (0.10%) with the PS Audio Statement SC power cable.

To summarize:

1. Petri-Larmi (1980 AES, not 1980 IEEE) demonstrated that the threshold of audibility for noise was 0.003% for trained and sensitive subjects.

2. The noise levels AT THE OUTPUT OF THE SPEAKER cables was consistently 0.16% with the amplifier's stock power cord and 0.10% with the PS Audio Statement SC power cord. These noise levels are 53x and 33x the audible noise levels of the Petri-Larmi (1980 AES, not 1980 IEEE) study.

3. The power line noise level varied no more than 0.002% over the measurement period. A significant portion of the incoming noise was filtered out by the power amp's power supply.

4. The noise levels at the output of the speaker cable were 80x the power line noise level maximum deviation in the case of the stock power cord and 50x the power line noise level maximum deviation in the case of the Statement SC power cord.

Question: Considering items 1-4, what other factors call into question the audibility of the noise reduction at the speaker cable output? In other words, a 0.002% fluctuation in incoming power line noise will render the speaker cable signal noise content reduction inaudible because ___________. (Please fill in the blank.)

pearsall001

Interesting yet humerous reading. Both sides having valid points & information. But in the end nothing more than the usual CYA stance.

tonyb

Neither side is going to convince the other of anything different. Why waste the time ?

It's akin to someone with no experience with women trying to tell someone with alot of experience with women what girl he should go out with.

halo71

tonyb wrote: »

Neither side is going to convince the other of anything different. Why waste the time ?

It's akin to someone with no experience with women trying to tell someone with alot of experience with women what girl he should go out with.

+1

The other thread was locked. And this one is almost a mirror image of it.

DarqueKnight

halo71 wrote: »

The other thread was locked. And this one is almost a mirror image of it.

tonyb wrote: »

Neither side is going to convince the other of anything different. Why waste the time?

Is going to a football game a waste of time? What about bowling or playing pool? I mean, couldn't the time spent sitting and watching a football game be more productively spent elsewhere? Is there any value to entertainment?

One person gets entertainment value from discussing controversial audio topics. Another person gets entertainment value discussing their toilet habits. Which one is "wrong"?

tonyb wrote: »

It's akin to someone with no experience with women trying to tell someone with alot of experience with women what girl he should go out with.

I agree...but we have had threads like that too...and no one complained.

When someone starts a thread discussing their toilet functions or problems with their cheating, stripper girlfriend, I do not feel compelled to participate or to complain about the existence of the thread. I just turn my attention elsewhere.

DMara

This kind of thread really takes the fun out of listening to music :biggrin:, unfortunately it happens all the time in our Clubhouse. There'll be no winner, just animosity.

DarqueKnight

DMara wrote: »

This kind of thread really takes the fun out of listening to music :biggrin:, unfortunately it happens all the time in our Clubhouse. There'll be no winner, just animosity.

The smiley face indicates you might be joking, but in case you aren't, why you would say that? I am enjoying music from my stereo as I write this.

I certainly don't have any animosity toward anyone because they disagree with my approach to audio. Nothing anyone else says can diminish the daily joy I receive from listening to a high resolution stereo system.

If there is any animosity, I would expect it would come from the scores of people who express dissatisfaction with their stereo systems by demanding proof that other people can hear certain things.

halo71

DarqueKnight wrote: »

Is going to a football game a waste of time? What about bowling or playing pool? I mean, couldn't the time spent sitting and watching a football game be more productively spent elsewhere? Is there any value to entertainment?

One person gets entertainment value from discussing controversial audio topics. Another person gets entertainment value discussing their toilet habits. Which one is "wrong"?

Yep, but the difference is a football game eventually comes to an end. :cheesygrin: Arguing over cables over and over, saying the same thing over and over trying to change ones mind but getting no where? Thats what I was getting at. If you can not at least get someone to try something new, why continue trying to convince them there is a difference.

DarqueKnight

halo71 wrote: »

Yep, but the difference is a football game eventually comes to an end. :cheesygrin: Arguing over cables over and over, saying the same thing over and over trying to change ones mind but getting no where? Thats what I was getting at. If you can not at least get someone to try something new, why continue trying to convince them there is a difference.

Well, in my case, I am not trying to "convince" anyone. I am merely participating in a thread on a subject that interests me and answering the inquiries posed to me.

Obviously, I understand that some people are mindset and that no evidence will shake their "faith". However, I think you will agree that it is always interesting to see the mental gyrations that people will come up with. Like, for example, this:

WilliamM2 wrote: »

I read the wrong study. Not surprising, when you consider that you didn't link to the actual study directly, but rather to your own opinion posts as usual.

Sure, after I provided accurate citation and a word-for-word abstract, it was still MY fault that he read the wrong paper.

DarqueKnight

Syndil wrote: »

Einstein once said that if you can't explain something in layman's terms, then you probably don't understand it well enough.

1. Einstein actually said "If you can't explain it simply, you don't understand it well enough".

2. Einstein also said "There are not more than twelve people in the whole world who can understand the Theory of Relativity." [New York Times, 11-9-1919]

So, if the first statement is taken at face value, then no one could have understood Einstein's Theory of Relativity when it was first published...including Einstein, because he should have been able to explain it in simple layman's terms to anyone...including the janitor.

Many people hang on Einstein's every word as if everything he said was some profound wisdom from the great beyond. Like many highly intelligent people, Einstein had a sarcastic sense of humor. For proper understanding, you must frame his more extreme comments within the context of his personality and sense of humor. Sometimes you must use some sense to properly interpret what he said.

Statement one, taken literally, must be framed within the context of the person receiving the explanation having the required knowledge to understand a simple explanation. A team of neurosurgeons would probably understand a simple explanation of a new neurosurgical procedure. A team of janitors probably would not. It is not reasonable to impugn a neurosurgeon's intelligence because he is not able to simply explain complex brain surgery concepts to a janitor.

Statement 2 was a statement made in jest to the editor of the journal that published the theory of relativity. I don't know how many people were on the peer review panel for his theory of relativity paper. I would assume a minimum of three...perhaps five or more considering the highly controversial subject matter. We must also assume that those reviewers understood the theory, otherwise the paper would not have been approved for publication. Worldwide, there were far more than twelve people who accepted the theory. We can assume that they understood it also.

Syndil wrote: »

I'm trying to keep this simple. A dissertation is not required to answer my simple, one sentence question, as to how blind testing might skew the results of a particular test to be inaccurate.

Stereophonic audio is not a simple concept. Everything can't be explained in a simple one or two sentence explanation.

This quote, from two Bell Laboratories researchers, should inform any reasonable person that the evaluation of stereophonic audio equipment is not a trivial task. For the last time, I refer you to Harvey, F. K. and Schroeder, M. R., "Subjective Evaluation of Factors Affecting Two-Channel Stereophony", Journal of The Audio Engineering Society, Vol. 9, No. 1, January 1961, pp. 19-28:

DarqueKnight wrote: »

"Critical listeners were sought in these tests because of a desire to set permanent standards. At the moment, only a small percentage of people fully appreciate high fidelity. Even less appreciate or understand stereo. However, there is a growing sophistication evidenced among users of stereo equipment. Anticipating the future, it seemed wise to avoid naive or unconcerned personnel in these tests to prevent establishing loose standards which eventually might have to be abandoned.

The listeners chosen were sophisticated in the art of sound localization either by working in this field or by education before testing. They were felt to be the equal of any serious listener who is accustomed to playing the same records many times and thus becomes familiar with the more subtle artistic and technical effects."

Syndil wrote: »

I'll say it again: Any test that can be performed unblinded can be performed exactly the same blinded. The only difference is the removal of the knowledge of what specifically you are listening to at the time, thus eliminating the potential for bias.

1. Blinding in trials with complex sensory stimuli and untrained subjects can lead to confusion, stress and results skewed by guessing (guessing bias). If you would like to read more on this phenomenon, I recommend the following papers.

A. Pointer, J. S., 1986. Toward the Elimination of Guessing Bias in Landolt Acuity Testing. Am. J. of Optometry and Physiological Optics. Vol. 63, No. 10, p. 813-818.

B. Windschitl, P., Smith, A., Rose, J. P., Krizan, Z., "The desirability bias in predictions: Going optimistic without leaving realism", Organizational Behavior and Human Decision Processes 111 (2010) p. 33?47.

C. Boettcher, K. L., Tenny, R. R., 1985. Distributed Decision Making With Constrained Decision Makers - A Case Study. Laboratory for Information and Decision Systems Case Study LIDS-P-1509, Massachusetts Institute Of Technology, Cambridge, MA.

2. Blinding in trials with complex sensory stimuli and trained subjects is unnecessary and adds an unwarranted level of complexity to experimental procedure. No credible scientist would advocate a standard test methodology that is more complicated than it needs to be.

Syndil wrote: »

Where is all this vehement hatred for blind testing coming from, really?

Hatred? I do not have hatred, vehement or otherwise, toward blind testing. I don't know of any other forum member, that I frequently correspond with, who has a hatred of blind testing. We have consistently said that blind testing is perfectly appropriate for some forms of audio, but not for stereo.

A couple of years ago I submitted an audio evaluation paper to a sensory science journal. The paper included a case study with a number of blind trials with a minimally trained subject. Prior to training, the subject could not perceive distinct images within the sound stage. She also could not perceive any sonic difference when equipment was changed. After brief training in stereophonic sound localization, the subject was able to perceive changes in image localization when equipment was changed. Two attempts to deceive the subject were made when the operator appeared to make equipment changes, but the equipment remained the same. After the first deception attempt, The subject asked:

"What did you do? None of the sounds moved around."

After the second deception attempt, which was the second trial after the first deception attempt, the subject had a quizzical facial expression and again noted that there were no changes in sound stage image location.

I was advised by the journal's review panel, which was composed of sensory science experts, that the blind nature of the trial was unnecessary. I was further advised that blinding with minimally trained subjects could induce guessing as the subject might focus on getting the right answer rather than focus on the sensory stimuli presented. More information on that journal publication can be read here.

In conclusion, I will restate the four points offered previously in post #78:

1. Forced-choice discrimination test methods, such as ABX, A/B, etc., are not suitable for multi-dimensional stimuli. They are more suited for simple discrimination tests where the subject's senses are not affected by a number of simultaneous stimuli. A "dimension" in this case being a characteristic or unit of measurement.

2. A stereophonic sound field is multi-dimensional in nature in that it presents a complex set of simultaneous stimuli to the listener: sound stage height, width and depth; the location and character (clarity, weight, detail, dynamics, etc.) of images in the sound stage, and tactile sensation.

3. The Bell Laboratories scientists who invented home stereo systems were experts in the application of forced-choice discrimination test methods to telephony audio. Such methods, like ABX and A/B, were well suited for discerning simple quality differences in limited bandwidth telephone voice circuits.

4. When these same Bell Laboratories telephony scientists turned their attention to developing home audio equipment, the evaluation methods for such equipment was based on trained listeners who knew how to properly evaluate all the dimensions in stereophonic sound reproduction. Furthermore, peer-reviewed scientific journal literature clearly states that the stereophonic test methods used by Bell Laboratories scientists were intended to be adopted by the consumer.

I have taken some time to address your concerns. I hope you will share some insights regarding your blind testing methods. It is always interesting to read how other people do what they do.

Specifically, how do you conduct an audio equipment trial (procedurally) and what performance metrics do you evaluate?

Thank you for your time and attention and for your patience with regard to my discoursive discussion.

falconcry72

thank you, DK.

DarqueKnight

You're welcome.

DarqueKnight

nwohlford wrote: »

The idea of an unbiased observer is a myth. Biases (or preconceptions) are very basic to human decision making.

I think we should make a distinction between relevant and irrelevant bias. For example. I hate the look of silver finish components. I hate the look of oak. However, when you look inside my two channel audio equipment cabinet, everything is silver. My speakers are clad in loathsome oak:


Two channel system amplification components. The silver is tolerable because I can hide it behind a black
mesh screen door.


The obnoxious oak trim is tolerable because so little of it is visible.

I recently had an opportunity to replace the oak trim on the front speakers in my home theater system discussion here

DarqueKnight wrote: »

The drab, common looking oak end caps and side strips of my SDA SRS's were replaced with curly (tiger) maple end caps and side strips.


Figure 1. Curly maple end caps awaiting installation.


Figure 2. I have never been a fan of any type of oak finish.


Figure 14. Original SRS oak trim ready for new service as fireplace fuel.

The oak trim on my two channel system speakers will be replaced once I decide on a replacement material.

Several years ago, when I was upgrading my two channel equipment (discussion here), I was bitterly disappointed that all the components whose sound I preferred only came in silver color. I was really, really, really betting on black. I was desperate for black to win. I vividly recall the bitter disappointment each time a silver component would win an audio trial.

Even though my bias toward black components is extreme...it did not prevent me from buying higher performing silver components. I don't enjoy the sound of my silver components any less because they are silver...I just don't like looking at them. My bias against oak trim did not prevent me from investing in two pairs of large (64" tall) loudspeakers. I don't enjoy the sound of my oak clad loudspeakers any less because of their appearance. For the loudspeakers whose loathsome oak trim was replaced by exotic curly maple, I don't derive greater listening enjoyment due to their exotic finish. They are simply more pleasing to the eye and they are a better aesthetic compliment to my home's decor.

The point is that my extreme bias for black components was irrelevant. I was able to fairly evaluate an aesthetically undesirable silver component against an aesthetically desirable black component solely on sonic performance.

nwohlford wrote: »

One can try to reduce his/her own biases by being very honest about their biases and in a sense trying to correct for them.

Again, biases are only relevant to the extent that they will prevent a fair and accurate evaluation.

nwohlford wrote: »

I can thinking of training listeners for preforming unblinded studies where a large part of the training would be learning your biases as a listener and learning how to try to minimize your biases.

I agree.

nwohlford wrote: »

I almost never use blinded testing in my own audio comparisons, but I am very careful about making any comments comparing two audio components. If I do state a preference for an audio component, I try to state the limitations of any comparisons made.

Excellent.

nwohlford wrote: »

Realistically, some of the enjoyment we get out of audio has nothing to do with sound. It partly has to do with the look of the components, how owning some brand name makes us feel, and so on.

Some brands market exclusively on snob appeal rather than performance. Then know that knowledgeable buyers will equate high price with high performance. Monster Cable has been very forthcoming in admitting that overpricing their products was part of a strategy of inducing consumers to attribute higher quality to higher prices. Noel Lee says he was surprised that it worked so well...for wire. By the way, I have quite a bit of Monster's products, which I bought at substantial discounts on closeout and on the used market.

However, a true audiophile is not a lover of expensive electronics. A true audiophile is a lover of high quality sound. If I were an audio snob I would not have a house full of Polk Audio loudspeakers.

nwohlford wrote: »

One though has to be carefully when making sweeping comments about whether a certain component will improve somebody else's experience.

I agree. I don't know of anyone this forum who does this. Manufacturers engage in this type of marketing hype, but the consumer has to use some common sense and realize that no one thing is going to be to everyone's benefit.

nwohlford wrote: »

As a statistician, I find a lot of this pretty funny. It is nice to have interests outside your area of expertise, but one must always remember that they are not an expert in that other field.

I don't agree that you need to be an expert in a field before your ideas and contributions can be taken seriously. This is an attitude of snobbery that can cause the dismissal of valuable contributions. If you consult with the U.S. Patent Office, you will find scores of valuable inventions made by people with no expertise in the field of the invention. One of the most famous examples of this is the invention in 1942 of spread spectrum communications technology by Hedy Lamarr (an actress) and George Antheil (a composer and musician).

I have served as a paper reviewer for IEEE and ASEE. I have never, and I don't recall that any of my colleagues have ever, pre-qualified a paper submission by reviewing the expertise or educational credentials of the author(r). The submission was reviewed solely on the basis of technical content and expression.

nwohlford wrote: »

Maybe DarqueKnight has some expertise in the area of testing with human subjects/observers, but I don't see any claim of that (nor evidence in his opinions). This would be like me reading a few electrical engineering journal articles and claiming some expertise. Expertise in any field such testing with human subjects really requires years of concentrated study/work experience.

Your mindset is the type of thinking that fuels ad hominem attacks: if you can't discredit an idea-discredit the person offering the idea. You may find this shocking, but you actually don't have to be a credentialed expert in a field to make a valid contribution. You do however, have to have some knowledge, understanding and reasoning skills.

I have some knowledge and experience in the area of testing human subjects/observers. I have expertise in the design, performance evaluation and manufacturing of electronic equipment.

DarqueKnight wrote: »

A couple of years ago I submitted an audio evaluation paper to a sensory science journal. The paper included a case study with a number of blind trials with a minimally trained subject. Prior to training, the subject could not perceive distinct images within the sound stage. She also could not perceive any sonic difference when equipment was changed. After brief training in stereophonic sound localization, the subject was able to perceive changes in image localization when equipment was changed. Two attempts to deceive the subject were made when the operator appeared to make equipment changes, but the equipment remained the same. After the first deception attempt, The subject asked:

"What did you do? None of the sounds moved around."

After the second deception attempt, which was the second trial after the first deception attempt, the subject had a quizzical facial expression and again noted that there were no changes in sound stage image location.

I was advised by the journal's review panel, which was composed of sensory science experts, that the blind nature of the trial was unnecessary. I was further advised that blinding with minimally trained subjects could induce guessing as the subject might focus on getting the right answer rather than focus on the sensory stimuli presented. More information on that journal publication can be read here.

When I submitted a paper on audio equipment evaluation to the world's premier sensory science journal, the review panel did not check my credentials. If you look at the editorial review board of the journal, you will see that it is staffed by people with international reputations in the field of sensory science. Many of them hold or have held leadership positions on national and international standards setting bodies. Fortunately, these people were not narrow-minded and ignorant to the point that they believed that someone outside their field couldn't possibly have something valuable to contribute.

WilliamM2

DarqueKnight wrote: »

Question: Considering items 1-4, what other factors call into question the audibility of the noise reduction at the speaker cable output? In other words, a 0.002% fluctuation in incoming power line noise will render the speaker cable signal noise content reduction inaudible because ___________. (Please fill in the blank.)

The power line noise ranged from 2% to 2.2% at the time of testing. A difference of .2% (not .002%). The difference between the two power cable measurements was .06%, much smaller than the variation of power line noise. Yes I realise that the measurements were taken at the speaker terminals.

You keep pointing out that Petri-Larmi were able to determine the threshold of audibility as .003% through blind listening tests. So why are blind listening tests suddenly inappropriate for determining the difference of .06% between the two power cables? A difference 20 times greater than the threshold of audibility.

gdb

Ya mean ya don't even like a little oak finish on a nice table wine ???

evhudsons

This thread is absolutely awesome! As I listen to superb Diana Krall from my upgraded within the year system due to this forum, I am not only learning and being entertained, I am listening to my own observations of cables.

One thing I have learned, that frustration at not understanding will be vented in hostility by some people, but vented at continued questions by others. It's at the point of general hostility without further questions or answers the conversation is not productive. As long as this thread continues as it is, I'll be reading. There is a lot here.

I agree about silver finish equipment, I prefer my boxes black. But I like Oak, as well as most other woods. I don't like plastic or painted wood.

Syndil

DarqueKnight wrote: »

1. Blinding in trials with complex sensory stimuli and untrained subjects can lead to confusion, stress and results skewed by guessing (guessing bias). If you would like to read more on this phenomenon, I recommend the following papers.

2. Blinding in trials with complex sensory stimuli and trained subjects is unnecessary and adds an unwarranted level of complexity to experimental procedure. No credible scientist would advocate a standard test methodology that is more complicated than it needs to be.

If a potential for bias exists, then the conclusion that the blinding is unnecessary is not a foregone one. And I would submit that the potential for bias nearly always exists, and it seems our resident statistics expert nwohlford agrees. Frustratingly complex or not, if the goal is to produce results free of bias, the complexity must be dealt with. If guessing is the concern, then the randomness of the guessing is easily discovered by spreading the comparison over numerous trials and looking for a result of 80% or better, or whatever standard you adhere to that would indicate a non-random result.

I was advised by the journal's review panel, which was composed of sensory science experts, that the blind nature of the trial was unnecessary. I was further advised that blinding with minimally trained subjects could induce guessing as the subject might focus on getting the right answer rather than focus on the sensory stimuli presented. More information on that journal publication can be read here.

In this case, the type of trial being conducted has nothing in common with the type of XY comparison an audiophile would conduct in order to rank one piece of equipment versus another. A blind trial may very well be unnecessary for an untrained listener trying to determine localization of images within a sound stage, but dismissing blind testing for XY comparisons based on this is a non sequitur. Apples to oranges--the logic does not follow.

In conclusion, I will restate the four points offered previously in post #78:

1. Forced-choice discrimination test methods, such as ABX, A/B, etc., are not suitable for multi-dimensional stimuli. They are more suited for simple discrimination tests where the subject's senses are not affected by a number of simultaneous stimuli. A "dimension" in this case being a characteristic or unit of measurement.

2. A stereophonic sound field is multi-dimensional in nature in that it presents a complex set of simultaneous stimuli to the listener: sound stage height, width and depth; the location and character (clarity, weight, detail, dynamics, etc.) of images in the sound stage, and tactile sensation.

I have to chuckle every time I see this word "multi-dimensional." I believe you first used it to describe food, which is not a correct usage of the term, but I understood the implication, which is actually multi-sensory. If something is multi-dimensional, it would take quantum mechanics to prove it. Anything outside our current dimension is essentially outside of our current realm of existence and unable to be observed directly.

But given the intended meaning, it is still a stretch. We do not need our sense of taste, sight or smell in order to listen to music. One can feel loud and low sound waves as they pass through your body, but this is not always present in audio, and one could argue that it is a secondary characteristic: nice to have, but not something that is necessary for the enjoyment of the music. Some may even find it detrimental to the experience or distracting. The enjoyment of audio is completely subjective, after all.

The perceived location of the origin of the sound as well as its character are all direct products of a single stimuli: sound waves. One does not need to view the woofers moving or the VU meters dancing or the graphic equalizers pulsing or what have you in order to enjoy the sound. Those other stimuli may add or subtract from the experience as a whole depending on your tastes, but when it comes to judging the quality of the audio they have no bearing, and can only serve to distract or perhaps even create bias, i.e. "those VU meters look cool, so I want to like this piece of gear more than the one it is being compared to." And bias is the enemy.

4. When these same Bell Laboratories telephony scientists turned their attention to developing home audio equipment, the evaluation methods for such equipment was based on trained listeners who knew how to properly evaluate all the dimensions in stereophonic sound reproduction. Furthermore, peer-reviewed scientific journal literature clearly states that the stereophonic test methods used by Bell Laboratories scientists were intended to be adopted by the consumer.

That's all well and good, but the scientists at Bell Labs are not the end-all be-all authority on hi-fi audio, despite your heavy reliance on their work, as you seem to refer back to them constantly. Regardless, my initial question has still not been answered. No one can tell me why a blinded test (appropriately conducted over several trials to eliminate randomness from guessing) should be inaccurate. Going back to my original example (trying to keep it simple), why should comparing cable X versus cable Y produce any different results from comparing a Pear Audio cable versus a Monoprice cable, if the only difference between the comparison methods is the knowledge or lack thereof of the specific gear being tested at the time? I submit that it is indeed a forgone conclusion that, unless bias towards one of the brands is present in the unblinded test, the tests should produce exactly the same results. And if that is indeed the case, then it is also logical to conclude that XY testing eliminates potential bias from unblinded testing. And if that is the case, then I am left to questioning the motivation for preference against blind testing, which would seem to me to be an admission of a preference towards bias.

I have taken some time to address your concerns. I hope you will share some insights regarding your blind testing methods. It is always interesting to read how other people do what they do.

Specifically, how do you conduct an audio equipment trial (procedurally) and what performance metrics do you evaluate?

I do a simple XY comparison with the help of a friend. They change the gear being tested, and I don't know which is which when I am listening. I am a willingly blind participant, so it's easy to accomplish the blinding without much effort. Sometimes, as in my latest pre/pro comparison, all it takes is covering the face of the equipment with poster board so that I am unable to see the lights from whichever is turned on. When you want to be blinded to what is being tested, it is very easy to accomplish. If you want to cheat, you are wanting to introduce bias.

Metrics? I am not using laboratory gear when judging audio gear, I am using my perception and my brain. It is a subjective comparison, so the only "metric" I have to determine is which one I like the sound of better. I may occasionally make notes about the differing characteristics of the sound if the comparison is close, but ultimately I prefer the deciding factor to be which sound is able to draw out the most emotion while I listen.

DarqueKnight

WilliamM2 wrote: »

The power line noise ranged from 2% to 2.2% at the time of testing. A difference of .2% (not .002%).

Correct. Thanks. You appear to be much better in math than reading. We all have our strengths and weaknesses. Is 0.2% variation in power line THD significant? If so why? If not, why not?

WilliamM2 wrote: »

The difference between the two power cable measurements was .06%, much smaller than the variation of power line noise.

Recall that the incoming noise content was up to 2.2%. Outgoing noise content was 0.16%. This is a 13.75x reduction in noise. In a scenario where the outbound noise is 13.75x less than the inbound noise, how significant is the incoming power line noise variation going to be on the audibility of noise reduction at the speaker cable output? Feel free to use math in your answer.

WilliamM2 wrote: »

Yes I realise that the measurements were taken at the speaker terminals.

Great.

WilliamM2 wrote: »

You keep pointing out that Petri-Larmi were able to determine the threshold of audibility as .003% through blind listening tests. So why are blind listening tests suddenly inappropriate for determining the difference of .06% between the two power cables? A difference 20 times greater than the threshold of audibility.

I wouldn't have to keep pointing it out if you would "get it". I already covered this here.

One thing to consider is that the Petri-Larmi study was a study in simple SIGNAL DISCRIMINATION. They were comparing signals corrupted with various levels of noise. They were not evaluating audible differences in similar types of AUDIO EQUIPMENT. Therefore, since Petri-Larmi was testing the discrimination ability of subjects, a discrimination test like A/B was appropriate.

Also worth considering is the fact that, before the Petri-Larmi A/B tests were conducted, much ground work was done to establish control factors. Listener hearing capability, listener experience with stereo sound, listener seating position, the resolution quality of equipment and length of trials were all appropriate to stereophonic testing. The Petri-Larmi study didn't put a group of random people in a room and start swapping gear in a low resolution system.

It is difficult to discuss these things with people who refuse to take the time to study and understand the basic principles of stereophony. I guarantee you that once you take the time to understand those principles, everything else falls into place.

Do you understand the difference between

1. Listening to discern a gross difference between two music samples (TWO SIGNALS), particularly when you know the nature of the potential difference (i.e. contamination with noise) in the signals

and

2. Listening to the entire stereophonic presentation presented by TWO PIECES OF EQUIPMENT and discerning the difference(s)?

Petri-Larmi was not concerned with having subjects evaluate stereophonic performance. Petri-Larmi asked one simple question:
"Here are two music samples. Is there a difference (change in tonal character) between them? There were five music samples ranging in length from 45 seconds to 1 minute-45 seconds. Subjects were allowed to play samples from 15 to 30 minutes. Subjects were also trained in the recognition of the particular type of distortion added to the passages. However, there was no evaluation of stereophonic performance. There was no deconstructing of the sound stage into into individual parts. Even though Petri-Larmi did not evaluate sterephonic performance, the results are important because valuable insight was provided into the threshold of noise detection in a stereo system.

A stereophonic evaluation of two components seeks answers to multiple questions such as:

1. Is there a difference in sound stage dimensions (height, weight, depth)?
2. Is there a difference in image placement within the sound stage?
3. Is there a difference in the tonal character of images?
4. Is there a difference in the apparent "weight" of images?
5. Is there a difference in clarity of images?
6. Is there a difference in dynamic range?
7. Is there a difference in the pace of music?
8. Is there a difference in clarity, dynamics, image weight among different parts of the sound stage?
9. Is there an overall difference in tactile sensation?
10. Is there a difference in tactile sensation from various parts of the sound stage?

Items 1-10 comprise just a partial list of the performance parameters I evaluate. Some people may feel the need "blind" themselves. I do not. The people who invented stereo intended for the individual to learn to evaluate stereo equipment on their on...without the need for assistance from second, third, or more parties in setting up blind trials. Either the sound stage is wider/deeper/taller or it is not. Either I hear more details and clarity or I do not.

Sometimes I will blind trial another trusted set of trained ears out of curiosity to see if the other person hears the same things at the same intensity as me. Sometimes I will blind trial a set of untrained ears...particularly if the differences I perceive are subtle.

New AC Regenerators - New Power Quality

The PS Audio P10 AC regenerator continuosly logs power quality metrics and transmits them over the Internet to PS Audio web servers. I can log into my account and monitor power, incoming/outgoing voltage and incoming/outgoing THD. I can also remotely control the P10, including turning it on and off.


Figure 1. This is a screen shot of the power monitoring stats from 4-19-12.


Figure 2. Incoming/Outgoing THD from 12 am Thursday 4-19-12 to 12 am Friday 4-20-12.

The power company's new transmission facilities have brought significant reductions (by half) in power line THD from a weekday average of 2.4% down to 1.2%. Whereas I used to see time of day variations in THD, it is now fairly consistent over a 24 hour period. The range of fluctuation is still in the neighborhood of 0.2%

DarqueKnight

Syndil wrote: »

I have to chuckle every time I see this word "multi-dimensional." I believe you first used it to describe food, which is not a correct usage of the term, but I understood the implication, which is actually multi-sensory. If something is multi-dimensional, it would take quantum mechanics to prove it. Anything outside our current dimension is essentially outside of our current realm of existence and unable to be observed directly.

I was not aware that the word dimensional could only be used within the context of quantum mechanics. Thanks for the clarification.

The term multi-dimensional is quoted from one of the standard textbooks in sensory evaluation of food that was authored by Dr. Harry Lawless and Dr. Hildegarde Heymann. I provided the citation in post #47. I don't know Dr. Heymann, but I have corresponded several times with Dr. Lawless. I'm sure he would appreciate you dropping him an email informing him of the required correction in his textbook. His contact info can easily be found on the Internet. Be sure to give him my regards.

Syndil wrote: »

That's all well and good, but the scientists at Bell Labs are not the end-all be-all authority on hi-fi audio, despite your heavy reliance on their work, as you seem to refer back to them constantly.

Syndil wrote: »

Metrics? I am not using laboratory gear when judging audio gear, I am using my perception and my brain. It is a subjective comparison, so the only "metric" I have to determine is which one I like the sound of better.

OK, I have a good understanding now.

1. Stereo sound can't be multi-dimensional because then it would be outside the current realm of existence.

2. The word dimensional can only be used within the context of quantum mechanics.

3. When two world renowned experts in the sensory evaluation of food used the term "multi-dimensional" with respect to food, they were incorrect, because the word "dimensional" only applies to quantum mechanics.

4. It is not appropriate to continually refer back to the teachings of Dr. Harvey Fletcher and his Bell Labs colleagues, who invented home stereo systems, because they are not the end-all be-all authority on hi-fi.

5. The only way to evaluate home audio equipment with "metrics" is with the use of laboratory gear.

6. Stereo equipment can only be properly evaluated with outside help, because it absolutely has to be done blind.

Thanks for everything. That was deluxe.

Syndil

Now you're just nitpicking semantics and being insincere, failing to seriously address any of my points. Throwing in the towel, I see. Guess we are done here!

BeefJerky

DMara wrote: »

This kind of thread really takes the fun out of listening to music :biggrin:, unfortunately it happens all the time in our Clubhouse. There'll be no winner, just animosity.

DarqueKnight wrote: »

The smiley face indicates you might be joking, but in case you aren't, why you would say that? I am enjoying music from my stereo as I write this.

I certainly don't have any animosity toward anyone because they disagree with my approach to audio. Nothing anyone else says can diminish the daily joy I receive from listening to a high resolution stereo system.

If there is any animosity, I would expect it would come from the scores of people who express dissatisfaction with their stereo systems by demanding proof that other people can hear certain things.

Exactly. I'm usually listening to music when reading or responding to threads such as this (or just on the computer in general). In fact, I'm enjoying some fantastic music as I'm writing this reply. I also don't hold any animosity towards anyone here, nor do I ever see that happening.

halo71 wrote: »

Yep, but the difference is a football game eventually comes to an end. :cheesygrin: Arguing over cables over and over, saying the same thing over and over trying to change ones mind but getting no where? Thats what I was getting at. If you can not at least get someone to try something new, why continue trying to convince them there is a difference.

Okay, let's equate it to the Harlem Globe Trotters then. It's not about winning or losing, it's about entertainment and pure awesomeness! I thoroughly enjoy these types of threads, whether I'm simply reading them or whether I'm participating in them.

headrott

Syndil wrote: »

Now you're just nitpicking semantics and being insincere, failing to seriously address any of my points. Throwing in the towel, I see. Guess we are done here!

As I said in my last post, you cannot see what is presented right in front of your face. DK directly addressed your points and you (again) pointed out you didn't feel anyone has addressed your "questions". You have not said you weren't satisfied with the answer(s). You have stated that no one has even addressed them. That is pure and utter BS. Open your eyes and see what is right in front of you (multiple times even). It's no wonder Ray is being insincere! It's simply because you are as well, IMO. You are completely dismissing his responses and acting like they were never written. That is true of everyone that has posted an answer to your "question(s)".

Also, this has less to do with stereophonic audio but I wanted to point out a response to this statement you said:

Syndil wrote: »

I have to chuckle every time I see this word "multi-dimensional." I believe you first used it to describe food, which is not a correct usage of the term, but I understood the implication, which is actually multi-sensory. If something is multi-dimensional, it would take quantum mechanics to prove it. Anything outside our current dimension is essentially outside of our current realm of existence and unable to be observed directly.

I disagree that anything outside of our current dimension can be observed directly. If you refine and utilize your consiousness, you can absolutely directly observe things outside of our dimension. I will give you that using electronic equipment it cannot be done, however.

Greg

Systems

I have never commented on a cable thread, or anything similar.
What I have found is when you have had the same EXACT system for over 3 years, you can tell even a miniscule amount of change.
Digital, IC's, PC's, whatever.
I have been listening to the same system, everday for over 3 years.
CDP, PRE, AMP, POWER CONDITIONER, ECT.
The same EXACT SYSTEM!
You can change something, anything, I can tell a difference, regardless.
Even in the room, I can tell a difference.
3 years, same system!
I may have an advantage in that regard.
But that said, I would never try to force my thoughts on someone else.
Knowing ALL you guys, I don't think that is what any of you are trying to do.
Simple discussion, simple facts.
Isn't that why I am a member here?
Discussions about audio?
No matter how technical?
Thats why I come here, I can assure you!

tonyb

DK has gone above and beyond in referencing his position. He has an infinite amount of patience, more so than me anyway. Yet his responses are exactly why most don't participate anymore in these types of discussions. Learning, in itself is a lost art for some.

That muti-dimensional thing.....in my day, we called it LSD. :cheesygrin:

BlueFox

Syndil wrote: »

Now you're just nitpicking semantics and being insincere, failing to seriously address any of my points. Throwing in the towel, I see. Guess we are done here!

This definitely deserves a ROFLMAO award. I guess the words, or ideas, were to big to grasp, so the semantics are off.