Stupid things you hear from the DBT/Null test crowd....

Syndil

heiney9 wrote: »

You and others have stated the only way for a proper test to be done is blind, now that DK has acquiesced that blind testing can be done if one chooses, but it's not necessary. You agree that you can obtain the exact same results with both methods.

Holy crap, people. We have a lack of reading comprehension--you totally misinterpreted what I said. Go back and read it again. I haven't changed it. You know that I haven't changed it, because on this forum you can't edit posts after so many minutes.

What I said was that both methods would produce the same results only if bias was not present in the unblinded test. That is not a minor caveat to be glossed over, it is the crux of the issue in its entirety. And yet you are somehow taking that statement as a validation of sighted testing, as long as bias is somehow removed? Guess how you remove the bias. Go on.

DarqueKnight wrote: »

I have laid out my evaluative process in detail. If you choose to take from that something diametrically opposite to what I wrote, that is fine.

Your going to have to clarify on that one. It almost sounds like you disagree, which would be backpedaling, considering what you said here:

Is it "wrong" to hide the identity of components when doing a stereophonic evaluation? Of course not. Just like it is not "wrong" to use bicycle training wheels after a child has learned to keep balance. But why would you?

The question is often asked "if an expert listener can tell a difference while knowing the identity of components, why can't they do the same if the component's identities are hidden?" The answer is they can.

This says to me that yes, you do agree that a blind test will indeed produce the same results as an unbiased sighted test, assuming that by "expert listener," you mean someone who is somehow above bias.

DarqueKnight wrote: »

This statement and post #175 seem to indicate that you act like you won't be able to enjoy your emotion driven evaluative process unless I agree with you.

You have been pulling stuff way out of left field for a while now, but this takes the cake when it comes to sheer arrogance. Your agreement or disagreement has zero bearing on my enjoyment of anything. I am only interested in your agreement/disagreement in as much as it pertains to what my next point will be.

And this... Really takes the cake.

DarqueKnight wrote: »

He is desperate to receive acceptance for his emotionally based stereo evaluation procedure.

Both if you have made an error in comprehension and have jumped all over my statement that blind testing will produce the same results as an unbiased unblinded test like some sort of crack fiend. I think it is clear who is desperate here, and it isn't me. My statement was not even slightly a validation of sighted testing. The way you remove bias from a sighted test is to make it a blinded test.

Seriously, it is downright comical how quickly you both pounced on that and just ran with it, all the while completely oblivious to the point that was made. This isn't going to be fun any more if you guys aren't going to pay attention. "I refuse to engage in a battle of wits with an unarmed man."

Notice how he dismissed as "irrelevant" the evidence that blind trials are not used in television. Why is this irrelevant? If bias can be in hearing can't it also be in seeing? I would think even moreso since humans are much more influenced by what they see than what they hear!!!

Surely someone can explain why blinding for audio is required but blinding for television is not.

Glad you asked. Video can be compared and evaluated without relying on that most unreliable part of the human psyche, memory. Audio, however, cannot. They have very little in common. Is this not obvious?

Now, moving on to the scant few relevant points that were made... [next post]

headrott

It is safe to say that the reason DBT/ABX, etc. test proponents need audiophiles to take part in their blind tests in order to disqualify or validate their listening abilities and (in their minds) take away the fears they have of being duped by high end companies selling snake-oil. But, what proponents of the DBT/ABX, etc. tests fail to realize is that by other people (audiophiles) taking part in these tests, it will not relieve their fears of being duped even if audiophiles "fail" a DBT test and can't tell a difference in cables. They only way these proponents will relieve their fears is by allowing themselves to learn how to listen for differences themselves. It is their responsibility and duty to do this. Only the person with a fear can relieve that fear, by learning, understanding and practice.

Syndil

heiney9 wrote: »

I'm sure now you will twist something you've said around to point to the fact that all along you were in favor of sighted testing being the same as blind testing as long as the bias is removed or some such string of thoughts and points you never brought up specifically.

Not at all. The way you remove bias from sighted testing is to make it blind testing.

DarqueKnight wrote: »

I have consistently held the position that, for the untrained listener, blinding can lead to inaccurate results and that for the trained listener, blinding is unneccessary.

And this is where myself and science wholeheartedly disagree.

http://www.harmanaudio.com/all_about_audio/audio_art_science.pdf

http://seanolive.blogspot.com/2009/04/dishonesty-of-sighted-audio-product.html

http://www.acourate.com/Download/BiasesInModernAudioQualityListeningTests.pdf

Knowledge of the products that are being evaluated is generally understood to be a powerful source of psychological bias. In scientific tests of many kinds, and even in wine tasting, considerable effort is expended to ensure the anonymity of the devices or substances being subjectively evaluated. In audio, though, things are more relaxed, and otherwise serious people persist in the belief that they can ignore such factors as price, size, brand, etc. In some of the ?great debate? issues, like amplifiers, wires, and the like, there are assertions that disguising the product identity prevents listeners from hearing differences that are in the range of extremely small to inaudible. That debate shows no signs of slowing down. In the category of loudspeakers and rooms, however, there is no doubt that differences exist and are clearly audible. To satisfy ourselves that the additional rigor was necessary, we tested the ability of some of our trusted listeners to maintain objectivity in the face of visible information about the products.

The results are very clear, and strongly supportive of the scientific view. Figure 4 shows that, in subjective ratings of four loudspeakers, the differences in ratings caused by knowledge of the products is as large or larger than those attributable to the differences in sound alone. The two left-hand striped bars are scores for loudspeakers that were large, expensive and impressive looking, the third bar is the score for a well designed, small, inexpensive, plastic three-piece system. The right-hand bar represents a moderately expensive product from a competitor that had been highly rated by respected reviewers.

When listeners entered the room for the sighted tests, their positive verbal reactions to the big beautiful speakers and the jeers for the tiny sub/sat system foreshadowed dramatic ratings shifts - in opposite directions. The handsome competitor?s system got a higher rating; so much for employee loyalty. Other variables were also tested, and the results indicated that, in the sighted tests, listeners substantially ignored large differences in sound quality attributable to position in the listening room and to program material. In other words, knowledge of the product identity was at least as important a factor in the tests as the principal acoustical factors. Incidentally, many of these listeners were very experienced and, some of them thought, able to ignore the visually-stimulated biases.

3.1 Biases Due to Appearance, Branding, Expectation, and Personal Preference
In general listeners? affective judgments can be biased by the appearance of the equipment, price, and branding. For example, Toole and Olive demonstrated that in preference tests (affective judgments) both experienced and inexperienced listeners were biased by the appearance and the brand names of the loudspeakers evaluated [38]. Then the scores from the listening tests were averaged across the listeners it was found that the results were different depending on whether the participants could see the loudspeakers or not. The maximum observed difference equaled 1.2 points (loudspeaker D, location 1) on a scale ranging from 0 to 10, which constitutes 12% of the range of the scale. This study is often quoted as a classical example of how important it is to undertake blind listening tests in order to reduce nonacoustic bias.

In summary, the sighted and blind loudspeaker listening tests in this study produced significantly different sound quality ratings. The psychological biases in the sighted tests were sufficiently strong that listeners were largely unresponsive to real changes in sound quality caused by acoustical interactions between the loudspeaker, its position in the room, and the program material. In other words, if you want to obtain an accurate and reliable measure of how the audio product truly sounds, the listening test must be done blind. It?s time the audio industry grow up and acknowledge this fact, if it wants to retain the trust and respect of consumers. It may already be too late according to Stereophile magazine founder, Gordon Holt, who lamented in a recent interview:

?Audio as a hobby is dying, largely by its own hand. As far as the real world is concerned, high-end audio lost its credibility during the 1980s, when it flatly refused to submit to the kind of basic honesty controls (double-blind testing, for example) that had legitimized every other serious scientific endeavor since Pascal. [This refusal] is a source of endless derisive amusement among rational people and of perpetual embarrassment for me..?

heiney9

Syndil wrote: »

The way you remove bias from a sighted test is to make it a blinded test.

This has got to be the most circular statement I have ever read? Please, please, please explain to me how you can have a sighted test if you are blinded? Please explain to me how that is possible. This is the kind of inane logic that you keep predicating your argument on. That sentence makes absolutely no sense at all, in fact it's not possible.

I refuse to debate anything with someone who uses that kind of logic to bolster their argument.

H9

heiney9

Syndil wrote: »

Holy crap, people. We have a lack of reading comprehension--you totally misinterpreted what I said. Go back and read it again. I haven't changed it. You know that I haven't changed it, because on this forum you can't edit posts after so many minutes.

I guess that means you are too lazy to show me or you can't. How is me going back and rereading going to change my interpretation of it? I asked you to show me specifically so as for you to clarify.

What I said was that both methods would produce the same results only if bias was not present in the unblinded test.

I asked you specifically not to paraphrase, that's not at all what I got from your posts. I don't recall you stating that at all. Please show me specifically where you said that. Either you can point to it or you can't.

H9

Syndil

OK lazy, here it is. http://www.polkaudio.com/forums/showthread.php?131718-Stupid-things-you-hear-from-the-DBT-Null-test-crowd....&p=1759805&viewfull=1#post1759805

Syndil wrote: »

So we agree that there is nothing wrong with blind testing; a properly conducted blind test of stereophonic equipment will produce the same results as an unblinded test--if the unblinded test is free of bias. Correct?

Which leads me to my next point...

heiney9 wrote: »

This has got to be the most circular statement I have ever read? Please, please, please explain to me how you can have a sighted test if you are blinded? Please explain to me how that is possible.

...which is that it isn't possible. Wow. I had no idea I was being that subtle. I thought for sure DK and company would see what my intention was with that set up and post some more chaff or some other type of dodge. Was not expecting to feel embarrassed for you guys. :redface:

Syndil

More on blind testing:

http://www.stekkie.com/?p=916

A very well-written article worth reading, regardless of what camp you're currently in.

heiney9

Syndil wrote: »

OK lazy, here it is. http://www.polkaudio.com/forums/showthread.php?131718-Stupid-things-you-hear-from-the-DBT-Null-test-crowd....&p=1759805&viewfull=1#post1759805

No, I meant in previous posts when you started the discussion, not where I believe you stated it for the first time, which would be the link you provided above from today a little before lunchtime. You stated you have been saying that all along, I asked you to show me instances of where you had been saying that all along, which would mean prior to stating it today at approx 11AM. All along means since the beginning of the discussion which has now taken place over several days.

H9

Jstas

Don't you people have anything better to do with your time?

Syndil

Now you're really grasping. Was there a cutoff time for making valid points that I missed or something? What possible difference does it make when I first posted my point using that particular phrasing? That was the exact quote that both you and DK decided was somehow my validation of sighted testing.

DarqueKnight

Oh Syndil, your posts keep getting longer and longer. I am appalled that you have so quickly abandoned the teachings of Einstein.

Syndil wrote: »

I am beginning to think that you don't understand this oft-quoted journal paper either, since all you are able to do it point to it and say that the answer lies within it somewhere. If you do understand how this paper proves that blind testing produces inaccurate results, then explain it to me simply.

Syndil wrote: »

We have a lack of reading comprehension--you totally misinterpreted what I said.

I would not be so quick to accuse others of lack of reading comprehension. Question: There is a huge difference in my statement and Sean Olive's statement:

Me:

DarqueKnight wrote: »

I have consistently held the position that, for the untrained listener, blinding can lead to inaccurate results and that for the trained listener, blinding is unneccessary.

Sean Olive

In other words, knowledge of the product identity was at least as important a factor in the tests as the principal acoustical factors. Incidentally, many of these listeners were very experienced and, some of them thought, able to ignore the visually-stimulated biases.

For example, Toole and Olive demonstrated that in preference tests (affective judgments) both experienced and inexperienced listeners were biased by the appearance and the brand names of the loudspeakers evaluated.

My statement concerns TRAINING. Olive's statements concern EXPERIENCE. There is a HUGE difference between training and experience.

Experience merely means active participation in an activity. Experience does not automatically equate to knowledge and skill. One can have 50 years of experience doing something yet have no expertise or true knowledge because the task may have been done incorrectly all along.

Training means to make proficient with specialized instruction and practice. This is what I, and the stereophonic pioneers, have always advocated. In the context of stereo evaluation, training means specialized instruction in sound localization and sound character description.

It is not clear from the quote whether G. Gordon Holt was talking about trained or untrained listeners.

Syndil wrote: »

Glad you asked. Video can be compared and evaluated without relying on that most unreliable part of the human psyche, memory. Audio, however, cannot. They have very little in common. Is this not obvious?

No, it is not obvious and it is not correct.

Tell me, how is it that video can be compared without relying on memory? Do you think that because two or more television screens are side by side, memory is not required to compare them? What if there are multiple television screens such as the arrangements shown in post #185? Since human eyes are the primary sense organs, short term visual memory is very good.

According to a study done by Michael A. Cohen, Todd S. Horowitz, and Jeremy M. Wolfe, short term auditory memory is markedly inferior to short term visual memory ["Auditory Recognition Memory is Inferior to Visual Recognition Memory", Proceedings of the National Academy of Sciences of the U. S. A., April 7, 2009; 106(14), [URL="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2667065/"]link[/URL].

Since short term auditory memory is so poor, heavy reliance on it in an evaluative situation often leads to inaccurate results, particularly if the auditory stimuli are complex in nature and particularly of the stimuli are short in length. This is one of the reasons why I am a vehement advocate of complete and thorough written documentation of the sonic characteristics of the sound stage in order to diminish the reliance on short term auditory memory. Long term auditory memory can be very good, as in the case of memorizing the characteristics of favorite songs.

I have specifically stated that I am often not aware of performance differences between two components until I compare sound stage maps and listening notes.

I'm glad you have found a stereo evaluation method that works for you. Whether or not it is scientifically justified is irrelevant if the results are satisfactory. One thing I would like to call your attention to is a comment in the "On Some Biases Encountered in Modern Audio Quality Listening Tests?A Review" paper, which you linked to:

"The ability to distinguish between different attributes of sound character varies across listeners. However, there is strong evidence that this ability can be developed by means of systematic training." [p. 431, col. 2, par. 4]

steveinaz

Meanwhile, I'm totally enthralled in the music, with my placebo induced cable purchases......................

DarqueKnight

I recently replaced all (8 fuses per amp) the stock fuses in my JC 1 monoblocks with audio grade fuses.

Jstas

steveinaz wrote: »

Meanwhile, I'm totally enthralled in the music, with my placebo induced cable purchases......................

Meanwhile, I'm totally enthralled by the bottom of my beer glass...wondering why it's empty.

BlueFox

DarqueKnight wrote: »

I recently replaced all (8 fuses per amp) the stock fuses in my JC 1 monoblocks with audio grade fuses.

I have not done that yet, but have been thinking about it. Back when I used to work on electronics the fuse holders on a circuit board were the length of the fuse with a U shaped connection at each end, and the fuse was pressed into it. They were pretty cheap, flimsy things, but worked. Do the amps have these flimsy holders, or are there upgrade fuse holders one can solder into the board?

DarqueKnight

I was able to catch some HiFi Tuning gold fuses on closeout at 45% off at Parts Connection and 20% off at Music Direct.

The seven internal fuses have the U shapped holders. The four fuses in the power supply were held tightly in their holders and the fuses had to be pried up using a length of coat hanger wire with a 1/4" bend at the end. The three fuses in the protection circuit fit snugly in their holders, but I could pull them out with needlenose pliers.

I don't know if there are "upgrade" fuse holders available. I have bought regular fuse holders that had a snug fit.

mdaudioguy

When will it end?? How will it end???????? Who will still be standing in the end?


The suspense is killing me. :rolleyes:

headrott

mdaudioguy wrote: »

When will it end?? How will it end???????? Who will still be standing in the end?


The suspense is killing me. :rolleyes:

I believe this was ended a while back. It's just one of the parties hasn't realized it yet.............

Greg

tonyb

steveinaz wrote: »

I believe there are people that can make eating a banana split a complex activity.

Ice cream ? Did someone say Ice cream ?? AHH...my other passion.

We could use the Good Humor truck in this thread.

Syndil

headrott wrote: »

I believe this was ended a while back. It's just one of the parties hasn't realized it yet.............

QFT.

I would not be so quick to accuse others of lack of reading comprehension. Question: There is a huge difference in my statement and Sean Olive's statement...

Actually that was not Sean Olive's statement, that was Floyd Toole writing about one of Sean Olive's experiments. I've noticed you are hanging your entire argument on that difference, training vs. experience. Tell me, is it possible that the word "experienced" might have been used for brevity, but to include the meaning of "trained" as well? If you are trained, are you not experienced? Given some of the other work Mr. Olive has done:

A Method for Training Listeners and Selecting Program Material for Listening Tests

Differences in Performance and Preference of Trained Versus Untrained Listeners in Loudspeaker Tests: A Case Study

It would seem counterintuitive to me that Mr. Olive would have thrown out that work and used untrained but experienced listeners. Regardless, if you read what Mr. Olive wrote himself, you would have found that Mr. Olive did say that they were indeed trained listeners:

"This question was tested in 1994, shortly after I joined Harman International as Manager of Subjective Evaluation [1]. My mission was to introduce formalized, double-blind product testing at Harman. To my surprise, this mandate met rather strong opposition from some of the more entrenched marketing, sales and engineering staff who felt that, as trained audio professionals, they were immune from the influence of sighted biases. Unfortunately, at that time there were no published scientific studies in the audio literature to either support or refute their claims, so a listening experiment was designed to directly test this hypothesis. The details of this test are described in references 1 and 2."

Moving on.

No, it is not obvious and it is not correct.

Tell me, how is it that video can be compared without relying on memory?

For the most part, you've answered your own question with the Harvard Medical School study comparing auditory vs. visual memory and the fact that yes, you can compare multiple TV screens at once. Combine that with the knowledge that even trained humans make for very poor measuring intruments, but can be very good at comparing stimuli [http://www.acourate.com/Download/BiasesInModernAudioQualityListeningTests.pdf], and it should become obvious.

I'm glad you have found a stereo evaluation method that works for you. Whether or not it is scientifically justified is irrelevant if the results are satisfactory. One thing I would like to call your attention to is a comment in the "On Some Biases Encountered in Modern Audio Quality Listening Tests?A Review" paper, which you linked to:

"The ability to distinguish between different attributes of sound character varies across listeners. However, there is strong evidence that this ability can be developed by means of systematic training." [p. 431, col. 2, par. 4]

I agree that the ability to describe sound characteristics can be trained. That is not the issue. The issue is and has always been the presence of bias. As you yourself said,

Is it "wrong" to hide the identity of components when doing a stereophonic evaluation? Of course not. ... But why would you?

The answer to that question is the answer to this entire debate. And the answer is bias.

Science has shown time and time again that the myth of the unbiased observer is just that: a myth, and that no amount of training can eliminate that bias, regardless of how convinced you may be that you are above it. The way to eliminate bias is through methodological controls governing the way the experiment is conducted, not by "training it out" of the participants. Your entire argument rests on the supposition that it is indeed possible to do so. Many audiophiles tend to agree with that sentiment, and in fact the entire audiophile market lives and dies by that sentiment. However in the world of testable reality, you are all on an island, alone. Or perhaps it's not an island, but a compound in the South American jungle...

markmarc

Last fall I was at RMAF, and Sean Olive was giving 40 minute presentations on his work at Harman International on training listeners for greater discrimination between speakers, both sighted and blind. The reason for both he pointed out was the tremendous value that both testing methodologies present. In fact, from an audience member's questions he recognized the stress problem of the listener in blind tests. With all the years he's spent in this field, that statement tells me a great deal. The large, inherit flaw in blind testing stereo gear is the tricks the mind plays. To me that is an uncontrollable variable.

As a teacher I support DK's effort to create his sound field mapping technique as a step beyond the typical audio test as it "asks or demands" the listener to be specific about the stereophonic experience. It's comparable to giving a one sentence versus a a five paragraph essay answer.

DarqueKnight

Syndil,

You believe that blinding is required to remove bias and that stereo audio trials must be done blind.

I showed that television trials are done non-blind.

I asked:

Surely someone can explain why blinding for audio is required but blinding for television is not.

You replied:

Syndil wrote: »

Glad you asked. Video can be compared and evaluated without relying on that most unreliable part of the human psyche, memory. Audio, however, cannot. They have very little in common. Is this not obvious?

Now, moving on to the scant few relevant points that were made... [next post]

Your "bum's rush" style answer implied that my question was irrelevant, but it was relevant..it goes to the central idea of evaluator bias. You did not answer within the context of evaluator bias.

Perhaps I didn't state my question clearly enough, so I will restate it:

If blinding in required to remove evaluator bias in stereo equipment evaluations, why is blinding not required to remove bias in television evaluations?

You went off on a tangent with your comment about visual memory vs. auditory memory. You said video can be compared and evaluated without relying on memory. How does the non reliance on memory in video trails, if true, eliminate bias?

It seems that the central point of the blind test for stereo argument is that an evaluator can be biased by such things as aesthetics, brand name, and price. Television evaluators are aware of all these things when evaluating televisions. So again, the discrepancy I find is that:

Apparently, there is no concern that television evaluators will be biased by the knowledge of aesthetics, brand name and price, hence the absence of blind trials for televisions. Conversely, there is concern that stereo evaluators will be biased by the knowledge of aesthetics, brand name and price.

I am asking you to help me reconcile this discrepancy.

Syndil wrote: »

I've noticed you are hanging your entire argument on that difference, training vs. experience. Tell me, is it possible that the word "experienced" might have been used for brevity, but to include the meaning of "trained" as well? If you are trained, are you not experienced?

If the words training and experienced were used interchangeably, it was evidence of poor scholarship because the words have very different meanings. However, Toole and Olive's 1994 paper makes it clear that they did not mean for experience to be synonymous with training. I explain in more detail later.

As I tried to explain before, training does not equal experience and vice versa. Training has to do with specialized instruction and practice. Experience is participation in an activity. A lawyer is assumed to be well trained after graduates from law school and passes the bar exam. Would you hire a recent law school grad to defend you in a murder trial? Of course not. You would want someone trained in the law and experienced in defending murder cases. I'm sure you have heard of people who have gotten caught practicing medicine or law without the required training and certifications.

In the realm of stereo, many people have many years experience listening to stereo systems, but have no training in how to evaluate them. Therefore, being an experienced listener does not equate to being a trained listener.

Syndil wrote: »

It would seem counterintuitive to me that Mr. Olive would have thrown out that work and used untrained but experienced listeners. Regardless, if you read what Mr. Olive wrote himself, you would have found that Mr. Olive did say that they were indeed trained listeners:

To my surprise, this mandate met rather strong opposition from some of the more entrenched marketing, sales and engineering staff who felt that, as trained audio professionals, they were immune from the influence of sighted biases.

The word "listener" does not appear anywhere in that statement. While it may be counter intuitive in your mind, the fact remains that there is no indication that the "experienced" listeners had any training in stereophonic sound evaluation. I explain further with quotes from the paper below.

Olive said they were trained audio professionals, not trained listeners. Every "medical professional" is not trained in the practice of medicine. Nurses, EMT's and medical doctors are all medical professionals, but all do not have the same medical training and experience. Saying that someone is a trained medical professional is not the same as saying they are trained in brain surgery. Saying that someone is an audio professional is not the same as saying they are trained in stereophonic evaluation.

Actually, the paper ("Hearing is Believing vs. Believing is Hearing: Blind vs. Sighted Listening Tests, and Other Interesting Things", Floyd Toole and Sean Olive, 1994) makes it very clear that the difference between being an experienced stereo listener is quite different from being a trained stereo evaluator:

"Experience is one of those variables among listeners that is very difficult to quantify. For example, musicians are experienced listeners but, is experience in focusing on musical attributes equivalent to that of focusing on timbral and spatial attributes? Some evidence suggests that it is not.

Gabrielsson found that musicians who were not also audiophiles, were not especially good judges of sound quality[4]. The famous pianist Glenn Gould came to appreciate the insights of non musicians[5]. Our own tests have confirmed this. So, listeners with different backgrounds could be expected to have differing abilities or preferences in subjective evaluations. This is an enormously broad topic, but we thought that it would be interesting to take a first step towards understanding the importance of this variable." (p. 2)

The 1994 Toole and Olive study divided subjects into two groups: inexperienced and experienced listeners as follows:

"The effects were tested using male experienced listeners and both male and female inexperienced listeners. In these tests, listeners were considered to be inexperienced if they had no previous experience in controlled listening tests. Other definitions are possible, which might include persons with no critical listening experience whatsoever. The participants were categorized under the following headings."

Therefore, "experience" in this case related to participation in controlled listening tests. There is no indication that any of the experienced listeners had training in stereophonic sound evaluation. Furthermore, the tests in this study were conducted in monophonic sound! (p. 3)

"The tests were conducted over a period of 1.5 weeks using a multiple ( 4 loudspeakers at a time) presentation method. The monophonic tests were conducted with the loudspeakers adjusted for equal loudness within 0.5 dB using B -weighted pink noise. Playback levels, which were constant throughout the tests, were set for typical "good listening." (p. 3)

Syndil wrote: »

I agree that the ability to describe sound characteristics can be trained. That is not the issue. The issue is and has always been the presence of bias. As you yourself said, The answer to that question is the answer to this entire debate. And the answer is bias.

All right then...since the central topic of this debate is evaluator bias, we have to deal with the issue of television evaluations using trained and experienced evaluators, but no product blinding. It appears that the television industry believes that an appropriately trained and experienced evaluator will not be biased by product identity. Why can't this evaluative model be applied to stereo sound?

If television evaluators can be trained to not be biased by product identity, why can't stereo evaluators be trained to not be biased by product identity?

To summarize

1. Your heroes, Toole and Olive, clearly demarcate a difference between listening experience and proper training for stereo sound evaluation.

2. Television evaluators rely on performance attribute training and experience to eliminate the effects of bias. Therefore, television trials are done non blind.

3. If training and experience in television can eliminate the effects of evaluator bias, the same can be said for stereo evaluation.

4. Do you understand that training does not equal experience and vice versa?

gdb

Snotty as they may be, I am really starting to appreciate the one-liner posts, in lieu of the full page variety !! I think all this hot air may be an untapped, alternative energy source !

F1nut

Originally Posted by Syndil
Glad you asked. Video can be compared and evaluated without relying on that most unreliable part of the human psyche, memory. Audio, however, cannot. They have very little in common. Is this not obvious?

Wow, just wow! That is the most asinine comment I've ever read here, bar none. In fact, you just earned a place on my BOZO list for your efforts.

Syndil

I stand by my original point that comparing TV trials to audio trials is apples to oranges. If that's what you want to hang your argument on, then it is my position you have been painted into a corner. I have no desire to waste effort confronting a weak analogy fallacy.

Regarding your summarized points:

1. As I have stated before, the issue is not training vs. experience--it is that bias cannot be trained out of a listener.

2. TV trials do not require blind testing because of the nature of video vs. audio, which are extremely dissimilar. The fact that they both come out of your AVR does not make them similar. Ask a blind person given retinal implants or a deaf person given cochlear implants how familiar their new sense is to their existing one.

There are numerous scientific experiments that do not need to be performed blind in order to eliminate bias. Finding one that does not, pointing at it, and then saying, "well, this doesn't need to be done blind, why should audio?" is not a valid argument.

To quote F1nut, "Wow, just wow!" I am amazed that this isn't as obviously apparent as I assumed--thus the brevity of the so-called "bum's rush" dismissal. It doesn't work. Move on. Find another analogy, or better yet, don't use an analogy and argue the points directly. Analogies are the crutch of those unable to directly argue the points at hand.

3. See point 2.

4. Same as point 1, isn't it? You seem to be insinuating that, at some point, I made the claim that experience and training were one and the same--and then spending a whole lot of words arguing against that claim. But you will find that no such claim has been made. So... Kudos on the effort, I guess, but you're tilting at windmills.

Makes for a nice smoke screen, perhaps, which I have come to determine is your preferred method of arguing: Throw up a wall of admittedly well-written, logical-sounding paragraphs, but none of which contain anything relevant to the actual points being argued. The TV analogy does not stand, and arguing that experience is not the same as training... I'll grant you that. I said as much in my last post.

Take those two points away from your last post, and what are we left with?

DarqueKnight wrote: »

Here is my own summary:

1. You don't see anything wrong with blind testing.

DarqueKnight wrote: »

Is it "wrong" to hide the identity of components when doing a stereophonic evaluation? Of course not.

2. However, with proper training, you feel that a listener can rise above any bias (either conscious/willful or unconscious/passive), rendering blinding an unnecessary complexity.

On point #2, we disagree. Sound about right?

BlueFox

Syndil wrote: »

1. As I have stated before, the issue is not training vs. experience--it is that bias cannot be trained out of a listener.

Really. One aspect of maturity is recognizing your biases, and consciously making an effort to over come those biases. If it can be done for race relations, and other more complex issues, then it certainly can be done for audio equipment.

To state as an absolute, "bias cannot be trained out of a listener", something that obviously is not true is just absurd.

Systems

I tell ya, Syndil, your too die hard.
You can argue with anyone, make them dislike ya, whatever.
Or you can make friends.
I have ALOT of friends on here.

I used to be a rebel myself (athanku, please sit down Jesse:cheesygrin:, have a seat).
Then I started actually talking to these guys as human beings.
Not just text posted on the net.

You did know they are human beings, right?

Talk my friend.
I can talk (through pm's whatever), we can all talk.

All these guys are the same as you.
Easy to talk to through pms, and gentlemen and scholars.

I didn't think they knew sh@t either at first.
They taught me different, just from reading their posts over the couple of years I have been here.
They truely are human beings, meaning you can talk to them, through pm's, and regardless, they will treat you the same way you treat them, like a human being.
I give most of them hell to this day!:cheesygrin:

Not just letters on a screen my friend.
Human!

I like Ford, you like Chevy.
They like Toyota!
Who gives a sh$t?
Would you want something simple as that to come between a friendship you could have for the next 10 years?

This is not the only audio forum.
But I think it may be the one you always come back too.:eek:

headrott

Syndil, your argument (from what I can gather) is that ALL people are biased ALL the time and the only way to overcome this is through a scientific process. Am I getting this right?

DarqueKnight

Syndil wrote: »

You seem to be insinuating that, at some point, I made the claim that experience and training were one and the same--and then spending a whole lot of words arguing against that claim. But you will find that no such claim has been made. So... Kudos on the effort, I guess, but you're tilting at windmills.

I do not need to insinuate when your comments are a matter of public record. Syndil's post #233:

Syndil wrote: »

If you are trained, are you not experienced?

Incredible!! This is a textbook example back of peddling:

Syndil wrote: »

The TV analogy does not stand, and arguing that experience is not the same as training... I'll grant you that. I said as much in my last post.

No, this is exactly what you said in your last post:

Syndil wrote: »

If you are trained, are you not experienced?

Do you suffer from multiple personality disorder? It appears that part of your mind is unaware of what the other part has said.

Syndil wrote: »

The TV analogy does not stand,...

It is amazing that you believe that the oft quoted paper by Toole and Olive (1994), which used untrained subjects and monophonic audio, is relevant to stereophonic systems evaluation, but studies showing that evaluator bias can be trained away is not relevant to stereophonic audio.

Syndil wrote: »

There are numerous scientific experiments that do not need to be performed blind in order to eliminate bias. Finding one that does not, pointing at it, and then saying, "well, this doesn't need to be done blind, why should audio?" is not a valid argument.

Oh really? Backpeddling much? Syndil's post #238:

Syndil wrote: »

As I have stated before, the issue is not training vs. experience--it is that bias cannot be trained out of a listener.

Syndil wrote: »

Science has shown time and time again that the myth of the unbiased observer is just that: a myth, and that no amount of training can eliminate that bias, regardless of how convinced you may be that you are above it. The way to eliminate bias is through methodological controls governing the way the experiment is conducted, not by "training it out" of the participants. Your entire argument rests on the supposition that it is indeed possible to do so.

So which is it? First you say that eliminating the effect of bias with training is a myth, then you grudgingly admit that it can be done, but the example I offered does not apply to audio...since it is video. Now you go so far as to say:

Syndil wrote: »

There are numerous scientific experiments that do not need to be performed blind in order to eliminate bias.

I agree...and the evaluation of stereophonic audio with trained listeners is one of them.

Syndil wrote: »

I stand by my original point that comparing TV trials to audio trials is apples to oranges. If that's what you want to hang your argument on, then it is my position you have been painted into a corner. I have no desire to waste effort confronting a weak analogy fallacy.

I was not comparing audio trials to TV trials. I was comparing the methodology used to train TV evaluators to the methodology I have proposed to train stereo audio evaluators. I used the TV trial example to show that evaluator bias can be effectively dealt with through training to the point that blinding is not required. It is interesting that you will not accept the "bias elimination through training" example from video, but you went head over heals in accepting a blind monophonic audio trial with untrained listeners as "proof" that stereophonic audio trials with trained listeners must be done blind. If I have been painted into a corner, then you have been buried under Mt. Everest.

BlueFox wrote: »

Really. One aspect of maturity is recognizing your biases, and consciously making an effort to over come those biases. If it can be done for race relations, and other more complex issues, then it certainly can be done for audio equipment.

To state as an absolute, "bias cannot be trained out of a listener", something that obviously is not true is just absurd.

Another aspect of maturity is the ability to accept when you have been proven wrong.


Summary of Syndil's Science:

1. Bias absolutely cannot be trained out of a person, or made to be irrelevant to a particular evaluation exercise. The only way to deal with subject bias in audio is with blinding.

2. The case of visual bias being effectively eliminated in video trials is irrelevant to stereophonic audio because video evaluators do not need to use their memories.

3. The fact that some hardcore racists, of all ages, races, ethnicities and genders, have overcome their biases through training and experience is irrelevant to stereophonic audio...because science says that visual bias in audio is absolutely insurmountable.

4. Comparing visual bias or racial bias to auditory bias is comparing apples to oranges, but a study using monophonic sound is directly applicable to stereophonic audio evaluation. The 1994 study by Toole and Olive which used blind and non-blind trials with untrained (but experienced) listeners and brief (looping 30 second) monophonic audio music samples is a perfect example of how people use stereo systems in the home.

5. The basic scientific principle that a thing must be tested according to the way it will be used does not apply to audio. In blind medical trials, medicine is administered exactly the way it would be in ordinary sighted treatment. Blind trials in audio do not have to be administered the way consumers use audio equipment in the home because of the insidious, insurmountable effect of visual bias.

Systems

Hell, eventually, I get to where I see them as a Polk member.
Fellow comrade, regardless of their beliefs/opinions/posts.

Really no different than yours eh?