Stupid things you hear from the DBT/Null test crowd....

DarqueKnight

Syndil wrote: »

At this point I am in fact throwing in the towel. I no longer find this thread amusing. I will address one final point, which has been the cause of my frustration throughout:

Originally Posted by DarqueKnight

Originally Posted by Syndil
I never claimed that they had the same definition. All A are B ≠ all B are A.

Yes you did:

Originally Posted by Syndil
I've noticed you are hanging your entire argument on that difference, training vs. experience. Tell me, is it possible that the word "experienced" might have been used for brevity, but to include the meaning of "trained" as well? If you are trained, are you not experienced?

If one cannot grasp the meaning of my above quotes, then one is not worth trying to have a debate with, as they simply lack the mental acuity to hold meaningful discussion.

If, for example, we lived in a world where all dogs happened to be red, and I saw a red object, I would not necessarily claim it was a dog. Apparently DK would.

I know the difference between a dog and a red object. I also know when someone is offering a "red herring" and irrational, circular logic. You attempt to deflect attention from your inability to answer legitimate questions by alleging a lack of mental acuity on my part.

1. Syndil asked "If you are trained, are you not experienced?". This would indicate to any rational person that he equated the two in his mind. In fact, he alludes to some error on my part in that I was posing an argument that training did not equal experience. If he actually agreed, at that time, that training did not equal experience, he would not have pointed out that I was arguing against training equaling experience. Furthermore, he certainly would not have asked "If you are trained, are you not experienced?", as if training automatically includes experience and vice versa.

2. I made the point that the Toole and Olive 1994 study used experienced rather than trained listeners. This is the study so many people point to as "proof" that blinding (hiding product identity from subjects) is absolutely required in stereo sound evaluations. However, this study used listeners untrained in stereophonic evaluation and the sound samples were 30 second loops of monophonic audio. Therefore, such a study does not apply to the question of the requirement of blinding for trained listeners evaluating stereophonic sound.

3. Syndil countered with "If you are trained, are you not experienced?".

4. I explained the differences between training and experience: training is specialized instruction and practice. Experience is participation in an activity. I also provided quotes from the paper which proved that Toole and Olive did not equate training with experience. The "experienced listeners" in their study were people who had previously participated in controlled listening tests. I provided examples such as training in the law does not equate to experience trying cases and having experience as an audio professional does not equate to training in stereophonic system evaluation. Indeed, the Toole and Olive study that Syndil hung so much of his hopes on was done in monophonic sound. Embarrassing to say the least. This is a perfect example that one should actually read the literature they reference, rather than just parroting comments from an Internet blog.

5. Syndil attempted to backpeddle by claiming that I misunderstood his meaning.

Items 1-5 above show no lack of mental acuity whatsoever on my part.

Now, Syndil is resigning from the thread discussion because of his frustration over my alleged lack of mental acuity and reading comprehension. Why am I such a big deal? I am not the only participant in this thread. This is just a dodge to avoid having to answer questions he obviously does not want to address such as:

DarqueKnight wrote: »

You have not explained why you think price, brand and aesthetics will not affect video evaluators. Black is not white, but there are different shades of black and white. One of the key performance parameters of televisions is black level. Is it not possible that an untrained evaluator might "see" deeper blacks and crisper whites in an expensive prestige brand television with sleek aesthetics than in a moderately priced unknown brand with plain aesthetics?

DarqueKnight wrote: »

Is it your belief that the knowledge of the price, brand and appearance of stereo equipment is a stronger biasing agent than racial prejudice and misogyny? Please explain...if you can.

DarqueKnight wrote: »

You did not address the point of why a monophonic test with untrained listeners is applicable to a stereo test with trained listeners. The main performance parameter of stereo is the spatial representation. Training is required to properly evaluate that spatial representation. Monophonic sound does not have a spatial representation. This is why I am asking you to elaborate on why the results of a mono test with untrained listeners is applicable to a stereo test that requires trained listeners.

Anyone else who has insights to share on the questions above is welcome to elaborate.

DarqueKnight

newrival wrote: »

I can appreciate all that, but I stilll do not understand how ABX would not be used in the training of listeners?

If ABX testing is not a scientifically valid method for the type of sensory information that stereophonic systems produce, why would you use it to train listeners?

Training for stereo listening consists of becoming proficient in sound localization and being able to describe the characteristics of those sounds. Please explain how ABX can be used as a training tool in such exercises.

tonyb

Just my opinion, but both sides of this seem to not be grasping the other. One side is planted in the science of it all, while the other is planted in the subjectivety of it all.

Granted, audio and video is subjective, but one side always claims B.S. because of lack of science, but when presented with the science, call it subjective. Unfortunately DK, most are not trained to listen and become proficient in sound localization, nor do I believe most even take the time to try a critical listening session as we like to call it. Unless it's blatenly obvious and smacks you in the face, differences in sound will blow right pass the casual listener. Which is more to DK's point too.

If your going to frame the arguement around testing methods, then one must presume some methods are better than others and DK did a fabulous job of pointing out those flaws in a variety of methods in this thread. You don't have to agree with any of this, just give thought to it and try to understand the science part of it all. In my opinion, the casual listener who doesn't take the time to listen correctly, may never pick up on the subtle differences in sound and therefore to them anyway, it doesn't exist. Which goes back to my point of the casual listener and the critical listener, both are listening for different things. In the end, your ears tell you what you like or dislike, and that guides any decisions you make in this hobby, science or no science, testing methods aside, placebo or not. Audio doesn't have to be this complicated, but it sure can be.

heiney9

Pretty much sums it up Tony. I am a critical listener, not to the point that it takes the enjoyment out, but to try and keep achieving more and more from my recordings and my rig. The past two nights I have rolled in a new pair of tubes, really inexpensive tubes, and the changes are very subtle, but there. I am liking what I am hearing based on the past two evenings of critical listening and evaluating. It's where I'm at in this hobby. It's amazing how much difference you can hear (good and/or bad) once you determine what to listen for. I am a soundstage freak, I love source material or tubes that re-create a huge soundstage or an intimate studio setting, or an intimate 3-4 microphone performance. That's what excites me and thrills me when listening. Hearing the scrape of the pick on the strings or the creak of a stool the performer is sitting on. Or the inhale during an acoustic solo performance, etc, etc. Some gear, cables, tubes, etc fall flat in this area some excel in this area. I don't need and ABX test to tell me which I prefer or which one is which. I know by extensive, critical listening.

It's just that plain and simple.

H9

sda2mike

plus the music needs to MOVE you, right???/\/\

evhudsons

I've learned a lot here, and the debate on testing has shed light on different ways to "view" my listening experience. I love my music but if I can be more attentive and interactive with my enjoyment I can enjoy it even more.

I like to scuba dive, and to see all the pretty corals and fish is great but hunting for Spanish treasure and artifacts with it's inherent difficulty and history is just fascinating and changes the whole experience of being underwater.

Both my wife and I heard a difference when I used the lowest end MIT cables. Neither of us are trained to know or describe fully what we heard, but knowing what to listen for would have certainly given me more insight into the whole experience of the change.

There are so many subtleties in life but to say some changes have to be glaring in order to validate the difference or the benefit of the difference is not really applicable to my personal home stereo experience. I can also cook up a nice duck, and you wouldn't know I slipped a little extra unexpected but nice spice such as cumin in. Even fantastic chefs would have a hard time picking up half a pinch of anything in that dish, but that doesn't mean it doesn't change the flavor. The appreciation would be in that finer detail but the trained AND experienced chefs could pick out and appreciate the change. Budding chef wannabees like me can learn to discern the subtle but applicable spice variations and the right applications.

At the same time, duck for dinner is not the same as home stereo. I can do them both though, and now that I'm kind of hungry I think I might have duck for dinner while I sample some unopened promo cd's I pulled out from back in my retail days.

Syndil

DarqueKnight wrote: »

1. Syndil asked "If you are trained, are you not experienced?". This would indicate to any rational person that he equated the two in his mind.

Negative. Holy crap, how many times do I have to spell it out for you? I even dumbed it down to an analogy to get you to understand.

I'll try one more time to get through. Simple exercise. Replace the word "trained" with the word "woman," and replace the word "experienced" with the word "human." So what we end up with is:

If you are a woman, are you not a human?

Now, since you are human, does that make you a woman?

Good grief, man, you must have failed the 6th grade.

Drenis

What does a persons' sex have to do with this subject?

evhudsons

In my opinion trained is not necessarily experienced, or at least experienced enough to fully be capable for that which experience only can give.

I was TRAINED as a hospital Corpsman in the Navy. Then I was TRAINED by the Marines to go into field battle. I was then dropped off in the Philippines as a bright eyed bushy tailed 18 year old, and learned the immediate difference between trained and experience. I got a bit more EXPERIENCE than most 18 year olds and it wasn't even just for what the Navy or Marines trained me! :razz: Historically my extracurricular activities were also used by fathers for their sons to use experience as a training tool to make their future wives much happier. Agree or not it is a historic fact.

In my thoughts, training leads to properly used experience in the path of learning. There is some element to training that could be experience, but they are not mutually the same or different. Some experience is training whether intended, or not, formal, or seat of the pants. Many guys get out of the service for instance and get into a field that they learned from their experience not their training although training had a huge role in gaining their experience.

I know that doesn't have any bearing on testing but to me there is a separation between training and experience.

newrival

headrott wrote: »

You can appreciate DK's points, but do you acknowledge them as being relevant? I don't think you would be asking the questions you asked if you found DK's points relevant.

obviously I find them relevant to the discussion at hand. I don't see how that would make me not ask the same questions. If his statements were irrelevant I would have asked what his points have to do with the argument. Perhaps you meant a different word?

What good does it do you if trained audiophiles verify there is a difference in audio equipment? Does that benefit you in hearing stereophonic audio by removing your bias against trying different audio components?

While I do think there is still going to be some level of bias, I am more concerned about interference. Interference to any scientific measuring device is always a concern and is a large consideration during the development of such devices. The human body is a very complex aggregate of mechanisms that are constantly measuring, synthesizing, and analyzing data from untold number of sources. It should be obvious to anyone that interference is a real concern considering all that is involved.

a/b or abx testing, in my mind, is just a tool that could be used for verification in conjuction with trained listeners. After all shouldn't any findings be reproducible?

DarqueKnight

Syndil wrote: »

Negative. Holy crap, how many times do I have to spell it out for you? I even dumbed it down to an analogy to get you to understand.

I'll try one more time to get through. Simple exercise. Replace the word "trained" with the word "woman," and replace the word "experienced" with the word "human." So what we end up with is:

Now, since you are human, does that make you a woman?

Good grief, man, you must have failed the 6th grade.

Syndil, you're back! I'm shocked (not really) that you would come back for an encore performance to explain some trivial point, but you don't want to address my questions regarding blind tests, monophony and bias that were summarized in post #274. However, I understand.

OK then, If you agreed that training does not equal experience, why did you question my pointing out that very fact? You offered an example of a paper that used "experienced" listeners. I merely pointed out the fact that experience does not equal training and that trained listeners are required for stereophonic audio evaluation.

Originally Posted by Syndil
I've noticed you are hanging your entire argument on that difference, training vs. experience. Tell me, is it possible that the word "experienced" might have been used for brevity, but to include the meaning of "trained" as well? If you are trained, are you not experienced?

This is your statement:

"If you are trained, are you not experienced?".

Syndil wrote: »

Replace the word "trained" with the word "woman," and replace the word "experienced" with the word "human."

If you do the word for word substitution you specified above, you do not end up with:

A.

Syndil wrote: »

Now, since you are human, does that make you a woman?

You end up with:

B. "If you are woman, are you not human?"

This is a true statement because you have to be a member of the human race to be a woman. However, the two terms are not reversible, but they are absolutely related. One term (woman) is a subset of the other term (human). Both terms denote membership in a group.

In the case of training and experience, the two terms are also not reversible, but they are also not absolutely related.

You reversed the order of the words in statement A, which changed the meaning entirely. If the word for word association that you specified is applied, you end up with B and not A.

In other words your analogy is false because:

You can have training and not have experience, because training has no absolute dependency on training or vice versa.

You can't be a woman and not be a human, because being human is an absolute prior requirement for being a woman.

To further illustrate this fallacy, I could substitute "trained as a lawyer" for "training" and "experienced in tying cases" for "experienced" into your original statement as follows:

C. "If you are trained as a lawyer, are you not experienced in trying cases?"

See how absurd this sounds? One can graduate from law school, pass the bar, then go work for the post office carrying mail. The former does not depend on the latter.

Now, if you reverse the word order, as you did in A, then it might make sense, because trying cases for other people requires training:

D. "If you are experienced trying cases, are you not trained as a lawyer?"

It should be noted that experience trying cases does not always equate to training as a lawyer. Some jurisdictions allow a defendant to represent themselves in court. Therefore, a person who has represented themselves many times in court, yet having no law degree, will have experience trying cases, but not have been trained as a lawyer.

Hope this helps.

DarqueKnight

tonyb wrote: »

Granted, audio and video is subjective, but one side always claims B.S. because of lack of science, but when presented with the science, call it subjective.

Isn't that amazing...and ironic? Obviously, the naysayers are only interested in the "science" that appears to support their positions. Naysayers frequently reference the 1994 Toole and Olive study as proof that stereo evaluation must be done blind. All the while they are oblivious to the fact that Toole and Olive used listeners untrained in stereophonic evaluation and that the sound samples were 30 second loops of monophonic sound.

When they are pressed to scientifically justify their positions, they refuse to do so. They try to turn the focus back on the questioner. However, audiophiles are consistently dogged for "proof" that they hear what they hear. If a difference is measured, well then, the measurement is not audible. If you prove the difference is within the range of audibility, well then, you didn't verify it with a blind test.

Syndil wrote: »

That's all well and good, but the scientists at Bell Labs are not the end-all be-all authority on hi-fi audio, despite your heavy reliance on their work, as you seem to refer back to them constantly.

Isn't it amazing that some people will scoff at the stereo sound evaluation methods proposed by Bell Laboratories scientists, the people who invented home stereo, but will fall head over heels in love with test methods developed for other forms of audio such as voice communications and monophonic sound?

tonyb wrote: »

Unfortunately DK, most are not trained to listen and become proficient in sound localization, nor do I believe most even take the time to try a critical listening session as we like to call it.

Yes, that is truly unfortunate that most people who own and listen to stereo have no idea what it is all about. I've lost count of the people who have sat in my living room and experienced shock and amazement at the sounds suspended in space and the speakers having "disappeared" sonically: "Wow...how does it do that?"

tonyb wrote: »

If your going to frame the arguement around testing methods, then one must presume some methods are better than others and DK did a fabulous job of pointing out those flaws in a variety of methods in this thread.

I also asked one of our blind test for stereo proponents to describe his audio evaluation methodology:

Syndil wrote: »

I do a simple XY comparison with the help of a friend. I may occasionally make notes about the differing characteristics of the sound if the comparison is close, but ultimately I prefer the deciding factor to be which sound is able to draw out the most emotion while I listen.

How very "scientific". If you ask, which preamp performed better, the answer will be "preamp Y drew out the most emotion while I listened". There will be no discussion of sound stage, image specifics or any of the other sound localization parameters stereo was designed for.

heiney9 wrote: »

I am a soundstage freak, I love source material or tubes that re-create a huge soundstage or an intimate studio setting, or an intimate 3-4 microphone performance. That's what excites me and thrills me when listening. Hearing the scrape of the pick on the strings or the creak of a stool the performer is sitting on. Or the inhale during an acoustic solo performance, etc, etc. Some gear, cables, tubes, etc fall flat in this area some excel in this area. I don't need and ABX test to tell me which I prefer or which one is which. I know by extensive, critical listening.

Stereo sound has many analogies to television in that they both trick the senses. Television creates a (rather poor) illusion of 3-D visual space using forced perspective. Stereo creates an illusion of 3-D auditory space using sound localization cues. No one argues that you don't need a blind test to see the difference between one color and another on a TV screen. However, these same people argue that a blind test is required to tell a difference between a pick scrape that exists with one amp that was missing with another. After all...ears can be tricked, but eyes are infallible.

newrival

DarqueKnight wrote: »

If ABX testing is not a scientifically valid method for the type of sensory information that stereophonic systems produce, why would you use it to train listeners?

Accepting your premise for the sake of efficiency only, it is because reinforcement of fundamental training, or foundational exercises of theory do not require the methods be scientifically valid. In this situation, you are only verifying the the listeners ability to consistently reproduce results or standards.

Training for stereo listening consists of becoming proficient in sound localization and being able to describe the characteristics of those sounds. Please explain how ABX can be used as a training tool in such exercises.

A listener, 1, is in a training session. The instructor has spent the previous courses providing examples of stereophonic audio characteristic Z. The instructor exposes 1 to two samples, A and B one of which exhibits an increased level of Z. 1 logs their experience. Then 1 is provided X, and again asked to log their perceptions. Upon completion and analysis of notes, 1 should be able to accurately identify X from A and B.

I find it analogous to a study of taxonomy and identification.

DarqueKnight

newrival wrote: »

A listener, 1, is in a training session. The instructor has spent the previous courses providing examples of stereophonic audio characteristic Z. The instructor exposes 1 to two samples, A and B one of which exhibits an increased level of Z. 1 logs their experience. Then 1 is provided X, and again asked to log their perceptions. Upon completion and analysis of notes, 1 should be able to accurately identify X from A and B.

What if sample B has an increased level of Z, but the level of Z only affects factor stereophonic attribute C, which is common to A and B?

What if the student has not been trained in the evaluation of stereophonic attribute C?

What if stereophonic attributes J and K can mask changes in C caused by increased B?

What conclusion(s) do you think the student might come to?

Syndil

DarqueKnight wrote: »

Blah blah...

This is a true statement because you have to be a member of the human race to be a woman. However, the two terms are not reversible, but they are absolutely related. One term (woman) is a subset of the other term (human). Both terms denote membership in a group.

Wait! Wait! What?? Can it be? Did you finally understand the exact point I was making? Eureka!

And... what's this? Knowingly or not, you also just admitted to being wrong about:

DarqueKnight wrote:

1. Syndil asked "If you are trained, are you not experienced?". This would indicate to any rational person that he equated the two in his mind.

Clearly, if they were equal, they could be reversed. Guess perhaps you weren't as rational as you initially thought!

Looks like we've made some real progress here. You get a gold star.


And this illustrates why I am bowing out of this thread. If I have to dumb everything down to a third grade level before it finally sinks in, I have no interest in participating.

heiney9

I think what DK has just shown is that stereophonic reproduction is very complex and reducing it to ABX is not applicable to stimuli that is complex among other things.

See it started out that the naysayers were simply asking for a change, any change between ABX. Now Newrival wants to quantify that change in relation to (2) other stimuli. Now naysayers are asking for a subjective judgment about differences, and that can't happen unless the subject is trained to observe and quantify those differences.

H9

newrival

DarqueKnight wrote: »

What if sample B has an increased level of Z, but the level of Z only affects factor stereophonic attribute C, which is common to A and B?

What if the student has not been trained in the evaluation of stereophonic attribute C?

What if stereophonic attributes J and K can mask changes in C caused by increased B?

What conclusion(s) do you think the student might come to?

I think you get my point, DK. Of course we could what-if until the cows come home, but that is nothing outside the realm of many teaching situations. Being able to read an NMR, for example, mirrors your questions exactly. Many bonding structures resonate differently in the presence of certain other elements or orientations. Some elements exhibit obvious characteristics in one situation, but in the presence of a different structure is completely masked. yet, we manage to teach people how to read them despite complex interdependencies and several "it is so except when it's not" scenarios.

Also, I might add that ABX testing (generally speaking) is heavily used in the training of NMR and other complex scientific procedures.

DarqueKnight

Syndil wrote: »

Wait! Wait! What?? Can it be? Did you finally understand the exact point I was making? Eureka!

And... what's this? Knowingly or not, you also just admitted to being wrong about:



Clearly, if they were equal, they could be reversed. Guess perhaps you weren't as rational as you initially thought!

Looks like we've made some real progress here. You get a gold star.

And this illustrates why I am bowing out of this thread. If I have to dumb everything down to a third grade level before it finally sinks in, I have no interest in participating.

Oh Syndil...you didn't understand. Just go back to this statement:

Point 1.

DarqueKnight wrote: »

OK then, If you agreed that training does not equal experience, why did you question my pointing out that very fact? You offered an example of a paper that used "experienced" listeners. I merely pointed out the fact that experience does not equal training and that trained listeners are required for stereophonic audio evaluation.

You can try to twist words and meanings all you want, but the fact that you challenged my statement that training does not equate to experience indicates that you believed they did.

Point 2: Women and humanity are related groups, wherein membership in one group is absolutely required for membership in the other group. Training and experience are not related subjects as each can be had without the other.

I don't know why you would run off just because of my alleged lack of mental acuity. Aren't there others here you deem to be on your high intellectual level?

I'm sure there are others here who would appreciate your guidance, particularly in regard to how to gauge the amount of emotion drawn out by stereo components under evaluation.

Furthermore, I would much rather you address the matters of blinding, bias, and monophony rather than these repeated encore presentations on something I have unequivocally proved you to be in error about...and I did it with the very words from the reference material you pointed to (but obviously had not read).

Saying that you are running off due to my alleged lack of understanding is just smokescreen, as if this is just a discussion between you and me. Even Stevie Wonder can see that you are avoiding providing scientific justification for your views on the requirement for blind tests for stereo.

This is not surprising...since your emotions are your primary evaluative tool for stereo.

Originally Posted by Syndil
I do a simple XY comparison with the help of a friend. I may occasionally make notes about the differing characteristics of the sound if the comparison is close, but ultimately I prefer the deciding factor to be which sound is able to draw out the most emotion while I listen.

heiney9

newrival wrote: »

I think you get my point, DK. Of course we could what-if until the cows come home, but that is nothing outside the realm of many teaching situations. Being able to read an NMR, for example, mirrors your questions exactly. Many bonding structures resonate differently in the presence of certain other elements or orientations. Some elements exhibit obvious characteristics in one situation, but in the presence of a different structure is completely masked. yet, we manage to teach people how to read them despite complex interdependencies and several "it is so except when it's not" scenarios.

Now bonding structures in science are the same as stereophonic reproduction. I guess anything is fair game to reinforce a point even if it has no relevance whatsoever.

H9

DarqueKnight

newrival wrote: »

I think you get my point, DK.

I'm not sure you got my point. The point was that there could be a difference, Z, but the difference could be masked by something else that the subject is not trained to recognize. if the subject is not trained to recognize it, they will not perceive a difference. A simple A/B or ABX test will give a false result.

newrival

heiney9 wrote: »

Now bonding structures in science are the same as stereophonic reproduction. I guess anything is fair game to reinforce a point even if it has no relevance whatsoever.

H9

H9, i love you man, but come on.

It's called an analogy.

newrival

DarqueKnight wrote: »

I'm not sure you got my point. The point was that there could be a difference, Z, but the difference could be masked by something else that the subject is not trained to recognize. if the subject is not trained to recognize it, they will not perceive a difference. A simple A/B or ABX test will give a false result.

No I got it. And my point was that this is a rudimentary issue of instruction. You are not going to test the trainee on something they have not been taught. It is something that can be used to reinforce theory.

A person designing the training would presumably know these interactions, and as such, would choose appropriate examples for the purposes of reinforcing previous lessons and to gauge ability or comprehension.

headrott

newrival wrote: »

obviously I find them relevant to the discussion at hand. I don't see how that would make me not ask the same questions. If his statements were irrelevant I would have asked what his points have to do with the argument. Perhaps you meant a different word?

So are you just ignoring the points he made then? I ask because in the point DK wrote below:

"1. Forced-choice discrimination test methods, such as ABX, A/B, etc., are not suitable for multi-dimensional stimuli. They are more suited for simple discrimination tests where the subject's senses are not affected by a number of simultaneous stimuli. A "dimension" in this case being a characteristic or unit of measurement.

2. A stereophonic sound field is multi-dimensional in nature in that it presents a complex set of simultaneous stimuli to the listener: sound stage height, width and depth; the location and character (clarity, weight, detail, dynamics, etc.) of images in the sound stage, and tactile sensation.

3. The Bell Laboratories scientists who invented home stereo systems were experts in the application of forced-choice discrimination test methods to telephony audio. Such methods, like ABX and A/B, were well suited for discerning simple quality differences in limited bandwidth telephone voice circuits.

4. When these same Bell Laboratories telephony scientists turned their attention to developing home audio equipment, the evaluation methods for such equipment was based on trained listeners who knew how to properly evaluate all the dimensions in stereophonic sound reproduction. Furthermore, peer-reviewed scientific journal literature clearly states that the stereophonic test methods used by Bell Laboratories scientists were intended to be adopted by the consumer."

It should be obvious that ABX and AB tests were not intended nor suited for stereophonic audio. Since this is the case, then why would it be suitible for training a listener in stereophonic audio?

newrival wrote: »

While I do think there is still going to be some level of bias, I am more concerned about interference. Interference to any scientific measuring device is always a concern and is a large consideration during the development of such devices. The human body is a very complex aggregate of mechanisms that are constantly measuring, synthesizing, and analyzing data from untold number of sources. It should be obvious to anyone that interference is a real concern considering all that is involved.

a/b or abx testing, in my mind, is just a tool that could be used for verification in conjuction with trained listeners. After all shouldn't any findings be reproducible?

Humans have something innate within them called awareness and consiousness that scientific measuring devices created by humans do not have. If you train yourself in using this awareness, you become aware of interferences that will affect your view of reality and this awareness allows you to manage them (intereferences). In the case of stereophonic audio, reality would be the factors that make up the stereophonic soundstage height, width, and depth, imaging, detail, etc. etc. Since, as established above, ABX and A/B testing are were never intended to be used for and are not suited for use in stereophonic audio, why would a listener use ABX and A/B testing as a verification of anything having to do with stereophonic audio?

newrival

headrott wrote: »

So are you just ignoring the points he made then?

nope, i considered them, I just don't feel I need to respond directly to each one. If I had more time to do so I would, but I don't have the time to pick apart everything everyone says.

I ask because in the point DK wrote below:

"1. Forced-choice discrimination test methods, such as ABX, A/B, etc., are not suitable for multi-dimensional stimuli. They are more suited for simple discrimination tests where the subject's senses are not affected by a number of simultaneous stimuli. A "dimension" in this case being a characteristic or unit of measurement.

2. A stereophonic sound field is multi-dimensional in nature in that it presents a complex set of simultaneous stimuli to the listener: sound stage height, width and depth; the location and character (clarity, weight, detail, dynamics, etc.) of images in the sound stage, and tactile sensation.

3. The Bell Laboratories scientists who invented home stereo systems were experts in the application of forced-choice discrimination test methods to telephony audio. Such methods, like ABX and A/B, were well suited for discerning simple quality differences in limited bandwidth telephone voice circuits.

4. When these same Bell Laboratories telephony scientists turned their attention to developing home audio equipment, the evaluation methods for such equipment was based on trained listeners who knew how to properly evaluate all the dimensions in stereophonic sound reproduction. Furthermore, peer-reviewed scientific journal literature clearly states that the stereophonic test methods used by Bell Laboratories scientists were intended to be adopted by the consumer."

It should be obvious that ABX and AB tests were not intended nor suited for stereophonic audio. Since this is the case, then why would it be suitible for training a listener in stereophonic audio?

I'm aware of what he said, as you have pointed it out several times. Not to mention I read it the first time around. OK so I will agree with you on it not being intended for stereophonic audio (i don't know this to be true, but DK does his homework), but so what? Things get repurposed all the time. Additionally, I have made a case for why it is suited to it just fine. I think a little openmidedness could be applied her.

newrival

headrott wrote: »

Humans have something innate within them called awareness and consiousness that scientific measuring devices created by humans do not have. If you train yourself in using this awareness, you become aware of interferences that will affect your view of reality and this awareness allows you to manage them (intereferences).

So they're all going to be zen masters? It's easy for you to talk about subtracting intereference from the equation, but it just isn't so. There are thousands of processes going on in your body at any given moment, not to mention any brought on by external circumstantial and emotional sources, and you're telling me that is of no concern? You're skimming the surface of the issue and ignoring the real problems with it. I've enjoyed this discussion, but I believe you are dealing with this superficially.

In the case of stereophonic audio, reality would be the factors that make up the stereophonic soundstage height, width, and depth, imaging, detail, etc. etc.

reality how? I'm sorry to break it to you, but stereo by its very nature is not reality, it is perception. Stereo is about tricking your mind. It is illusory by design.

Since, as established above, ABX and A/B testing are were never intended to be used for and are not suited for use in stereophonic audio, why would a listener use ABX and A/B testing as a verification of anything having to do with stereophonic audio?

I've already answered this, and made it abundantly clear. In case I haven't, take this example: a screwdriver was never intended to do anything but drive or remove screws, yet they sure do a great job opening a can of paint.

DarqueKnight

newrival wrote: »

I've already answered this, and made it abundantly clear. In case I haven't, take this example: a screwdriver was never intended to do anything but drive or remove screws, yet they sure do a great job opening a can of paint.

I prefer to use a paint can opener. It is a tool with a flat curved end that does not cause dents creases or warps the rim of the paint lid. Such dents, creases and warps can cause air gaps that cause the paint to dry out in the can, particularly if there are many around the rim.

A screwdriver can get the job done, but it can also cause damage.

It is always best to use a tool intended for its purpose.

newrival

DarqueKnight wrote: »

I prefer to use a paint can opener. It is a tool with a flat curved end that does not cause dents creases or warps the rim of the paint lid. Such dents, creases and warps can cause air gaps that cause the paint to dry out in the can, particularly if there are many around the rim.

A screwdriver can get the job done, but it can also cause damage.

It is always best to use a tool intended for its purpose.

Yes, I realize. But as a device to prove my point it is sufficient, and not to be taken literally. Had that comment been directed to you, I likely would not have used that analogy. Moreover, I likely would not have needed an analogy at all.

At any rate, I am more curious what you think of my previous assertion.

DarqueKnight

newrival wrote: »

No I got it. And my point was that this is a rudimentary issue of instruction. You are not going to test the trainee on something they have not been taught. It is something that can be used to reinforce theory.

A person designing the training would presumably know these interactions, and as such, would choose appropriate examples for the purposes of reinforcing previous lessons and to gauge ability or comprehension.

Your questions are valid and informative.

In sensory training, sometimes a student is tested on things they have not been taught in order to assess the student's perceptive abilities. For example, prior to teaching someone how to localize instruments in a sound stage, I will simply ask the subject to list how many instruments there are and where are they located. I typically get a chart with instruments grouped around the front of the speakers and some instruments, particularly percussion and synthesizers, are missed. After some training is done in looking directly at a sound image and being able to point to it, subsequent localization maps are more accurate.

Forced choice discrimination methods are short-term tests designed to find simple overall differences such as:

1. Is one sample louder than the other?
2. Is one sample clearer than the other?
3. Is one sample higher/lower in pitch than the other?

Forced choice methods rely on a subject's memory rather than documentation acquired over time.

If you are listening to a stereo sample that is simple in nature (not a lot of instruments spread across the sound stage) and you are only testing for a simple overall difference between two music samples, an A/B or ABX test might work. The A/B protocol was successfully used with the stereo music samples in the Petri-Larmi study I referenced earlier in this thread. In that case, where music samples were contaminated with various levels of noise, trained and sensitive subjects only had to tell if there was a difference between A and B. There was no requirement for describing the character of the difference. Subjects noticed a change in tonal character, but were not required to provide a detailed description of the change. In other words, there was no description such as "the high frequencies were brighter", "the low frequencies were more more prominent" or "the midrange frequecies were recessed from the highs and lows".

For someone interested in puchasing amplifier A or amplifier B, simply saying A has a different tonal character than B does not provide serious performance information. An accurate description of the details of the differences in tonal character would be required for a consumer to know if amp A or B should be auditioned. However, taking time to characterize differences is outside the realm of forced choice discrimination tests such as A/B and ABX. Descriptive tests are designed for this purpose. If an accurate description of a component's tonal character is desired, many samples of music must be listened to over days, weeks and, depending on the variety and complexity of the music listened to, perhaps months.

Sound stage dimensions can be particularly difficult to measure because it will vary from recording to recording.

Some ask, "can't you do a combination of descriptive analysis and A/B or ABX?" That is like asking "can't you do a combination of speed dating and long term dating?" The answer is no, you can't. They are different methodologies for different things. The former looks at gross differences in order to quickly weed out undesirables from a group of potential dating partners. The latter is for really getting to know someone. You cannot assess character in a speed dating scenario. That takes time...and focus. A speed dating session can inform about a person's looks, occupation and lifestyle. It cannot inform about the things that contribute to a satisfying long term relationship such as honesty, reliability, character and mental health.

You can be a knucklehead and say, "well, I'm going to use speed dating to narrow my choices and then use long term dating to get to really know the top two candidates". Again, you can't assess character in a high-pressure time-limited environment. Inevitably, something important will be missed. You don't have time to do a criminal background check in a speed dating session.

You may have noticed that I asked one of the thread's former participants to describe their stereo evaluation method. That person said that an A/B protocol was used and that the component that "drew out the most emotion" was the one selected. If you were considering amp A or amp B, tell me which of the two evaluations would be of most practical use to you:

Evaluator 1: "I had a friend come over and switch between A and B. B was selected because it drew out the most emotion."

Evaluator 2: "I listened to A for three days and recorded the location, size and auditory weight of each image in the sound stage. The clarity and detail of each image was also recorded. For the next three days I did the same for B. On the seventh day I compared notes. A had a wider and deeper sound stage, but B had greater clarity - I noted some percussion instruments and bass growls that I had not noticed with A. I went back and listened for the same percussion instruments and bass growls with A, but they weren't there. Although B was the most accurate with regard to overall sound quality, I selected A because sound stage size is more important to me than overall clarity."

Notice that Evaluator 2 did not need to retain the memory of amp A's performance when B was being tested. The focus was only on B. Can you see where being able to focus on only one component at a time might lead to more accurate results rather than having to mentally juggle the sound profile of two components?

Sometimes I do not become aware of a difference in components until I compare evaluation notes and sound stage maps. When I am comparing one component, the other is out of my mind...unless some glaring difference is there. It must be realized that, when you get into high performance stereo equipment, the differences are often not subtle, but they are often not glaring either. Sometimes it's just a difference in the sound stage expanding or (choke!, gasp!) contacting by 1 or 2 feet.

Let's assume that someone is hardheaded and says they were going to fully characterize two amplifiers using the ABX or A/B protocol. Knowing that the length of time required to fully characterize an audio component can run into days and weeks, what is the practicality of getting someone to do the required equipment switching over such a time span? Who among us can mentally juggle the sound profiles of two components over a days and weeks?

I would think that most people would want a stereo evaluation protocol that they could do themselves and that would allow them to fully document the performance differences in audio components over a reasonable time span in a non-stressful manner.

I hope you didn't mind my taking the time to expound on your questions in some length. As I said, I found your questions to be valid and informative for those interested in serious discussion about this topic.

headrott

newrival wrote: »

So they're all going to be zen masters? It's easy for you to talk about subtracting intereference from the equation, but it just isn't so. There are thousands of processes going on in your body at any given moment, not to mention any brought on by external circumstantial and emotional sources, and you're telling me that is of no concern? You're skimming the surface of the issue and ignoring the real problems with it. I've enjoyed this discussion, but I believe you are dealing with this superficially.

I never said, "intereference", as you are phrasing it, was not a concern. I would use the term distractions, however. I said that distractions can be dealt with by using your consciousness and awareness to overcome them.

Your consciousness is the most fundament part of what makes you, you. So, I am certainly not skimming the surface of and ignoring the real problems with the issue. It is people that are overly distracted that are ignoring the real problems.

newrival wrote: »

reality how? I'm sorry to break it to you, but stereo by its very nature is not reality, it is perception. Stereo is about tricking your mind. It is illusory by design.

If you read what I wrote again, you would notice that I said reality would be the factors that make up the stereophonic soundstage which include: soundstage height, width, depth. Imaging, detail, etc. etc.
A stereophonic image is the illusion of two independant audio signals on the left and right combining to make a stereo one, but just because it is an illusion doesn't make your perception if the imaging, soundstage height, width, and depth and detail of the signal any less real does it? Just because a magician uses an illusion to make you think that he cut a lady in a box in half doesn't make it any less real that you witnessed his illusion, correct? Is you sitting in the audience witnessing this illusion not reality? Or are you saying that you would't really be in the audience because what the magician is doing on stage is just an illusion?

newrival wrote: »

I've already answered this, and made it abundantly clear. In case I haven't, take this example: a screwdriver was never intended to do anything but drive or remove screws, yet they sure do a great job opening a can of paint.

Ray gave a good example to answer this, so I won't bother.

newrival

DarqueKnight wrote: »

I hope you didn't mind my taking the time to expound on your questions in some length. As I said, I found your questions to be valid and informative for those interested in serious discussion about this topic.

I don't mind at all, in fact I appreciate it. It's good to hear to hear that side of the training. A few questions popped up for me while reading your response:

How do you see implementation of this process? I would assume they need to be independent evaluators. If they were employed by manufacturers that would pose obvious concern. It seems recording studios would be the ideal vector for such a system.

If they are indeed independent "review houses," do you see them meeting some regulatory standard? How could it be met and verified (if not by abx)? And this begs the question of what kind of economic viability would a company like this have? Although this could be moot if it is not a structure you have in mind.

Also, to the point of recording variability, wouldn't there have to be some common material used at EVERY audition to establish some sense of baseline?

Moreover, since many descriptors are relative quantifiers (deep, wide, light, heavy, more, etc.), what would be the relative control, or the standard, from which relative quantifiers would be deriving their delta?