Observations Of A "Good" Cable Test
Comments
-
I have no idea why you would need to be blinded for a ABX test. It seems silly.
I can understand the desire to remove "visual bias". A lot of high end gear is designed to be very visually appealing. Some people ascribe more positive attributes to things that are "pretty" and expensive. Most people would prefer to have a more physically attractive, and rich, partner if they have to make a choice in a short period of time. However, if you give them six months to evaluate between a physically gorgeous, and rich, person and an average looking person, with modest income, other attributes will become more important. The way to remove visual bias is to spend an appropriate amount of time using proper evaluation techniques.
It is the same with audio. A person may be dazzled by the appearance of a piece of gear and really want it to work for them, but if they spend enough time with it they will begin to focus more on the sonic performance aspects.My problem is what differences to look(hear) for during the ABX hearing test. As I undertsand it DK uses and compares the locations of the various instruments during playback. I have never tried this and I also understand why you would need to write down these observations. Please correct me if I have this wrong.
It is not just the evaluation of locations of sound stage images, but also the evaluation of the sonic characteristics of those images and the careful documentation of those locations and sonic characteristics. Sometimes, I do not become aware of sonic differences until I compare listening notes and image placement charts between two pieces of gear.
When I listen, I am not focused on the perception of differences, I am focused on the accurate characterization and documentation of what I am hearing.This type of comparisons would require exact seating placement which is also fine.
Well, the technical specification for stereophonic music systems REQUIRES the listener to be seated on-axis in the stereo sweet spot. Any seating position outside of the sweet spot compromises imaging.However, is there a standard for what sound field is best or correct or is it based on our opinion?
The sound field generated will depend on recording technique and the quality and characteristics of the playback equipment.
Of course, there are standards for the proper setup of stereophonic audio systems and for the optimum listening location. The modern version of two-channel stereo was invented at Bell Laboratories in the 1920's and the term "stereophonic" was coined there. It is not commonly known that many components of modern hi-fi technology are byproducts that came out of research done by Bell Telephone System scientists.Proud and loyal citizen of the Digital Domain and Solid State Country! -
LessisNevermore wrote: »Sounds like you had an unwilling participant.
Wasn't my test. maybe you should actually read it before comenting. He seemed very willing. -
hearingimpared wrote: »Why is that irrelevent Bill?
Because at a concert, you are not comparing two different things. How is that relevant to an ABX test?Also what have the eyes to do with music coming through a hifi system.
Nothing. But people don't seem to be able to tell different cables apart without them. -
With some cables, gear, tubes, the finger snapping is buried in the mix barely audible and with others it's right at the fore front, very crisp and clear, and I can hear the ambience around the finger snapping and then with some gear it's all over in between.
How someone isn't able to pick up on a single, simple cue like this is beyond me.
You have spent a lot of time (years) training your ears in critical listening. For most people, a "good" stereo is one that plays adequately loud and clear. They aren't concerned with subtleties and nuances like the ambiance around finger snaps.
I think that many audiophiles discount the perceptive ability they have acquired over years of listening. Like yourself, they are boggled when someone else can't hear something that is blatantly obvious to them. Thinking back to when you first got involved with audio, were things like the ambiance around finger snaps even part of your consciousness? It certainly wasn't part of mine.Proud and loyal citizen of the Digital Domain and Solid State Country! -
LessisNevermore wrote: »I sort of get the sighted bias theory, but I also think that any sane, rational adult can put preconceived notions aside and remain objective. The self proclaimed, "scientists" apparently believe that people are too stupid to do anything objectively, well maybe they can't....
This shows how little you understand bias. It's subconscious, and can't be controlled or predicted. -
One thing I think this test was "good" at doing is pushing the agenda that they wanted to push. That is, that cables don't make a difference. That is what I got out of reading your post Raife. Maybe I am reading too much into it though, but maybe not.:)
Greg
I like this quote....posted recently in a different thread....kinda flips the coin the other way
:D
P.S. just because someone publishes a paper doesn't make it gospel. Just like "I read it on the internet".
And the subconcious bias thing....kinda applies to those running the test - I seriously doubt that alot of folks that run these so called ABX tests are not themselves unconsiously biased for what they want the results to be.....
H9: If you don't trust what you are hearing, then maybe you need to be less invested in a hobby which all the pleasure comes from listening to music. -
In my first post, I laid out the reasons why I believe this test was not a valid one:
1. Test specimens is not clearly identified.
2. Associated audio gear not specified.
3. No mention whatsoever of imaging, sound stage characteristics or tactile sensations from the sound stage.
4. No mention of the length of time musical selections were listened to.
5. No mention of the type and quality of the recordings.
6. No discussion whatsoever of performance metrics (clarity, detail, imaging, tactile sensations, etc.).
7. Some of the test attendants/participants seemed to be more interested in playing pool than conducting audio trials.:)
So, I would like someone to explain to me why this is a "good" audio equipment evaluation test. Is this not a reasonable request?
Such Good Science Proud and loyal citizen of the Digital Domain and Solid State Country! -
Another observation I made awhile ago that stands out is when I added MIT speaker cables to the mix. There is a Jewel song Down So Long where with the MIT's I noticed a breath Jewel takes at one point in the song and it's pretty obvious. I had never heard that before with any other cable/gear swaps. At that point I had listened to that song hundreds of times, but it took the MIT's for me to hear more nuances in the song.
H9"Appreciation of audio is a completely subjective human experience. Measurements can provide a measure of insight, but are no substitute for human judgment. Why are we looking to reduce a subjective experience to objective criteria anyway? The subtleties of music and audio reproduction are for those who appreciate it. Differentiation by numbers is for those who do not".--Nelson Pass Pass Labs XA25 | EE Avant Pre | EE Mini Max Supreme DAC | MIT Shotgun S1 | Pangea AC14SE MKII | Legend L600 | BlueSound Node 3 - Tubes add soul! -
LessisNevermore wrote: »Which is why I said a sane, rational adult.:) Kids are fickle, and too easily excitable.
It applies just as much if not more to adults. The longer a belief or idea is held, the more times it has to be contradicted to change what the person believes. -
Wasn't my test. maybe you should actually read it before comenting. He seemed very willing.
Very willing, just not willing enough to see it through to the end. And I did read it, or I wouldn't have commented.This shows how little you understand bias. It's subconscious, and can't be controlled or predicted.
Really? Care to list your qualifications?
All you've ever done on this site is parrot what the pretend scientists say on the internet. I know what I hear and don't, and I hear it or not, with no agenda. -
It applies just as much if not more to adults. The longer a belief or idea is held, the more times it has to be contradicted to change what the person believes.
Ok, what if you listen to something with no preconceived notions? If you are telling me this is not possible, then you're nuts. It doesn't even have to be gear, it could be music. Do I need to be blindfolded to decide if it is to my taste? Do I need test instruments to decide if I like it?
Is it possible to be indifferent to a brand, or all brands? It sounds like you are saying that it's not. No matter what I listen to, it's with a bias of some type, right? I don't hold any allegiances to equipment, there is ALWAYS something better, and I'd like to get there, no matter where 'there' is. If is says Boulder, fine, if it says Sony, so be it.
Ok, I'm done with derailing this thread. Sincere apologies to DK, who is still waiting for an answer, as are the rest of us. -
DarqueKnight wrote: »Does anyone know of any A/B or ABX audio tests in which the subject/subjects were legally blind?
At my home I poke out the eyes of anyone wishing to do a blind test.
madmaxVinyl, the final frontier...
Avantgarde horns, 300b tubes, thats the kinda crap I want...
-
DarqueKnight wrote: »You have spent a lot of time (years) training your ears in critical listening. For most people, a "good" stereo is one that plays adequately loud and clear. They aren't concerned with subtleties and nuances like the ambiance around finger snaps.
I think that many audiophiles discount the perceptive ability they have acquired over years of listening. Like yourself, they are boggled when someone else can't hear something that is blatantly obvious to them. Thinking back to when you first got involved with audio, were things like the ambiance around finger snaps even part of your consciousness? It certainly wasn't part of mine.
hmm depends I think upon the person. Ever since I was young and started playing an insturment (alto sax) I was constantly listening to music and trying to pin point the different instruments playing and their tone ect. It was a way I learned how to play, I would listen to a whole jazz track, concentrate on the sax being played and then try to learn the song that way. It did take a bit of time to learn how to focus on the one instrument and then figuring out what they were playing was always fun too, but now its easy to pin point in a track, ok that is the high hat, there is a snare drum, or in classical, flute, horn, ect and listen to what they do through out the track.
To me its even easy to hear this in the simpliest of gear, for example an AVR and two different pair of speakers. If you have horns listen to the high hat on a track, something easy to typically hear, then switch to a speaker with a tweeter, it will sound different. I don't feel it took me really years to do this, as my one teacher often wanted me to play based upon hearing when I was in 5th grade and that was the year I started to really focus on it. I think by the end of that year I got to the point when I really could pin point things, and just got better after that. If someone take the TIME to do it I'm sure it wouldn't take too long. Also I did it as a young child, so it was easier I think for me to take the time to sit and listen (its really all I did and read, I didn't have a life until high school).
I think that people in general that listen are not really listening is one problem. Some people who claim that they don't hear differences often tell me they are doing something else while listening, either testing the gear for measurements, on a computer ect. If people do take the time to sit down and listen to a song (something most people can't seem to figure out how to do in today's world) then they can start picking up on differences.
Apart from that thanks for the post DK. I really enjoy your viewpoints on the subject and how you explain things. I really think you come at it logically and in a positive way that makes me want to go out and listen to a system even more! -
DarqueKnight wrote: »You have spent a lot of time (years) training your ears in critical listening. For most people, a "good" stereo is one that plays adequately loud and clear. They aren't concerned with subtleties and nuances like the ambiance around finger snaps.
I think that many audiophiles discount the perceptive ability they have acquired over years of listening. Like yourself, they are boggled when someone else can't hear something that is blatantly obvious to them. Thinking back to when you first got involved with audio, were things like the ambiance around finger snaps even part of your consciousness? It certainly wasn't part of mine.
The finger snap sounds like a good test for your system. I also think that the live applause at the end of the recording is a good test of the system and recording.
I agree that most people aren't concerned with music nuances. I would also guess that I am much less interested in these nuances than you and many others on this forum. I am however concerned, this is just me now, with background noise. When I was younger I listened to music with all the windows shut while it was 90 degrees because I couldn't tolerate the outside car noise from the road. That's why a don't enjoy LPs as Cds because of the pops and background hiss. My nuances is reproducing the dynamic range of music with a quiet background. It's most important to me. My problem is that dynamic range in many recordings today is compressed.
I guess we all have our nuances.
Another question is what is the correct or most accuate finger snap reproduction?
Leaving the LP and CD recording problems and issues aside you really need to hear the snaps live without any electronics to find a some type of referance.
Good luck with that. -
I think that people in general that listen are not really listening is one problem. Some people who claim that they don't hear differences often tell me they are doing something else while listening, either testing the gear for measurements, on a computer ect. If people do take the time to sit down and listen to a song (something most people can't seem to figure out how to do in today's world) then they can start picking up on differences. QUOTE]
Agree completely. No time to listen. Don't understand it or why.
Multitasking??????? -
LessisNevermore wrote: »Ok, what if you listen to something with no preconceived notions? If you are telling me this is not possible, then you're nuts. It doesn't even have to be gear, it could be music. Do I need to be blindfolded to decide if it is to my taste? Do I need test instruments to decide if I like it?
Is it possible to be indifferent to a brand, or all brands? It sounds like you are saying that it's not. No matter what I listen to, it's with a bias of some type, right? I don't hold any allegiances to equipment, there is ALWAYS something better, and I'd like to get there, no matter where 'there' is. If is says Boulder, fine, if it says Sony, so be it.
Ok, I'm done with derailing this thread. Sincere apologies to DK, who is still waiting for an answer, as are the rest of us.
I may be nuts, but I'm not saying it is impossible to listen without preconceived notions.:D I'd say it's possible to be indifferent to any and all brands, but often not likely. I also agree there is always something better. I was just trying to show that bias in experiments often shows up no matter how hard we try to avoid it (which I think is in keeping with the tread topic). -
DarqueKnight wrote: »
Question: Why would someone conduct music evaluations with music they can't stand to listen to repeatedly? This isn't good, or logical, test methodology.
.
.
.
5. What was the type and quality of the recordings? There was no mention of this, although there was mention of listening fatigue after only 8 trials. Really? Listening fatigue in that room with that equipment? It is interesting that Mr. Lavigne has never mentioned listening fatigue in any of his prior discussions about his room and system.
I think it may not be the music, but the testing methodology producing the fatigue. By putting themselves in an unnatural listening situation to the point of fatigue the test becomes invalid. You are no longer listening within the same framework by which the subtle differences reveal themselves.SystemLuxman L-590AXII Integrated Amplifier|KEF Reference 1 Loudspeakers|PS Audio Directream Jr|Sansui TU-9900 Tuner|TEAC A-6100 RtR|Nakamichi RX-202 Cassette -
These cables threads are like this. Both believers and non-believers are in constant war but are in complement to each other.
Trying out Different Audio Cables is a Religious Affair. You don't discuss it with anyone. :redface::biggrin: -
hmm depends I think upon the person. Ever since I was young and started playing an insturment (alto sax) I was constantly listening to music and trying to pin point the different instruments playing and their tone ect. It was a way I learned how to play, I would listen to a whole jazz track, concentrate on the sax being played and then try to learn the song that way. It did take a bit of time to learn how to focus on the one instrument and then figuring out what they were playing was always fun too, but now its easy to pin point in a track, ok that is the high hat, there is a snare drum, or in classical, flute, horn, ect and listen to what they do through out the track.
To me its even easy to hear this in the simpliest of gear, for example an AVR and two different pair of speakers. If you have horns listen to the high hat on a track, something easy to typically hear, then switch to a speaker with a tweeter, it will sound different. I don't feel it took me really years to do this, as my one teacher often wanted me to play based upon hearing when I was in 5th grade and that was the year I started to really focus on it. I think by the end of that year I got to the point when I really could pin point things, and just got better after that. If someone take the TIME to do it I'm sure it wouldn't take too long. Also I did it as a young child, so it was easier I think for me to take the time to sit and listen (its really all I did and read, I didn't have a life until high school)..
Great post, very similar to my experience. Figuring out a musical part- note for note, along with all of the subtleties and accents, then teaching yourself how to duplicate it. I've heard lots of drummers play a Phil Rudd (AC/DC), or John Bonham drum part, note-perfect, but the feel was just not right. They learned the notes, but not how to play them. These are timing nuances that made these drummers unique, and why I'm usually disappointed when seeing cover bands.;)
Another aspect is trying to duplicate a specific tone, whether it be guitar, drums, or other instrument. You are forced to learn what factors affect the tonalities of an instrument, and what subtle changes do to that sound. Also, what combination of woods, strings, size dimensions, etc can be used to approximate a sound.
These experiences taught me how to listen, but I'm not sure how you translate that to a non-musician, so they know what to listen for. -
Okay, I've read this entire thread again. I also read the "Obervations of a cable test" thread again for the umpteenth time.
I still have one simple question especially you William (since you post this test a lot as a rebutal to subjectionists) . . . what make this "Obervations of a cable test" in any way a valid test?
It is so full of holes and I'll give a few examples;
1) it appears this test was not a scientifically or subjectively controlled test;
2) the listener was asked to identify which cable is which, this is not a good test, it should have been which sounds better to the listener (the fact that the listener couldn't identify which cable was which see #1;
3) the test was less than half-way completed i.e., 8 tries out of 20 (they just gave up!?! because he couldn't identify which cable was which on the first 8 tries, how do we know as the listener got more and more tries he wouldn't have found the differences as well as being about to state which cable he prefered;
4) going back to #1, not controlled, they went off to play pool for awhile which is absurd in that it takes the listener out of the listening and concentration mode;
5) the tester and listener both contradict the findings and admit that the test was not followed through and thus flawed;
6) many repliers to the OP of that thread punched holes in the methodology;
7) reading the test over and over again it seems like this was a half-hearted attempt by the tester in a truly uncontrolled and a not very well planned or thought out method to show no difference could be heard between the two speaker cables thus slanted to put the listener at a disadvantage.
There are many more but you get the idea.
So again how is this test in any way, shape or form a valid test?
IMHO this is test was even more flawed than the ABX Power Cable Test. -
I think the ABX test DK refers to has many problems.
These tests are very difficult to do because we all hate tests and get nervious during them.
When we get nervious we may not be able to listen well or critically which is required to determine differences in audio equipment.
I would suggest that any ABX audio test be design for the comfort of the person being tested.
Do the test with the following
-In your home listening room in your sweet spot with your equipment
-Let the person being tested use any music at any volume they want
-Let the person being tested play with the switcher box and components being compared for as long as they want
-Bottom line let the person being tested get used to the components being compared.
-During the ABX test let the person being tested play each component as long as required with any music they want.
What we are trying to determine with these ABX testing if these equipment changes are audable. Give the person being tested what they need to show they can hear the differnces. Just because I can't hear any differnces doesn't mean other people with better hearing and better training can hear differences.
I may have missed other items/requirements that the tester needs to have a fair test. -
These tests are very difficult to do because we all hate tests and get nervious during them.
People who enjoy the audio hobby and who enjoy listening to stereophonic music should not find audio evaluation difficult or unpleasant unless it is conducted in an unnatural manner in an unnatural environment.When we get nervious we may not be able to listen well or critically which is required to determine differences in audio equipment.
I would suggest that any ABX audio test be design for the comfort of the person being tested.
Do the test with the following
-In your home listening room in your sweet spot with your equipment
-Let the person being tested use any music at any volume they want
-Let the person being tested play with the switcher box and components being compared for as long as they want
-Bottom line let the person being tested get used to the components being compared.
-During the ABX test let the person being tested play each component as long as required with any music they want.
As I discussed in my ABX Power Cord Test inquiry (link), ABX (duo-trio balanced reference) tests are designed to find differences between two items. According to standard statistical mathematics and sensory science, for scientifically valid results:
1. An ABX (duo-trio balanced reference) test generally requires a subject population of at least 16 persons. Optimum subject population is at least 32 or more persons.
2. An ABX (duo-trio balanced reference) test which employs a subject size of less than 28 persons generates high rates of beta error (false negatives or "no differences between samples") in the results.
3. An ABX (duo-trio balanced reference) test must compare samples which are unknown (unfamiliar) to the test subjects.
4. An ABX (duo-trio balanced reference) test must use untrained test subjects.
The fundamental flaw in applying ABX tests to audio evaluation is in that people participating in ABX tests are searching for differences between equipment rather than conducting a performance evaluation against standard performance criteria (metrics).
In other words, ABX is fundamentally flawed because it compares gear to gear rather than comparing gear to performance standards. This is ludicrous and demonstrates an ignorance of the purpose of stereophonic music reproduction. This seems to be a concept that the "objectivists" and "scientists" cannot, or will not, grasp.
The reason for the existence of stereophonic audio equipment is the lifelike reproduction of a musical performance. The metrics that I, and others interested in accurate comparative analysis, use for evaluating audio gear are those which gauge how much sonic realism is generated. When I critically listen, I am not thinking about trying to find a difference between what I am currently hearing and what I heard earlier. I am thinking about perceiving and accurately describing, on paper, what is being perceived. I am focused on trying to obtain as much three dimensional stereophonic information from the sound stage as possible. Sometimes, a performance difference is immediately audible...but that is not due to me going and looking for it. This comes about as a natural consequence of experience, familiarity with my audio system and focused aural and tactile information gathering.
Sometimes, I do not become aware of performance differences until I compare listening placement charts and listening notes. I will then go back and re-listen, often repeatedly, to reconfirm the differences. In the cases of subtle differences, written documentation is critical. For example, one of the primary performance metrics of stereophonic music reproduction is image placement in the sound stage. However, how often do you see this very important metric discussed in ABX test results? How often do you see performance metrics of any kind discussed in ABX test results?What we are trying to determine with these ABX testing if these equipment changes are audable.
Yes...and that is one of the greatest tragedies of ABX tests: Using short term memory to gauge audible performance differences.
Now, if you say that you now want to move toward evaluator training and letting the evaluator spend long amounts of time with items under test, then by definition you are no longer doing true ABX testing, you are moving toward descriptive analysis techniques. You cannot "redeem", "refine", or "improve" ABX testing by "merging" descriptive analysis techniques into it. They are two different methodologies with two entirely different purposes:
1. By formal scientific definition, ABX testing is strictly used for noting differences between two unfamiliar samples of something and depends on short term memory and the use of untrained subjects.
2. By formal scientific definition, descriptive analysis is used for describing, and documenting, how close an item is to a formal standard. Rather than relying on short term memory, descriptive analysis depends on documentation and long term memory that is the result of training.
In the case of evaluating how close a piece of stereophonic audio gear or a stereophonic audio system comes to the standard of live music, between (1) and (2) which do you think is the better evaluative method?
I'd be willing to bet my next amp upgrade that the terms "stereophonic" and "stereophony" are not even in the vocabularies of most of the ABX "faithful". For those in which it is, I'd be willing to bet my next speaker upgrade that most of them could not give an accurate explanation of what the terms mean. If they really understood stereophony, they wouldn't faithfully and irrationally embrace an evaluation methodology that consistently produces absurd results due to the restriction of sensory data reaching the subject.
It is ironic that audiophiles are criticized for having "faith" that the differences they perceive in audio gear really exist. The irony is in the fact that the evaluative methods used by audiophiles are closer to a scientifically proven method that gives accurate, repeatable results (descriptive analysis) whereas the ABX method used by the "objectivists" and "scientists" is, by definition, not scientifically appropriate for the evaluation of the aural and tactile stimuli generated by stereophonic audio systems.
I'm still waiting on one of the "believers" to explain to me why the AV Science forum speaker cable test is a "good" and scientifically valid test. I'm concerned that I might have (must have) missed something...since this is supposed to be such a "good" test.
Such Good "Belief"
Keep The "Faith" Proud and loyal citizen of the Digital Domain and Solid State Country! -
Intelligent Audiophile wrote:The fundamental flaw in applying ABX tests to audio evaluation is in that people participating in ABX tests are searching for differences between equipment rather than conducting a performance evaluation against standard performance criteria (metrics).
This is IT in a nutshell. This is EXACTLY what the flaw is for those who try to differentiate between gear using standard ABX methods.
H9"Appreciation of audio is a completely subjective human experience. Measurements can provide a measure of insight, but are no substitute for human judgment. Why are we looking to reduce a subjective experience to objective criteria anyway? The subtleties of music and audio reproduction are for those who appreciate it. Differentiation by numbers is for those who do not".--Nelson Pass Pass Labs XA25 | EE Avant Pre | EE Mini Max Supreme DAC | MIT Shotgun S1 | Pangea AC14SE MKII | Legend L600 | BlueSound Node 3 - Tubes add soul! -
This is IT in a nutshell. This is EXACTLY what the flaw is for those who try to differentiate between gear using standard ABX methods.
H9
. . . I would also like to add that the ABX tests that I've read require the test subjects to identify each cable or piece of gear rather than log what differences there are (if any) and which is more pleasent to their ear. What does identifying the name brand of the cable or piece of gear being tested prove?
