Discussion of the UpTone / J.Swenson EtherREGEN 'white paper'

On 3/12/2020 at 6:27 PM, Confused said:

 

What is "evidence based science?"

Back in the early 90's I was doing a University course for a post graduate diploma in occupational health and safety. We were required to design and conduct a research project of our own as one of the course requirements. One of the first steps in the design process was a review of the published research into the topic we were going to research for out project which meant doing a search for all of the research published in the peer reviewed journals plus any relevant book publications.

I was doing a project on how often computer operators should have their eyesight tested. I had a literature search conducted by the university's professional librarians. It turned up nothing in the peer reviewed journals and nothing in any book publication. The only documentation available on the topic of frequency testing for computer operators that I could find was a paper commissioned by the Australian federal health and safety body. This paper was authored by a professor of ophthalmology and a professor of optometry at Australian universities, both well respected researchers in their fields. It contained a recommendation for the frequency to testing but while it cited a lot of references for other parts of the paper, there was no research cited for the actual recommendation itself and no mention of anyone having done any studies on that specific question.

I did my study, a small sample of 75 people in the organisation I worked for who were following the recommendation in that paper. My findings supported the recommendation which was that computer operators under 40 years of age should be tested when they reported symptoms of eye problems, those over 40 years of age should be tested every 2 years or when they reported problems.

My supervisor for the project, a lecturer and researcher in my university's optometry department, suggested that I present the study at an optometry science conference in Melbourne which I did. The conference was chaired by the professor of optometry who had co-authored the recommendation who had been running a long study into eyesight issues for computer operators which had resulted in several publications. He spoke to me afterwards about my study and said he'd been quite interested to see my results. I told him I'd read his paper with the recommendation which hadn't cited any research and asked  him where he and the second author had found the data on which they had based the recommendation. He replied that they had none, that it was based on his and his co-author's clinical experience which was that problems in people under 40 always had symptoms but that after 40 years of age the dominant problem tended to be age related deterioration of sight which was gradual. People adapted to that by increasing their reading and/or screen distance and problems didn't show until their arms got too short or they couldn't push the screen far enough away but testing on a 2 yearly basis for those people could identify the deterioration and allow the prescription of spectacles which allowed them to continue reading or screen based work at normal distances and help avoid eyestrain. There was research available into how age affected eyesight problems but no one had done any research into how often computer operators should get their eyes tested. What the 2 authors of the recommendation had done was simply to apply the evidence related to eyesight problems in general to a specific group of people based on their own clinical experience but there was no evidence for a recommendation in relation to computer operators as a specific group of people who were looking at computer screens for the whole of each working day.

Was the recommendation "evidence based science"? There was no evidence to specifically support the recommendation but I think it was evidence based science, even though it most definitely had never been peer reviewed. My study was published in a peer reviewed health and safety journal 15 months later.

In general, if you want peer reviewed studies you have to wait at least 18 months after the research is done for the results to be published and add a year or two to that for book publication. Research can move fast and people in a field are often well aware of research results long before they're published in a peer reviewed publication. Absence of peer reviewed data doesn't necessarily mean that the data isn't there, it could simply still be in the pipeline for publication and will appear in a year or two's time but it may already be well known in the actual field concerned.

You can wait for the peer reviewed studies to appear if you like but when I retired 10 years after I did my research and 8 or so years after it was published, I still hadn't  heard of any other published paper on the topic but mine and I doubt there will be any. There were extremely sound reasons for the recommendation and it probably wasn't worth any professional researcher designing and conducting a study to prove or disprove the validity of the recommendation which was based on well accepted professional experience. It made a great student project for me and I learnt a lot about how research is done and how to draw valid conclusions from your data, and it went some way to validating the published recommendation but I doubt any professionals doubted the validity of the published recommendation, even though no research was cited, and I doubt any professionals found my study provided essential evidence to support the recommendation.

The test of whether something works or not is whether people using it find it works or not. Research showing how and why it works is very useful but its real value is not because it proves that something works, it's because that research provides the information that helps designers and manufacturers to build products that can work even better, and to build those products more quickly and more easily than they could without that research.

Weeks after the Wright brothers made the first powered flight at Kittyhawk, a respected scientist used the peer accepted research at the time to write a paper which "conclusively proved" that powered flight was impossible  and that the  claims of the Wright brothers' flight was a hoax. Relying on peer reviewed science rather than actual observations of whether something works or not can lead to the wrong conclusions. Even today, research which goes through peer review and is initially accepted gets pulled from publication because of errors in the research that wasn't picked up in peer review. Peer review isn't an infallible guarantee that the research is correct. It's definitely a necessary part of the scientific process but just as research can get it wrong, peer review can also get it wrong.

I would really like to see peer reviewed evidence for the ER's effectiveness but it could be a long time coming and, in the end, the real proof of whether or not the ER works is what it achieves in practice. So far the user reports are almost completely positive and that is evidence as well. 

@David A - there is certainly some evidence that you quoted the wrong person in the post above.   🙂  Maybe you were confused?  Oh hang on, maybe that’s me?  

32 minutes ago, David A said:

People adapted to that by increasing their reading and/or screen distance and problems didn't show until their arms got too short or they couldn't push the screen far enough away but testing on a 2 yearly basis for those people could identify the deterioration and allow the prescription of spectacles which allowed them to continue reading or screen based work at normal distances and help avoid eyestrain. There was research available into how age affected eyesight problems but no one had done any research into how often computer operators should get their eyes tested.

Well David, given the length of your post and its long paragraphs I found I had to sit about 3 feet away to read it. Just another data point from 57 year old eyes. 

32 minutes ago, David A said:

The test of whether something works or not is whether people using it find it works or not. Research showing how and why it works is very useful but its real value is not because it proves that something works, it's because that research provides the information that helps designers and manufacturers to build products that can work even better, and to build those products more quickly and more easily than they could without that research.

Indeed. I promise you all, @JohnSwenson did not just drop over $14K on an expensive phase-noise analysis system just to appease the skeptics out there.  Certainly it will be useful (from a marketing perspective) to produce measurements proving the efficacy of the EtherREGEN--a product that was designed based on theories (based on decades of chip-level design  experience) which so far have been validated mostly by 1,000+ sets of ears. The reason he bought the unit (and the reason I have funded other measurement gear projects of his) is so that he can get to the bottom of some of this and use what is learned on future projects (of which we have several on the drawing board).

There are some other threads--here at AS and elsewhere--where they endless ridicule the very concept of upstream phase-noise causing ground-plane bounce and thus jitter at the clock pin of a DAC. Until we produce measurement proof there really is little point in engaging further with those folks, especially as the arguments (on both sides) have become repetitious.

I am reminded a bit of Shunyata Research, as they--for the past 23 years--have quietly been developing, producing, and selling a series of highly unusual products based on the work of research scientist Caelin Gabriel. While the skeptics and naysayers used to ridicule Shunyata and lump them in with all manner of fuzzy/voodoo audio products, Caelin and his associates kept going and developed test techniques to validate and prove their work. Their clients always enjoyed the products and most don't care about the measurements. But for any firm to continue at the leading edge, there must be both knowledge and the tools to develop with. And now some Shunyata power products are used in the medical field, to reduce noise in electrophysiology surgery equipment.

4 hours ago, David A said:

 

What is "evidence based science?"

 

Back in the early 90's I was doing a University course for a post graduate diploma in occupational health and safety. We were required to design and conduct a research project of our own as one of the course requirements. One of the first steps in the design process was a review of the published research into the topic we were going to research for out project which meant doing a search for all of the research published in the peer reviewed journals plus any relevant book publications.

 

I was doing a project on how often computer operators should have their eyesight tested. I had a literature search conducted by the university's professional librarians. It turned up nothing in the peer reviewed journals and nothing in any book publication. The only documentation available on the topic of frequency testing for computer operators that I could find was a paper commissioned by the Australian federal health and safety body. This paper was authored by a professor of ophthalmology and a professor of optometry at Australian universities, both well respected researchers in their fields. It contained a recommendation for the frequency to testing but while it cited a lot of references for other parts of the paper, there was no research cited for the actual recommendation itself and no mention of anyone having done any studies on that specific question.

 

I did my study, a small sample of 75 people in the organisation I worked for who were following the recommendation in that paper. My findings supported the recommendation which was that computer operators under 40 years of age should be tested when they reported symptoms of eye problems, those over 40 years of age should be tested every 2 years or when they reported problems.

 

My supervisor for the project, a lecturer and researcher in my university's optometry department, suggested that I present the study at an optometry science conference in Melbourne which I did. The conference was chaired by the professor of optometry who had co-authored the recommendation who had been running a long study into eyesight issues for computer operators which had resulted in several publications. He spoke to me afterwards about my study and said he'd been quite interested to see my results. I told him I'd read his paper with the recommendation which hadn't cited any research and asked  him where he and the second author had found the data on which they had based the recommendation. He replied that they had none, that it was based on his and his co-author's clinical experience which was that problems in people under 40 always had symptoms but that after 40 years of age the dominant problem tended to be age related deterioration of sight which was gradual. People adapted to that by increasing their reading and/or screen distance and problems didn't show until their arms got too short or they couldn't push the screen far enough away but testing on a 2 yearly basis for those people could identify the deterioration and allow the prescription of spectacles which allowed them to continue reading or screen based work at normal distances and help avoid eyestrain. There was research available into how age affected eyesight problems but no one had done any research into how often computer operators should get their eyes tested. What the 2 authors of the recommendation had done was simply to apply the evidence related to eyesight problems in general to a specific group of people based on their own clinical experience but there was no evidence for a recommendation in relation to computer operators as a specific group of people who were looking at computer screens for the whole of each working day.

 

Was the recommendation "evidence based science"? There was no evidence to specifically support the recommendation but I think it was evidence based science, even though it most definitely had never been peer reviewed. My study was published in a peer reviewed health and safety journal 15 months later.

 

In general, if you want peer reviewed studies you have to wait at least 18 months after the research is done for the results to be published and add a year or two to that for book publication. Research can move fast and people in a field are often well aware of research results long before they're published in a peer reviewed publication. Absence of peer reviewed data doesn't necessarily mean that the data isn't there, it could simply still be in the pipeline for publication and will appear in a year or two's time but it may already be well known in the actual field concerned.

 

You can wait for the peer reviewed studies to appear if you like but when I retired 10 years after I did my research and 8 or so years after it was published, I still hadn't  heard of any other published paper on the topic but mine and I doubt there will be any. There were extremely sound reasons for the recommendation and it probably wasn't worth any professional researcher designing and conducting a study to prove or disprove the validity of the recommendation which was based on well accepted professional experience. It made a great student project for me and I learnt a lot about how research is done and how to draw valid conclusions from your data, and it went some way to validating the published recommendation but I doubt any professionals doubted the validity of the published recommendation, even though no research was cited, and I doubt any professionals found my study provided essential evidence to support the recommendation.

 

The test of whether something works or not is whether people using it find it works or not. Research showing how and why it works is very useful but its real value is not because it proves that something works, it's because that research provides the information that helps designers and manufacturers to build products that can work even better, and to build those products more quickly and more easily than they could without that research.

 

Weeks after the Wright brothers made the first powered flight at Kittyhawk, a respected scientist used the peer accepted research at the time to write a paper which "conclusively proved" that powered flight was impossible  and that the  claims of the Wright brothers' flight was a hoax. Relying on peer reviewed science rather than actual observations of whether something works or not can lead to the wrong conclusions. Even today, research which goes through peer review and is initially accepted gets pulled from publication because of errors in the research that wasn't picked up in peer review. Peer review isn't an infallible guarantee that the research is correct. It's definitely a necessary part of the scientific process but just as research can get it wrong, peer review can also get it wrong.

 

I would really like to see peer reviewed evidence for the ER's effectiveness but it could be a long time coming and, in the end, the real proof of whether or not the ER works is what it achieves in practice. So far the user reports are almost completely positive and that is evidence as well. 

Excelllent post. 

As someone who spent his entire career as a professional scientist, and am first-author of papers published in Science, The Proceedings of the National Academy of the Sciences, American Journal of Human Genetics, Blood, etc., I agree with your points. 

Oh, and BTW, the Principia Mathematica was not peer-reviewed. 

17 hours ago, Superdad said:

Well David, given the length of your post and its long paragraphs I found I had to sit about 3 feet away to read it. Just another data point from 57 year old eyes. 

 

 

Indeed. I promise you all, @JohnSwenson did not just drop over $14K on an expensive phase-noise analysis system just to appease the skeptics out there.  Certainly it will be useful (from a marketing perspective) to produce measurements proving the efficacy of the EtherREGEN--a product that was designed based on theories (based on decades of chip-level design  experience) which so far have been validated mostly by 1,000+ sets of ears. The reason he bought the unit (and the reason I have funded other measurement gear projects of his) is so that he can get to the bottom of some of this and use what is learned on future projects (of which we have several on the drawing board).

 

There are some other threads--here at AS and elsewhere--where they endless ridicule the very concept of upstream phase-noise causing ground-plane bounce and thus jitter at the clock pin of a DAC. Until we produce measurement proof there really is little point in engaging further with those folks, especially as the arguments (on both sides) have become repetitious.

 

I am reminded a bit of Shunyata Research, as they--for the past 23 years--have quietly been developing, producing, and selling a series of highly unusual products based on the work of research scientist Caelin Gabriel. While the skeptics and naysayers used to ridicule Shunyata and lump them in with all manner of fuzzy/voodoo audio products, Caelin and his associates kept going and developed test techniques to validate and prove their work. Their clients always enjoyed the products and most don't care about the measurements. But for any firm to continue at the leading edge, there must be both knowledge and the tools to develop with. And now some Shunyata power products are used in the medical field, to reduce noise in electrophysiology surgery equipment.

 

Great post. 

I'll just add some additional info for context here. In the medical imaging arena, where the products of Shunyata Research's sister company, Clear Image Scientific, have developed a range of products that have been a breakthrough in digital medical imaging, the doctors who use these products don't get into "arguments" about whether a power can make a difference, because....they use these products to save people's lives. 

7 hours ago, Tintinabulum said:

Not sure ablation is "saving peoples lives", a little emotional. Looking at your kit, you've already bought off on the theory. 

A really wonderful gift was given to us:  ears that can hear - and hear with an acuity that still isn't fully understood by science.   Our gift makes it easy to hear the benefits of their products.

Quote

 

I assume that noise reduction occurs after the AC has entered the hifi gear. A lot happens when converting to DC and after, I'm not absolutely convinced that the noise makes it through.

Why not get some hands-on experience with these products?  

Quote

 

I would assume hospitals are noisy places when it come to interference, both RFI etc plus the amount of electrical gear plugged into their AC. 

 

Unconvinced personally. I wonder whether the hospital were enriched for this.

Are you really questioning the ethics of all those who allowed their names to be used here?  Do you really think that medical professionals would put their reputations at risk by accepting payments for endorsements?  Are they even permitted to accept "enrichment"?

Maybe Occam's razor applies here:  

1) these endorsements Clear Image Scientific has received are legitimate

2) those who purchase Shunyata products do so because they deliver real benefits that can be heard

3) a person who would discount both #1 and #2 has likely made the choice to remain unconvinced

On 3/17/2020 at 9:35 AM, David A said:

What is "evidence based science?"

"What is "evidence based science?" Admittedly glib but it is -  science based on evidence. As a medical scientist I have participated in research and more relevantly have been a 'consumer of research' for 40 years. If not a chef , then a food critic, if you will.

In the early days of my training my impression was that there was considerable stock placed in what others were doing in so called "centers of excellence". I saw a shift in emphasis to less authority based beliefs to a more "scientific" approach, based on "evidence"..."evidence based Medicine".

As I attended conferences around the world I then observed an interesting phenomenon. There was less discussion/debate about who was doing what where, but rather what was the evidence. I considered this a good thing. I then noticed that discussion/debates were no less robust and ironically people were arguing about.....the evidence!.....and what it meant, was it relevant, accurate,where did it come from, you name it.

Medical evidence is now rated for quality but even so it is not supposed to be proof unless in the instance of a counter example disproving the null hypothesis...(at that point in time). Evidence is data points to help determine probability.

Evidence is malleable, it changes with the results of new experiments and is only as good as the last experiment and to the extent the experiment was good (done well).

Evidence (factual data points) supports or does not support hypothesis which is based on theory.The extent and analysis of that support is very often subjective interpretation.Woozle effect factoids is not evidence.

Importantly, evidence is great to the extent that we have it (ie it exists - an absence of evidence is not evidence of absence), it is of quality, and firmly points to supporting or rejecting hypothesis, and is interpreted in an unbiased fashion.

Kudos to Uptone Audio for seeking evidence through observations and wishing to further explore the possibilities with test measures of hypothesis in the advancement of Audio.

Lastly, evidence is still very important.

“The only way to have real success in science, the field I'm familiar with, is to describe the evidence very carefully without regard to the way you feel it should be. If you have a theory, you must try to explain what's good and what's bad about it equally. In science, you learn a kind of standard integrity and honesty.”― Richard Feynman, What Do You Care What Other People Think?

/rant

I do not want this post to be taken as a criticism of EtherREGEN, Switch of which I am happy owner since the second manufacturing batch and for whose research effort I congratulate Alex and John.  It is a very good device that clearly improved my AQvox Switch.  It is evident that ADIM is doing something since the input-output connection to the AB sides is better than both on the A side. However, the “total” isolation of what happens upstream of the EtherREGEN, frankly I think that it has not been  got.  ALWAYS that I have made an improvement in the only device I have upstream, which is my Router (LPS, best power cable to LPS, connection of the LPS power cable to a Denali and Best ethernet cable between Router and EtherREGEN), the sound of the  system has improved.  I only find two possibilities to this, either the ADIM only partially isolates, or from the Router comes some kind of digital “garbage” for whose elimination, the EtherREGEN has not been designed.

As a user I encourage Alex and John to continue the investigation.  I think EtherREGEN has been a very good first step, but in the future we will get much better Switch, this is only beginning.

Regards