A Pefect Example of how NOT to do a Study - Part One
A week ago, I posted an article that touched on a truly appalling study of autism and mercury, "Reduced levels of mercury in first baby haircuts of autistic children" (Holmes AS, Blaxill MF, Haley BE. Int J Toxicol. 2003 Jul-Aug;22(4):277-85).
On careful reading, I have determined that this study is so bad it deserves a more careful analysis. So, in the interest of public education, I have decided to dissect it here in my blog. Those of you who are squeamish may want to check back in a few days. OK, gloves on, protective eyewear in place...here we go!
Authors:
The first thing to look at in any study is who did it. Although some people persist in the egalitarian notion that a study should stand on its merits regardless of who did it, I have found it helpful to examine the authors. All authors have - believe it or not - a bias or preconceived notion of how the study should turn out. A few of them (very few, fortunately) have a history of letting this bias interfere with their conclusions or - even worse - the data.
In this study the authors are as follows:
Amy Homes: A radiation oncologist who turned to biomedical therapies of autism (especially mercury chelation) after having a child with autism. She had a practice in Baton Rouge, Louisiana for the treatment of autism, but that has apparently closed.
Mark Blaxill: A Senior VP at The Boston Consulting Group with expertise in global corporate strategy, Mark has an A.B. from the Woodrow Wilson School of Public and International Affairs at Princeton and an M.B.A. from Harvard Business School. He has no formal training or education in any of the sciences, but has been talking as though he did. He is also on the Executive Board of SafeMinds, an organization that states:
Fortunately, the study wasn't affected by this preconception (heavy sarcasm).
Boyd Haley: Professor and Chairman of the Department of Chemistry at the University of Kentucky at Lexington, he has an impressive list of publications, including three on the effects of mercury on tubulin. Unfortunately, he hasn't updated his web page to include this study. Nor has he mentioned on his University of Kentucky web page his research interests in autism. Strange.
Dr. Haley is also a part owner (47.6% of the shares) in ALT, Inc., a company which - in one of its two for-profit divisions,
In addition to providing this desperately needed service they also provide information about the dangers of mercury in dental amalgam and the thimerosal preservative in vaccines. Dr. Haley seems quite convinced that autism is caused by mercury.
Methods:
The basic idea of the study was to collect hair samples taken at the first haircut ("first baby haircut", in the study) and saved by parents for a group of autistic children and a group of non-autistic controls. These samples were then analyzed for mercury at a commercial laboratory that specializes in "toxicity" analysis.
There are a lot of problems with the methods in this study, but I will only hit the "high points", since some of the errors may seem tedious and nit-picking to non-scientists. Besides, an exhaustive listing of the errors would take me longer than it probably took them to write the article.
[1] Sample age - The median age at haircut was 17 - 18 months in both the autistic and control groups. The median year of birth was 1994 for both groups. The study was received for publication in October 2002. Although the authors fail to mention when the study was carried out, even assuming that it was done in 2001, the hair samples had been in storage under unknown conditions for a median of five and a half years.
[2] Calculation of "Maternal Exposures" - The mothers of the study subjects were interviewed for possible exposures to mercury from:
[a] Amalgam fillings
[b] RhoD (anti-Rh antibody) injections (given when the mother is Rh negative)
[c] Childhood vaccinations (presumably given to the children in the study)
[d] Fish consumption ("none", "little", "moderate" or "heavy") during pregnancy
To begin with, this information is provided - largely from memory - about events that occurred a median of seven (7) years and as long as sixteen (16) years previously.
Based on this rather questionable information, the authors then applied a formula - which they derived from the data - to calculate (remember this - it will come up later) the amount of mercury that the chilren were exposed to in the womb (in utero, for you purists).
Note - the authors did not measure the mercury intake of the mothers, they calculated it.
In addition, they used the data they collected in order to come up with the formula they used to calculate the mercury exposure - which they then used to explain the data they came up with. The same data they used to calculate the mercury exposure. See the circularity? Like a puppy chasing its tail. This is not good science - it is something my high school biology teacher would have marked me off for.
Tomorrow: Results and the startling Conclusion!
'Til then,
Prometheus.
On careful reading, I have determined that this study is so bad it deserves a more careful analysis. So, in the interest of public education, I have decided to dissect it here in my blog. Those of you who are squeamish may want to check back in a few days. OK, gloves on, protective eyewear in place...here we go!
Authors:
The first thing to look at in any study is who did it. Although some people persist in the egalitarian notion that a study should stand on its merits regardless of who did it, I have found it helpful to examine the authors. All authors have - believe it or not - a bias or preconceived notion of how the study should turn out. A few of them (very few, fortunately) have a history of letting this bias interfere with their conclusions or - even worse - the data.
In this study the authors are as follows:
Amy Homes: A radiation oncologist who turned to biomedical therapies of autism (especially mercury chelation) after having a child with autism. She had a practice in Baton Rouge, Louisiana for the treatment of autism, but that has apparently closed.
Mark Blaxill: A Senior VP at The Boston Consulting Group with expertise in global corporate strategy, Mark has an A.B. from the Woodrow Wilson School of Public and International Affairs at Princeton and an M.B.A. from Harvard Business School. He has no formal training or education in any of the sciences, but has been talking as though he did. He is also on the Executive Board of SafeMinds, an organization that states:
"Our mission is to end the health and personal devastations caused by the needless use of mercury in medicines."
Fortunately, the study wasn't affected by this preconception (heavy sarcasm).
Boyd Haley: Professor and Chairman of the Department of Chemistry at the University of Kentucky at Lexington, he has an impressive list of publications, including three on the effects of mercury on tubulin. Unfortunately, he hasn't updated his web page to include this study. Nor has he mentioned on his University of Kentucky web page his research interests in autism. Strange.
Dr. Haley is also a part owner (47.6% of the shares) in ALT, Inc., a company which - in one of its two for-profit divisions,
"...provides oral toxicity testing products and services and develops tests to aid in the diagnosis of neurological diseases."
In addition to providing this desperately needed service they also provide information about the dangers of mercury in dental amalgam and the thimerosal preservative in vaccines. Dr. Haley seems quite convinced that autism is caused by mercury.
Methods:
The basic idea of the study was to collect hair samples taken at the first haircut ("first baby haircut", in the study) and saved by parents for a group of autistic children and a group of non-autistic controls. These samples were then analyzed for mercury at a commercial laboratory that specializes in "toxicity" analysis.
There are a lot of problems with the methods in this study, but I will only hit the "high points", since some of the errors may seem tedious and nit-picking to non-scientists. Besides, an exhaustive listing of the errors would take me longer than it probably took them to write the article.
[1] Sample age - The median age at haircut was 17 - 18 months in both the autistic and control groups. The median year of birth was 1994 for both groups. The study was received for publication in October 2002. Although the authors fail to mention when the study was carried out, even assuming that it was done in 2001, the hair samples had been in storage under unknown conditions for a median of five and a half years.
[2] Calculation of "Maternal Exposures" - The mothers of the study subjects were interviewed for possible exposures to mercury from:
[a] Amalgam fillings
[b] RhoD (anti-Rh antibody) injections (given when the mother is Rh negative)
[c] Childhood vaccinations (presumably given to the children in the study)
[d] Fish consumption ("none", "little", "moderate" or "heavy") during pregnancy
To begin with, this information is provided - largely from memory - about events that occurred a median of seven (7) years and as long as sixteen (16) years previously.
Based on this rather questionable information, the authors then applied a formula - which they derived from the data - to calculate (remember this - it will come up later) the amount of mercury that the chilren were exposed to in the womb (in utero, for you purists).
Note - the authors did not measure the mercury intake of the mothers, they calculated it.
In addition, they used the data they collected in order to come up with the formula they used to calculate the mercury exposure - which they then used to explain the data they came up with. The same data they used to calculate the mercury exposure. See the circularity? Like a puppy chasing its tail. This is not good science - it is something my high school biology teacher would have marked me off for.
Tomorrow: Results and the startling Conclusion!
'Til then,
Prometheus.
posted by Prometheus at 3:51 PM
0 Comments:
<< Home