Finding Truffles amongst the Clods

ARI’s Parent Ratings of Behavioral Effects of Biomedical Interventions

For years I have advised people to avoid this site because it presents poor data in a poor fashion. The “data” is gathered (if that is the proper word) from an on-line survey (it started as a paper survey) of parents. The parents, by and large, are people who already have a favorable opinion of “biomedical interventions” (meaning: “non-drug”, or at least, not the FDA-approved use of “drugs”) and – by corollary – an unfavorable opinion of “mainstream” medical therapies for autism.

As a result, the “data” gathered is skewed in favor of “biomedical interventions” – as the results so clearly show.

There is also no attempt made to confirm the reported results or to control for the initiation of other therapies at the same time.

In addition, the arrangement of the categories of interventions – into “drugs”, “biomedical/non-drug/supplement” (with Pepcid and “chelation” listed as non-drugs) and “special diets” – biases the survey, as a PhD psychologist like Dr. Rimland (and his heir-apparent, Dr. Edelson) should know.

Finally, the way the “data” is “analyzed” is … well, unusual. They take a six-point scale (another oddity, with three favorable ratings, two unfavorable and one neutral) and bin the results into “got better” (top two favorable ratings), “no effect” (the “possibly helped” and “no definite effect” responses) and “got worse” (the two unfavorable responses).

They then effectively discard the “no effect” responses to come up with a “better:worse” ratio. This may have some utility, but a better rating would be to lump all the favorable responses together and compare that to the sum of the neutral and unfavorable responses. If you do that, then some of the current favorites start to look a bit less attractive:

Examples:

Antifungals: Diflucan -
ARI better:worse = 11:1 -- better:not better = 1.2:1

Secretin: Intravenous -
ARI better:worse = 6.7:1 - better:not better = 0.9:1

Vitamin A -
ARI better:worse = 23:1 -- better:not better = 0.7:1

Pepcid (non-drug?) -
ARI better:worse = 3.2:1 - better:not better = 0.4:1

Vit B6 and Magnesium -
ARI better:worse = 10:1 -- better:not better = 0.9:1


You might agree, after seeing those numbers, that the ARI “better:worse” rating is a bit misleading.

And what about that all-time autism treatment favorite: chelation? Well, ARI lumps it with “Detox. (Chelation)”:

Detox. (Chelation) -
ARI better:worse = 35:1 -- better:not better = 3.2:1

Still, if you were to believe in the numbers generated by this poorly designed survey, you might think that some of these treatments had something to offer. You might be right – you might also be wrong – there’s no way to tell from this “data” because it was gathered in such a poor way.

But there is something you can glean from the ARI Parent Ratings of Behavioral Effects of Biomedical Interventions (PRBEBI). You can find what the natural progression of autism might be.

You see, buried in the ARI-PRBEBI is a treatment that has been rigorously studied – a treatment that was extensively tested in a multi-center, placebo-controlled, double-blinded study. That treatment is secretin.

Secretin has been exhaustively tested for effectiveness in autism and found to be no better than placebo (i.e. no effect). This is not in question. Even the company that held the patent to make recombinant secretin, which could have earned them a lot of money if it had been an effective treatment for even 10% of children with autism, has admitted that secretin has no effect in autism.

Let me repeat that: Secretin has no effect in the treatment of autism.

But, if we look on the ARI-PRBEBI, we find that the parents who had secretin administered (intravenously) to their children rated it effective (“got better”) 48% of the time, ineffective 44% of the time (“no effect”) and saw their children worsen 7% of the time.

This is pretty much all you need to know about the validity of the ARI-PRBEBI. It rates secretin as an effective treatment for autism despite the fact that large, well-designed and well-funded studies have found secretin no better than placebo.

But wait, there’s more!

Since we now know that secretin is without positive effect (and the studies also found very few negative effects), this is a fair placebo trial for the natural progression of autism without treatment: 48% get better, 44% are unchanged (at least over the short time the parents waited for an effect from secretin) and 7% worsen.

Of course, this result also carries with it the limitations of the ARI survey, namely:

[1] The reported results are not confirmed by anything other than parental impressions.

[2] There is likely to be a confirmation bias, i.e. the parent population taking this survey is more likely to report improvement than worsening.

[3] This is certainly not random or evenly distributed parent or patient population.

[4] It is likely that other “treatments” were instituted at the same time as the secretin, which would confuse the issue.


Still, as a first approximation of what parents might experience as the untreated progression of autism, it isn’t bad. And it rather confirms the clinical impressions of physicians and psychologists who treat autistic children – that most of them show improvement over time.

Given that the time period over which improvement was looked for after secretin injection was probably a matter of weeks to months, the fact that 44% of the children showed “no effect” (i.e. no change) is also confirmation of the periods of stasis (no discernible developmental progression) that autistic children experience.

So, having tried my darnedest to derive something positive and useful from the ARI-PRBEBI, I leave you with the following summary points:

[1] The ARI Parent Ratings of Behavioral Effects of Biomedical Interventions (ARI-PRBEBI) is not a reliable gauge of how effective a treatment really is.

[2] The ARI Parent Ratings of Behavioral Effects of Biomedical Interventions (ARI-PRBEBI) can be used to demonstrate that a significant percentage of untreated autistic children can be expected to show noticeable improvement over a short time course (weeks to months).

This pretty much puts an end to the claim (oft-repeated but never substantiated) that, but for the brave use of “biomedical interventions”, autistic children would be stuck at the same developmental stage they were at the time of diagnosis. It should also help people realize that autistic children do continue to develop – to improve - even without treatment.

I can only hope that some people will take notice.


Prometheus

Note: Prometheus will be attending a conference of minor mythological figures next week and will not be able to moderate comments. Rest assured, when he returns, all pending comments will be dealt with in a firm but fair manner.