By Daniel W. Mruzek, PhD, BCBA-D, Keith Gordon, BA & Eric Larsson, PhD, LP, BCBA-D
The scientific method is a wonderfully powerful tool for investigating potential treatments for individuals with autism spectrum disorder (ASD). By using strategies such as randomization of participants, “blinded” raters,
and standardized outcome measures, researchers help to elevate our evaluations of potential ASD treatments above the limitations of mere conjecture and personal testimony. For example, behavioral interventions, such as early intensive behavioral intervention, have been established as effective interventions for young children with ASD (though much remains to be discovered regarding how to maximize outcome for the individual child; Orinstein, et al., 2014, Peters-Scheffer, Didden, Korzilius, & Sturmey, 2011).
Not to Be a Wet Blanket, But…
Can science be used to prove that an intervention is not effective? We were reminded of this question recently when we reviewed a well-controlled study on the use of weighted blankets, conducted by Gringas and colleagues (Gringas, Green, Wright, Rush, Sparrowhawk, Pratt, Allgar, et al., 2014). The researchers set out to investigate the effectiveness of a common intervention for individuals with ASD, the use weighted blankets in treating sleep problems. To accomplish this, they conducted a randomized, placebo-controlled crossover design study. This is a study in which participants are randomly assigned to an experimental condition (and receive the active treatment) or a comparison condition (and receive a placebo or inactive treatment). Then, at a predetermined point in time, each participant is switched to the other group. At each stage, key measures of treatment outcome are collected. In this study, 73 children and youth aged between 5 – 16 years with severe sleeping problems were randomized to either a weighted blanket condition or to an otherwise identical blanket of usual weight at bedtime. Each type of blanket was used for two weeks prior to the crossover to the other condition. The researchers found no differences in total sleep time, as measured by electronic monitoring of movement while in bed (i.e., actigraphy) during each two-week period. Furthermore, the researchers found no group differences in total sleep time, how long it took participants to fall asleep or the proportion of time spent in bed asleep across the two conditions (i.e., parent reports in daily “sleep diaries” and parent questionnaires). They concluded that the weighted blanket intervention in their study was “no more effective than …(a regular blanket) at improving TST (total sleep time) or any other commonly measured parameter of sleep quantity or quality” (p. 303).
What’s This Got to Do With Proving That a Treatment Is Not Effective?
Does this study prove that weighted blanket intervention is not effective for the population of children with ASD? Well, in a word, no. However, that is only part of the story. In one sense, this is a question of generalizability – the degree to which results from a study are relevant for people other than the participants in the study (e.g., other individuals with ASD). In this instance, we are left to conjecture about individuals with ASD who are younger or older than the 5 – 17 age range, have unique or at least different sleep or related difficulties. Perhaps there are some children who have very specific and rare disorders who were not adequately sampled in this study. And, of course, the results of this one study may be a fluke, and other scientists may not replicate these findings. One might also quibble by saying that perhaps the measurement system was inadequate to detect the effect – a measurement system must be shown to be able to detect an effect before its failure can be taken as useful data. Thankfully, treatment studies do not exist in isolation; rather, each study is part of a broader record of scientific inquiry. The results of any one study can only be interpreted in the context of findings from other studies. In the case of weighted blankets, it is notable that there is virtually no scientific evidence that they are helpful in promoting sleep; therefore, it is reasonable to suspect- but not prove- that weighted blankets are ineffective in helping children with ASD get a good night’s rest.
OK, But What About Bigfoot?
Stepping beyond the question of generalizability, there is a more fundamental point to be made, and this brings us to Bigfoot: it is virtually impossible to prove that something – anything – does not exist. How would you, dear reader, prove that we did not see a Bigfoot in our yards last night? You might point out that it is highly unlikely and that we must have each seen something else, like a neighbor getting firewood. But we could simply say, “you might not believe us, but we saw Bigfoot!” You may point out that we should have taken a picture, but, of course, we may simply report that, “by the time I got my iPhone, he was gone!” (or one of us would have a grainy picture that looks a bit like a big shrub). In other words, proving that something does not exist- whether it is Bigfoot or the benefits of a weighted blanket- is a maddening exercise – and, really, one to be avoided. Yet, when marketers of ASD interventions lean on the fact that there is no evidence to suggest that their product does not work, they are asking you to engage in this exercise. That is why we assert that it is incumbent upon them to prove that their product does work. This is the standard to keep in mind when looking at treatment evidence.
How Does This Relate to My Family Member’s ASD Intervention?
Well, thankfully, it is a standard of scientific practice that proponents of proposed ASD interventions prove that they are effective. If there is no scientific record of treatment effectiveness, the practitioner (or marketer) should be prepared to demonstrate how they would evaluate possible effectiveness for your family member if their treatment were to be employed (e.g., by using single subject design methodology to measure change in relevant behavior). As consumers of ASD interventions, families can maintain a healthy skepticism – a cornerstone of the scientific method- when confronted with potential ASD treatments, whether they are behavioral, biological or “sensory integrative.”
That’s Real Money!
By the way, a quick Google search using the search term “weighted blanket” reveals the efforts of several companies in the weighted blanket business. For example, one marketer of weighted blankets (www.sensorygoods.com) price their blankets between 56.99 to at least 190.99 US dollars (with additional costs for special features but, thankfully, free postage and handling). They report that their blankets “continue to change lives by giving families and individuals the sleep they desire and deserve” and “regulate a normal sleeping pattern as well as calm anxiety and stress.” As consumers with limited financial resources, what might be an appropriate first response to these marketers? We recommend, “Prove it.”
References
Gringas, D., Green, D., Wright, B., Rush, C., Sparrowhawk, M., Pratt, K., Allgar, V., Hooke, N., Moore, D., Zaiwalla, Z., Wiggs, L. (2014). Weighted blankets and sleep in autistic children – a randomized controlled trial. Pediatrics, 134, 298-306. http://goo.gl/CCNzuB
Orinstein, A., Helt, M., Troyb, E., Tyson, K. E., Barton, M. L., Eigsti, M., Naigles, L., Fein, D. A. (2014). Intervention for optimal outcome in children and adolescents with a history of autism. Journal of Developmental and Behavioral Pediatrics, 35, 247-256. http://goo.gl/ym7Zsx
Peters-Scheffer, N., Didden, R., Korzilius, H., & Sturmey, P. (2011). A meta-analytic study on the effectiveness of comprehensive ABA-based early intervention programs for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 5, 60-69. http://goo.gl/2gxWDk
Citation for this article:
Mruzek, D., Gordon, K., & Larsson, E. (2014). Focus on Science: Can scientists prove that a treatment does not work? And… is Bigfoot real? Science in Autism Treatment, 12(1), 12-14.
