Mandell, D. S., Stahmer, A. C., Shin, S., Xie, M., Reisinger, E., & Marcus, S. C. (2013). The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention. Autism, 17(3), 281-295.
Reviewed by: Ethan Eisdorfer,
Rutgers University Graduate School of Applied and Professional Psychology.
Why research this topic?
In general, intervention research that is conducted in highly controlled settings, such as university based laboratories and clinics, shows stronger results than research in community settings, such as schools or clinics. This “implementation cliff” is the focus of a growing body of scientific study (Weisz, Ng & Bearman, 2014; Weisz et al., 2005). One contributor to this difference in outcomes may be how well the interventions are conducted (“treatment fidelity”). The current study sought to answer the question, “If previous research shows that a program is effective when it is conducted very well (by its designers), how effective is that program when it is not executed as well as the designers intended (as it might be in many community settings)?”
What did the researchers do?
Thirty-three teachers and their classrooms, containing 119 students with autism, were randomly assigned
to be trained in either Strategies for Teaching based on Autism Research (STAR) or Structured Teaching (ST). STAR is an ABA approach that incorporates three empirically supported teaching strategies: discrete trial instruction, pivotal-response training, and functional routines. ST is based upon the Treatment and Education of Autistic and Communication Handicapped Children (TEACCH) program, which aims to maximize student independence by using visual cues alongside verbal teaching and clearly signaling transitions.
What did the researchers find?
Overall, STAR and ST appeared to be about equally effective. Students in both programs improved from pre- to post-intervention. These findings could indicate either that both programs were beneficial or that students would have gotten better regardless of intervention. Interestingly, treatment fidelity was low (57% for STAR and 48% for ST, compared to 80% or more, which would be expected in research settings). Also of interest, STAR appeared most effective at either high or low levels of fidelity, while ST performed best at moderate levels of fidelity.
What are the strengths and limitations of the study?
This study appears to have been conducted rigorously, but nevertheless had some limitations. First, although ST was intended as a contrast to STAR, the researchers found that these programs had considerable overlapping content, which might account for the similarity in outcomes. Second, although the randomization of students to interventions produced groups that were comparable on most preintervention variables, the ST group contained a disproportionate number of Latino students. Finally,
fidelity was measured via monthly videos, which may have been too infrequent to accurately capture teachers’ implementation of STAR and ST.
What do the results mean?
This study provides evidence that students with autism can improve in behaviorally-based teaching programs, such as STAR and ST, in general public school settings, even when treatment fidelity is imperfect. However, it is unclear whether students improved more than they would have without specialized instruction or whether higher fidelity would have produced more favorable outcomes.
Citation for this article:
Eisdorfer, E. (2016). Research review: The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention. Autism Science in Autism Treatment, 13(2), 13-14.
