Sneed, L., Little, S. G., & Akin-Little, A., (2023). Evaluating the effectiveness of two models of applied behavior analysis in a community-based setting for children with autism spectrum disorder. Behavior Analysis: Research and Practice. 23(4), 238-253. https://doi.org/10.1037/bar0000277

Reviewed by:
Eric Larsson, PhD, LP, BCBA-D, Lovaas Institute Midwest; Mary Jane Weiss, PhD, BCBA-D, LBA, Endicott College; and Lori Mastrogiacomo, MA, BCBA, LBA, Endicott College

Why research this topic?

Research Synopses - Psuchological, Educational and Therapeutic InterventionsEarly intensive behavioral intervention (EIBI) can make a tremendous difference in the lives of young children with autism. Since the publication of Lovaas (1987), it has been clear that EIBI can change lives and developmental outcomes. Still, many questions continue to be posed about the nature of the intervention. How many hours are needed per week? For how many years is treatment at a high-intensity level necessary? What treatment procedures are most effective? Are there certain child characteristics best matched to certain kinds of treatment?

The article by Sneed et al. (2023) focuses on the impact of who delivers the treatment. Specifically, the authors compared parent-mediated intervention to paraprofessional-mediated intervention. This study focused on differential outcomes when the primary interventionist was either a paraprofessional or a parent. They followed 106 children diagnosed with ASD, who were between the ages of 3 and 7.

The researchers compared two models of ABA intervention. Such work is very important, as it represents an attempt to identify the most effective elements of treatment. As noted, many questions remain about the elements of the most effective approaches within EIBI. Systematic research is an excellent means of identifying effective features of treatment. This study examines an important component of intervention, the interventionist.

What did the researchers do?

Specifically, this study used a between-groups design to compare the very low intensity of one-to-one treatment across two independent variables. The independent variables differed by format of service delivery: parent-mediated ABA vs. paraprofessional-mediated ABA. Both groups received fewer hours per week and fewer months of intervention than the control groups in the accepted high-quality EIBI outcome studies. The results were based on an average treatment duration of 20 months and gains were much lower than in the literature of high-quality studies of EIBI.

The data below summarize the level of intervention for the two independent variables “Paraprofessional Mediated ABA group” and “Parent Mediated ABA group.”

Average age at intake: 4.4 years, with SD of 1 year.

Average intervention duration: 20 months, with SD of 13 months.

Paraprofessional Mediated ABA group:

“BCBA” – 0.6 hrs per week

“Mid-level clinician” – 1.9 hrs per week

“Paraprofessional” – 6.0 hrs per week

Total – 8.5 hrs per week

Parent Mediated ABA group:

“BCBA” – 1 hr per week

“Mid-level clinician” – 2.1 hrs per week

“Paraprofessional” – 0 hrs per week

Total – 3.1 hrs per week

What did the researchers find?

The researchers reported child progress on two dependent variable measures that demonstrate a range of skill acquisition in the learner- the Vineland (a measure of adaptive skills) and the VB-MAPP (a measure of curricular mastery). They note that “the Vineland was always completed through an online system by the client’s parent(s).” The BCBA assigned to the cases conducted the VB-MAPP. In both cases, there was no interobserver agreement (IOA) on the accuracy of the assessment; no independent ratings occurred. This raises questions about the objectivity and accuracy of the assessments.

The authors reported some advantages to the parent-mediated group. However, alarmingly, the comparison could have been affected by the parent-mediated group getting more direct training on behavior identification, and more motivation to report their own progress. Still, gains were modest and not representative of a large impact of treatment.

On the Vineland, the Parent Group gains barely exceeded the 90% confidence interval in only the ABC and the Communication scores and did not reach the normal range in any scores.

(See Table 3 below)

Table 3 Comparison of Parent-Mediated ABA and Paraprofessional-Mediated ABA on Vineland-3 Domains
Parent-mediated ABA Paraprofessional-mediated ABA
Baseline Discharge Baseline Discharge
Standard Scores M M Gain 90% Confidence Interval +/- M M Gain 90% Confidence Interval +/-
ABC 74.16 81.06 6.9 5 74.27 75.92 1.65 5
Communication 73.49 83.04 9.55 7 73.1 77.56 4.46 7
Daily living skills 78.5 83.43 4.93 7 77.4 79.25 1.85 7
Socialization 73.74 80.61 6.87 8 76 76.13 0.13 8

 

On the VB-MAPP, the Paraprofessional Group achieved more milestones, reaching 51% of the total milestones, while the children in the Parent Group only achieved 42% of the milestones.

(See Table 5 below).

Table 5 Comparison of Parent-Mediated ABA and Paraprofessional-Mediated ABA on VB-MAPP
Parent-mediated ABA Paraprofessional-mediated ABA
Baseline Discharge Baseline Discharge
VB-MAPP M M Gain Percent of Total Possible at Discharge M M Gain Percent of Total Possible at Discharge
Total Score 42.91 71.09 28.18 42% 53.36 86.96 33.6 51%

At first glance, this paper looks like it will add to the understanding of effective ABA intervention for children with autism. However, the actual results were based on a comparison of two low-intensity treatment options. This significantly lowers the impact of the study, as low-intensity treatment has been shown to be ineffective, and to result in changes that are not meaningful.

What are the strengths and limitations of the study?

The study has very significant limitations. Several common elements of quality research are absent from this article. For example, there is no reported inter-observer agreement or procedural fidelity data. The lack of inter-observer agreement data prevents us from having confidence in the findings, as subjective biases were not controlled for. Furthermore, it is not possible, in the absence of procedural integrity data, to know if the interventions were followed as designed.

Also, in terms of fitting into existing literature, the authors compared interventions at a much lower level of intensity than is commonly accepted as necessary. In addition, elements of the intervention were different enough to introduce additional differences between the treatments.  All in all, many of these issues prevent us from making conclusions about the study. Indeed, it would need to be replicated with all these quality elements and safeguards against bias in place. However, given the differences noted from conventional practice, such a replication may not be meaningful.

In addition to low hours, this could also have resulted in counterproductive treatment effects due to low levels of staff training. In other words, insufficient training leads to ineffective treatment. Furthermore, there is nothing reported to indicate that the ITPs addressed the meaningful goals of each family, and the low hours of contact suggest that comprehensive goals weren’t addressed. Hence, it is not likely that these treatments were individualized at the level that is characteristic of the field of ABA.

A major concern is that this program is the threat to external validity, given that the authors have repeatedly published claims that low intensity ABA is sufficient. In this study the authors began with a total participant pool of 3,815, which after applying exclusions, was cut down to a total remaining of 106 participants. Some of their exclusion variables therefore have a strong risk of bias to external validity. For example, since age was one reason for exclusion, these results can’t be generalized to other ages. Second, since lack of documented assessments was another reason for exclusion, the applicability to other children is unknown. Perhaps most importantly, since high levels of treatment were excluded, these results can’t be generalized to high-intensity treatment.

What do the results mean?

The results should be interpreted with caution, given the limitations in experimental design and the concerns raised about the interpretations of the data. While it is very important to examine elements of treatment that may influence outcomes, this study does not systematically examine the variables in a way that facilitates conclusions. It is hoped that future research will build on existing research so that more clarity on the effective elements of intervention can be achieved. At this time, it appears that more research is needed to compare parent-mediated and paraprofessional-mediated treatment. It is also essential that we examine the reports of findings, to ensure that data are interpreted judiciously, with caution, and in the context of limitations noted. It is important that the field tackle these questions from the lens of science, and that data are presented in the context of scientific inquiry, control for external factors of influence, and attention to issues such as quality intervention, socially significant outcomes, and meaningful changes.

References (if other articles were cited in the article):

Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3-9.

Sparrow, S. S., Cicchetti, D. V., & Saulnier, C. A. (2016). Vineland adaptive behavior scales, third edition (Vineland-3). Pearson.

Sundberg, M. L. (2008). Verbal behavior milestones assessment and placement program: A language and social skills assessment program for children with autism and other developmental disabilities: Guide. AVB Press.

Reference for this article:

Larsson, E., Weiss, M.J., & Mastrogiacomo, L. (2024). Research synopsis: Evaluating the effectiveness of two models of applied behavior analysis in a community-based setting for children with autism spectrum disorder. Science in Autism Treatment, 21(12).

Other Related Research Synopses:

Related ASAT Reviews:

Other Related Articles:

 

#BehaviorAnalysts #Researchers

 

Print Friendly, PDF & Email