Sarah E. Frampton, PhD, BCBA
University of Nebraska Omaha
Amaya Rocheleau, BS, RBT
University of Nebraska Omaha, University of Nebraska Medical Center
Humans, especially children, are always changing. In early years, nearly everything about a child is changing from their height to their motor skills to their speech. This, of course, includes children diagnosed with autism spectrum disorder (ASD). When conducting research evaluating various treatments, researchers must ensure that the changes observed from the beginning to the end of the study cannot simply be attributed to the passage of time and related maturation of participants (Ledford, 2018; Ledford et al., 2019; Slocum et al., 2022). If uncontrolled, maturation may threaten the internal validity of a study by providing an alternative explanation of the outcomes (see Frampton, 2024 for more information). Essentially, was the positive outcome obtained because of the treatment or could developmental progress, alone, explain the result? Researchers must take steps to prove that it was the treatment, not maturation.
How Maturation May Impact Outcomes
Let’s consider an example in which researchers are evaluating the effects of a new treatment on increasing vocalizations made by young children diagnosed with ASD. In early childhood, vocalizations exponentially increase (Hart & Risely, 1995). For children with ASD, vocal development may or may not follow a similar trajectory, but improvements over time are still often reported (Tager-Flusberg & Caronna, 2007). The researchers must make choices in their study methodology that will result in a convincing argument that the new treatment was responsible for changes in vocalizations and that the results cannot be attributed to only expected developmental progress. When the results of a study can be directly attributed to the treatment, the study is said to have high internal validity.
Controlling for Maturation: Single Subject Designs
Let’s unpack how threats to internal validity can happen with single subject research design when we look at changing behavior of an individuals or a small number of individuals. As growth and maturation cannot be stopped by the researcher, its effects must be anticipated then controlled through strong and planful research tactics. To control for maturation threats, it has been recommended that researchers collect repeated measures of behavior (e.g., Ledford, 2018; Slocum et al., 2022). This tactic involves collecting data in a consistent way over many days. Close examination of data helps to reveal patterns and trends that might indicate a relationship between time (and other uncontrolled variables) and the dependent variables (i.e., behaviors) of interest in the study.
It has also been suggested that researchers continue to collect data in baseline until a steady state of behavior is observed (e.g., Ledford, 2018; Slocum et al., 2022). This tactic involves collecting enough data, before treatment begins, to show that either a flat line or data moving in the direction opposite of what is expected in intervention. For example, if the treatment is expected to result in increased child vocalizations, wait until the number of vocalizations is approximately the same each day or decreasing. Then, when treatment is introduced, any changes can more readily be attributed to the introduction of this new variable. The introduction of treatment is noted on a graph by a vertical line as shown below.
Finally, researchers should repeatedly demonstrate this cause-effect relationship between the treatment and a change in the dependent variable (e.g., Ledford, 2018; Slocum et al., 2022). A change in behavior that occurs immediately after the treatment is introduced is compelling. But there are still alternative explanations that could explain this change (e.g., history- another internal validity threat to be discussed in our series). But if the same treatment results in the same or similar change in behavior at three different points in time, as is done in a multiple baseline design (Ledford, 2018), this is downright convincing.
Graph 1 shows research tactics in alignment with these recommendations, making a convincing case for a causal relationship between the treatment (independent variable) and the change in vocal behavior (dependent variable). From these data, the researchers can claim that improvement in vocal behavior was related to the treatment.
Graph 1
Multiple Baseline Design Across Participants with Strong Internal Validity
Let’s contrast with Graph 2. Repeated measures were collected for all participants. However, if we look at Participant 1 on the top tier, in the baseline phase there was an increasing trend. This means behavior was already improving before the treatment even began. The gains observed in the treatment phase appear to be simply a continuation of that trend which began in baseline, suggesting maturation alone resulted in improvements in vocalizations. As a result, the researchers should not be claiming that improvement in vocal behavior was related to the treatment.
For Participants 2 and 3 there were repeated measures showing a steady state of behavior in baseline. However, for Participant 2 when treatment began no substantial changes in behavior were observed. The eventual change in behavior occurred long after the treatment started, suggesting an outside event or variable may have been responsible for the sudden change (i.e., a history threat, to be covered in a future installment). As with Participant 1, the researchers should not be claiming that improvement in vocal behavior was related to the treatment.
For Participant 3, when the treatment was introduced, there was an immediate change in level and trend. However, the intervention for Participants 2 and 3 began at the same time point. This introduces additional doubts regarding the efficacy of the treatment. Maybe changes in behavior occur after a certain number of exposures to the baseline conditions (a testing threat). Maybe something happened in the setting that influenced performance of both participants (a history threat). Maybe the participants naturally developed the skills over time (a maturation threat). Presently, these doubts cannot be completely overruled. More demonstrations of a relationship between the treatment and improved participant outcomes are necessary to form a convincing argument. As with Participants 1 and 2, the researchers should not be claiming that improvement in vocal behavior was related to the treatment.
Graph 2
Multiple Baseline Design Across Participants with Weak Internal Validity
Controlling for Maturation: Group Designs
Group design methodology requires slightly different, but logically consistent, tactics to control for maturation. It is recommended that results from the experimental group be compared to a control group that does not receive treatment. If participants had generally the same characteristics at the beginning of the study and were randomly assigned to groups, any maturation that occurs should affect results from both groups equally (Ledford, 2018). If the experimental group shows statistically significant improvement in vocalizations in comparison to the control group, this suggests that neither chance nor maturation could explain the result. From these data, the researchers can claim that improvement in vocal behavior was related to the treatment.
Conclusion
When evaluating findings from research studies it is important to remain skeptical and consider potential alternative explanations for outcomes. Maturation of participants is unavoidable- all people change over time. However, if researchers claim that a treatment has a positive effect for individuals with ASD, this must be proven beyond a reasonable doubt. In the discussion section of the published article, the researchers should disclose all known threats and their severity. As noted by Frampton (2024), the severity of a threat ranges from very minor to quite severe. If a severe threat to internal validity is reported, we can have limited confidence in the findings of the study. As time and resources are precious, we must rely on research conducted using strong tactics to enable greater confidence that observed changes are due to the intervention itself and not a participant’s maturation due to passage of time.
References
Frampton, S. (2024). An overview of internal validity: Was it really the treatment that made a difference? Science in Autism Treatment, 21(08).
Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experiences of young American children. Paul H. Brookes Publishing.
Ledford, J. R. (2018). No randomization? No problem: Experimental control and random assignment in single case research. American Journal of Evaluation, 39(1), 71-90. https://doi.org/10.1177/109821401772311
Slocum, T. A., Pinkelman, S. E., Joslyn, P. R., & Nichols, B. (2022). Threats to internal validity in multiple-baseline design variations. Perspectives on Behavior Science, 45(3), 619-638. https://doi.org/10.1007/s40614-022-00326-1
Tager-Flusberg, H., & Caronna, E. (2007). Language disorders: Autism and other pervasive developmental disorders. Pediatric Clinics of North America, 54(3), 469–481. https://doi.org/10.1016/j.pcl.2007.02.011
Citation for this article:
Frampton, S., & Rocheleau, A. (2024). Science Corner: Maturation as a threat to internal validity. Science in Autism Treatment, 21(09).
Other Science Corner Articles:
- An overview of internal validity: Was it really the treatment that made a difference?
- Role of replication in scientific validation
- Some cautions on the exclusive use of standardized assessments in recovery-oriented treatment
- Determining the effectiveness of treatments available to persons with autism – Part One
- ASD Intervention: How do we measure effectiveness?
- What is evidence-based practice and why should we care?
- Treatment Integrity: Why it is important regardless of discipline
- “Verification” and the peer review process
Other ASAT Articles:
#EarlyCareerAndStudents #Faculty #Researchers #SavvyConsumer
