Emily K. Sullivan, PhD, BCBA-D, LBA (NE, NY)
University of Rochester Medical Center

ASAT's Science CornerWhen reading research articles, it’s easy to focus on whether the treatment in the study “worked.” However, just because research shows that a treatment was effective, it does not mean that a “control, copy, paste” approach in practice would produce the same results. For example, although a study may show that the participant’s behavior changed as intended in a highly controlled research setting, this does not automatically mean that these outcomes translate into continued success in the home, school, or community setting. As such, this remains a key consideration when extrapolating treatment outcomes from research articles to what you might hope to achieve with your own child or client. This is where ecological validity becomes valuable.

What is Ecological Validity?

You might be thinking, “Really, another term?” I promise this is a good one. Ecological validity refers to the degree to which the features of a research study resemble the participants’ typical environment and the conditions under which the behavior change achieved is expected to occur after the study ends (Chezan et al., 2022; 2024; Fahmie et al., 2023). Many factors are highly controlled in research studies (e.g., implementers, procedures, setting, and session length) to demonstrate internal validity, or confidence that the treatment caused the observed results (Frampton, 2024). However, a treatment “working” under ideal, controlled conditions does not guarantee it will work in real-world conditions.

As Frampton (2025) describes, when we consider external validity, what becomes of interest is the likelihood that the findings of a study with strong internal validity can be attained in less controlled settings that more closely resemble the typical environment. While external validity evaluates whether treatment effects reported in scientific research articles generalize across populations, settings, measures, and treatment features, ecological validity focuses on whether the features of a single study reflect the everyday conditions participants experience (Fahmie et al., 2023). In other words, external validity concerns outcomes demonstrated across studies, while ecological validity focuses on the features of single studies. Although there is overlap between external validity and ecological validity, the distinction matters for both caregivers and service providers.

Why Does Ecological Validity Matter?

Consider a child receiving applied behavior analytic (ABA) services to address challenging behavior. Treatments described in research studies may be delivered by highly trained therapists in specialized, well-supervised clinic spaces, and sessions may occur several times each day and multiple times each week. In home settings, caregivers and service providers may experience limited support, resources, and time, as well as various barriers and distractions that make it difficult to use the same treatment procedures. These differences in conditions across contexts can influence the feasibility and outcomes produced by a treatment. Without considering these factors, even evidence-based treatments with lines of research demonstrating generality across studies may be difficult to implement consistently and correctly under certain real-world conditions.

The value of ecological validity lies in elucidating how the features of a research study or intervention resemble the everyday experiences of the individuals experiencing the procedures. Despite its value, examining ecological validity is complex. As such, research has emerged describing indicators of ecological validity (Chezan et al., 2022; 2024) and tools aimed to facilitate the planning and appraisal of ecological validity in relation to the features of research studies (Fahmie et al., 2023).

Value in the Literature

When reviewing research studies, ecological validity is considered in relation to the study’s participants. That is, how the study’s features reflect the participants’ everyday experiences. Considering sources of ecological validity is challenging, and despite many researchers’ dismay, it is not an exact science. Rather, evaluating ecological validity relies on careful judgment centered on the unique, real-world experiences of the participants. Several questions (based on the ecological validity indicators described by Chezan et al., 2022; 2024) may assist with making these careful judgments based on the features of research studies:

      • Who were the implementers?
      • In what setting was the study conducted?
      • What day-to-day activities were involved in the study?
      • How long were sessions, and how often were they conducted?
      • What resources were required as described in the study?
      • Did the reported outcomes occur across implementers, across settings, and over time?
      • What safeguards were in place to ensure that the intervention was implemented as planned?

Let’s use functional communication training (FCT) as an example (see Cicero, 2024; Shawler & Celiberti, 2019). FCT is among the most evidence-based and widely used behavioral treatments for challenging behavior (e.g., aggression, self-injury, destruction of property). In a review of over 200 studies on FCT published over several decades, Ghaemmaghami et al. (2021) revealed that although many of the studies showed that FCT worked under very controlled conditions (what researchers often call “efficacy”), far fewer have shown that it worked well under everyday conditions (e.g., with caregivers, in typical settings, over time, etc., an outcome researchers often call “effectiveness”). This is important because, for a study to produce an effective outcome, many of the included features have greater ecological validity (Fahmie et al., 2023; Ghaemmaghami et al., 2021). Table 1 shows examples of weaker and stronger ecological validity evidence from FCT studies included in the review by Ghaemmaghami et al. (2021) for each of the questions listed above.

It’s unlikely that all study features will have strong ecological validity. There is overlap between external and ecological validity, and evidence of supporting each tends to covary (Ghaemmaghami et al., 2023; Fahmie et al., 2023). For example, studies with strong external validity tend to have strong ecological validity as well. Studies with greater internal validity and lesser external validity are crucial for advancing science. As are studies with greater external validity and lesser internal validity. But how the outcomes are interpreted is distinct. Evidence for validity exists on a continuum for this reason, which is why being able to identify it, or lack thereof, in research articles is important. This information helps clarify what the treatment outcomes reported in a study do and do not demonstrate. The goal is not to dismiss research based on strength in one area versus another. Rather, the goal is to critically evaluate the strengths and limitations. By doing so, caregivers and service providers can identify where adaptations to evidence-based treatments may be needed to ensure their implementation is practical and effective for their child or client.

Value in Your Living Room

Use of the questions above is one way to determine how ecologically valid the features of a study are, and how well these features align with the context in which you aim to apply a treatment described in a research article. However, from a caregiver’s perspective, interest in ecological validity may extend beyond the pages of a research article to include how ecological validity can be promoted in treatments for their child.

Suppose you’re the caregiver of a child who engages in aggression or self-injury and receives ABA services at home. You’ve reviewed several articles using the questions above; however, your focus now turns to considering the ecological validity of an FCT treatment recommended for your child at home. Caregivers play a vital role in ensuring that treatments are feasible, effective, and sustainable. Evaluating the ecological validity of a treatment recommended for your child can be done in much the same way as you would evaluate it in a research study. However, at times, the lens through which ecological validity is considered from a service provider perspective may not match your perspective as a caregiver. And disconnects can be challenging to navigate.

The first step in remedying these ecological validity issues is to identify them. The second is to collaborate with service providers to adapt evidence-based treatments to ensure they are feasible, effective, and sustainable in real-world settings, like the living room. To do so, both caregivers and service providers may consider using these three strategies:

      • Proactively plan: Proactive planning helps minimize potential disconnects between caregivers and service providers. Describe the goals, priorities, prerequisite skills, and potential barriers related to ecological validity from the outset of treatment planning.
      • Openly Communicate: Open communication ensures that any barriers are addressed and adjustments are made early on. Discuss the typical environment and routines, along with your child or client’s strengths and needs, not only from the start of the treatment process, but also on an ongoing basis.
      • Thoughtfully Adapt: All treatments described in research articles require adaptations to meet the unique needs of any child or client and their everyday experiences. Thoughtful adaptations are best made when boundaries are clear about what is nonnegotiable and where there is flexibility in schedules, materials, or procedures, while preserving essential components of the treatment.

Take Away Considerations

Ecological validity plays a crucial role in determining the likelihood that treatment will work under real-world conditions. This is true for both the participants in studies in the literature and for your own child or client in their living room. Consider factors such as the implementers, the setting, and how treatment aligns with daily routines and available resources. These details will help you assess the ecological validity of procedures described in research articles and of treatments recommended for your child or client. Effective collaboration between caregivers and service providers is essential to maintain the key aspects of an intervention while adapting it to be feasible in real-world settings. As a consumer of research, it is helpful to ask whether a study accurately reflects the context in which participants live and learn, and to approach claims about ecological validity with healthy skepticism. When in doubt about the ecological validity of an intervention recommended for your own child or client, a proactive approach with open dialogue can promote effective collaboration and facilitate thoughtful adaptations to ensure the treatment is both meaningful and feasible.

Table 1

Questions to Guide the Evaluation of Ecological Validity Evidence in Scientific Articles

Weaker evidence Stronger evidence
Who were the implementers?
“Members of the research team (i.e., doctoral students with master’s degrees and Board-Certified Behavior Analyst credentials) implemented the procedures during all sessions throughout the study.”

(Falcomata et al., 2018; p.318)

 “All assessment and treatment contingencies were delivered by the child’s parent.”

(Derby et al., 1997; p. 509)
In what setting was the study conducted?
“All sessions were conducted by trained researchers on the hospital unit.”

(Borrero & Vollmer, 2006; p. 375)

 “All procedures conducted in the setting in which behavior support was needed.”

(Lloyd et al., 2015; p. 190)
What day-to-day activities were involved in the study?
“Materials in the room included a desk, two chairs, and other relevant session materials (i.e., demand materials, stimulus card).”

(Derosa et al., 2015; p.118)

 “All sessions, except for the community probe, occurred at the child’s home (e.g., kitchen, playroom, backyard) within the context of her regular service delivery.”

(Beaulieu et al., 2018; p. 386)
How long were sessions, and how often were they conducted?
“Sessions lasted for 15 min and were conducted two to four times daily, 4 to 5 days per week.”

(Goh et al., 2000; p. 197)

 “All sessions were carried out in four supermarkets normally used by the group home staff in their shopping expeditions…Three to five sessions were conducted per week.”

(Carr & Carlson, 1993; p. 158)
What resources were required?
“All sessions were 10 min in duration and were conducted in either a room (3 m x 3 m) with a one-way observation window or an open living area on the unit (generalization data).”

(LeBlanc et al., 2001; p. 41)

 “Adaptations were generated in collaboration with the families that focused on responding to the caregiving demands, family support, and social interactions that characterized their daily routines.”

(Moes & Frea, 2002; p. 529)
Did the reported outcomes occur across implementers, across settings, and over time?
“We could have also improved ecological validity by introducing caregivers or teachers who were likely to be conducting the treatment following the participants’ discharge from the outpatient clinic.”

(Jessel et al., 2018; p.280)

 “Treatment effects were extended across multiple teachers and home care providers who worked closely with the participating children.”

(Santiago et al., 2016; p. 807)

Note. The examples are from studies included in the Ghaemmaghami et al. (2021) review of functional communication training. The questions included are those that clinicians and service providers can use to evaluate the ecological validity of research study features.

References

Beaulieu, L., Van Nostrand, M. E., Williams, A. L., & Herscovitch, B. (2018). Incorporating interview-informed functional analyses into practice. Behavior Analysis in Practice, 11(4), 385–389. https://doi.org/10.1007/s40617-018-0247-7

Borrero, C. S., & Vollmer, T. R. (2006). Experimental analysis and treatment of multiply controlled problem behavior: A systematic replication and extension. Journal of Applied Behavior Analysis, 39(3), 375–379. https://doi.org/10.1901/jaba.2006.170-04

Carr, E. G., & Carlson, J. I. (1993). Reduction of severe behavior problems in the community using a multicomponent treatment approach. Journal of Applied Behavior Analysis, 26(2), 157–172. https://doi.org/10.1901/jaba.1993.26-157

Chezan, L. C., Bauer, A. M., McCammon, M. N., & Drasgow, E. (2024). Functional communication training in schools: A systematic analysis of the evidence for ecological validity. Behavior Modification, 48(5-6), 502–536. https://doi.org/10.1177/01454455241264816

Chezan, L. C., McCammon, M. N., Drasgow, E., & Wolfe, K. (2022). The ecological validity of research studies on function-based interventions in schools for children with autism spectrum disorder. Behavior Modification, 46(1), 202–229. https://doi.org/10.1177/0145445520964921

Cicero, F. (2024). A treatment summary of functional communication training. Science in Autism Treatment, 21(7).

Derby, K. M., Wacker, D. P., Berg, W., Deraad, A., Ulrich, S., Asmus, J., Harding, J., Prouty, A., Laffey, P. & Stoner, E. A. (1997). The long‐term effects of functional communication training in home settings. Journal of Applied Behavior Analysis, 30(3), 507-531. https://doi.org/10.1901/jaba.1997.30-507

Derosa, N. M., Fisher, W. W., & Steege, M. W. (2015). An evaluation of time in establishing operation on the effectiveness of functional communication training. Journal of Applied Behavior Analysis, 48(1), 115-130. https://doi.org/10.1002/jaba.180

Fahmie, T. A., Rodriguez, N. M., Luczynski, K. C., Rahaman, J. A., Charles, B. M., & Zangrillo, A. N. (2023). Toward an explicit technology of ecological validity. Journal of Applied Behavior Analysis, 56(2), 302–322. https://doi.org/10.1002/jaba.972

Falcomata, T. S., Muething, C. S., Silbaugh, B. C., Adami, S., Hoffman, K., Shpall, C. & Ringdahl, J. E. (2018). Lag schedules and functional communication training: Persistence of mands and relapse of problem behavior. Behavior Modification, 42(3), 314-334. https://doi.org/10.1177/0145445517741475

Frampton, S. (2024). An overview of internal validity: Was it really the treatment that made a difference? Science in Autism Treatment, 21(8).

Frampton, S. (2025). Science Corner: An overview of external validity. Science in Autism Treatment, 22(7).

Ghaemmaghami, M., Hanley, G. P., & Jessel, J. (2021). Functional communication training: From efficacy to effectiveness. Journal of Applied Behavior Analysis, 54(1), 122–143. https://doi.org/10.1002/jaba.762

Goh, H., Iwata, B. A., & DeLeon, I. G. (2000). Competition between noncontingent and contingent reinforcement schedules during response acquisition. Journal of Applied Behavior Analysis, 33(2), 195–205. https://doi.org/10.1901/jaba.2000.33-195

Jessel, J., Ingvarsson, E. T., Metras, R., Whipple, R., Kirk, H., & Solsbery, L. (2018). Treatment of elopement following a latency‐based interview‐informed, synthesized contingency analysis. Behavioral Treatments, 33(3), 271– 283. https://doi.org/10.1002/bin.1525

LeBlanc, L. A., Hagopian, L. P., Marhefka, J. M., & Wilke, A. E. (2001). Effects of therapist gender and type of attention on assessment and treatment of attention‐maintained destructive behavior. Behavioral Treatments, 16(1), 39–57. https://doi.org/10.1002/bin.73

Lloyd, B. P., Wehby, J. H., Weaver, E. S. Goldman, S. E., Harvey, M. N., & Sherlock, D. R. (2015). Implementation and validation of trial-based functional analyses in public elementary school settings. Journal of Behavioral Education, 24(2), 167–195. https://doi.org/10.1007/s10864-014-9217-5

Moes, D. R., & Frea, W. D. (2002). Contextualized behavioral support in early treatment for children with autism and their families. Journal of Autism and Developmental Disorders, 32(6), 519–533. https://doi.org/10.1023/A:1021298729297

Santiago, J. L., Hanley, G. P., Moore, K., & Jin, C. S. (2016). The generality of interview-informed functional analyses: Systematic replication in school and home. Journal of Autism and Developmental Disorders, 46(3), 797–811. https://doi.org/10.1007/s10803-015-2617-0

Shawler, L., & Celiberti, D. (2019). Clinical Corner: What is functional communication training? Science in Autism Treatment, 16(12).

Reference for this Article:

Sullivan, E. (2026). Science Corner: Why ecological validity matters in the literature and in your living room. Science in Autism Treatment, 23(3).

Other Science Corner Articles Related to External and Internal Validity:

Other ASAT Articles:

#Researchers #SavvyConsumer #Educators #Parents

Print Friendly, PDF & Email