Review of How to Spot Hype in the Field of Psychotherapy

Meichenbaum, D., & Lilienfeld, S. O. (2018). How to spot hype in the field of psychotherapy: A 19-item checklist. Professional Psychology: Research and Practice, 49, 22-30. https://psycnet.apa.org/doi/10.1037/pro0000172

Reviewed by Caroline Simard, MS, BCBA and David Celiberti, PhD, BCBA-D
Association for Science in Autism Treatment

How to Spot Hype in the Field of Psychotherapy With hundreds of autism treatments on the market, how can we ensure that we are doing everything we can to properly assess whether an intervention has sufficient quality research to support its claims or is simply being hyped up as the new ‘evidence-based’ intervention for autism? Meichenbaum and Lillienfeld’s 2018 article, How to spot hype in the field of psychotherapy: A 19-item checklist, is a stark reminder that pseudoscience and false claims are by no means unique to autism treatment and that a far broader group of consumers and providers can be deleteriously impacted. This important article is also a call to action as one of the best things we can do when evaluating the potential effectiveness of psychotherapy interventions (or really any intervention) is to be skeptical. This would apply regardless of whether you are a practitioner, consumer, potential client, or student. Whether you are contemplating psychotherapy or selecting interventions for a newly diagnosed child with autism, it is easy to get swept up in exaggerated claims of miracle cures, powerful endorsements, or a strong sales pitch that promises remarkable results.

In this incredibly helpful article, Meichenbaum and Lilienfeld (2018) propose a 19-item checklist to help ‘spot the hype’ and ensure that we are being critical in our assessment of the potential benefits and risks of psychotherapies, especially those claiming results that seem too good to be true. The checklist is said to be primarily designed for practitioners and practitioners in training; however, many of the presented warning signs and red flags for identifying hype in various sciences can be helpful to anyone wanting further evidence of a treatment’s effectiveness and potential limitations, including the treatment of autism.

The checklist provided by Meichenbaum and Lilienfeld (2018) expands on previous lists of pseudoscientific indicators and encourages readers to not only be skeptical of the marketing of interventions, but the evidence behind the claims made and the quality of the research conducted. They remind us that just because someone claims a treatment is highly effective and evidence-based, we owe it to ourselves and those we are supporting to question its claims. This is particularly true with psychotherapies that are largely untested and/or lack a well-supported and cogent conceptual basis. Such advice must sound very familiar to our readers as it characterizes so much of what ASAT amplifies in our articles. The themes and recommendations included in this article resonate deeply with the ASAT team and, most importantly, readers of Science in Autism Treatment. As indicated by the authors, the absence of controlled clinical trials for many psychotherapies out there is even more reason for clinicians and consumers alike to be extra cautious. It truly takes a village to foster healthy self-doubt, skepticism, and to share a responsibility to further explore any red flags or warnings signs, especially in those interventions that are overhyped and over-promoted as will be described later. Multiple warning signs should sound the alarm bells and should prompt further assessment of the intervention and its claims.

The goal of this provisional checklist, according to the authors, is to safeguard practitioners and consumers of psychotherapy against exaggerated claims and encourage healthy self-doubt in clinicians. The items that comprise the checklist have been selected from various areas including, but not limited to, academic publications and presentations, inspections of printed and online advertisements of treatments, and consultations with colleagues. This thought-provoking checklist could be broken down into two larger subgroups:

Red flags surrounding marketing and promotion of the intervention; and
Red flags surrounding the research evidence of the intervention.

Warning signs related to promotion and marketing put forth by Meichenbaum and Lilienfeld (2018) include such considerations as exaggerated or unsubstantiated claims of efficacy, interventions that rely heavily on endorsements, and the use of psychobabble or neurobabble (i.e., verbiage that sounds scientific but in fact contains little or no content) to promote their approach. At first glance, remarkable claims may seem credible. Upon further assessment, they contain no empirically supported evidence of the effectiveness of their approach. Other notable warning signs identified by the authors include advocates who are defensive about their approach and hesitate to discuss the limitations or criticisms of the approach and those claiming the treatment fits all. We should also be cautious around the all-too-common mistake of relying too heavily on anecdotal evidence rather than taking the time to do controlled studies that could provide concrete data on the treatment approach. Fad treatments often blossom from anecdotes due to the promotion of the heartbreaking stories of the individuals involved, and their ‘remarkable’ recovery after the treatment. All these ring true for the autism community.

Research evidence should also be reviewed as reflected by the last six items on the checklist. Some red flags presented include promoting their treatment as ‘evidence-based’ with weak data, small sample sizes, or loose criteria; or comparing their findings only to others with weak findings. The authors also caution against approaches that neglect to present a critical account of the scientific validity or theoretical basis, for the effectiveness of the proposed treatment as well as resorting to loopholes to explain any negative findings. Again, individuals with autism and their families are bombarded by abundant examples of these red flags. We encourage our subscribers to read the original article as the authors unpack each of the 19 items with precision, clarity, and transparency. As we read through this very compelling analysis, we were both comforted that the concerns transcended the autism community and troubled that these concerns were so far reaching.

You may be wondering why we paired a drawing of a Dodo bird with this article, which is a reference to the Dodo Bird in Lewis Carroll’s Alice in Wonderland who granted prizes for everyone who ran a race. Early in the article, Meichenbaum and Lilienfeld (2018) discuss the Dodo Bird Verdict (Rosenzweig, 1936), which posited that unsubstantiated treatments may work as well as well-established ones. As anticipated, the authors cast doubts on such a position, which we would like to amplify further as it relates to autism. The tremendous heterogeneity observed within the autism spectrum and the wide range of presenting concerns of family members cries out for treatment specificity. A young child with severe self-injurious behavior, school-aged child learning to raise their hand at appropriate times in class, a young man learning to interact with his co-workers on the jobsite would get very individualized treatment.

In sum, the authors argue that when science is involved, skepticism, humility, and healthy self-doubt are critical for clinicians towards their own practice and psychological treatments in general. They demonstrate this humility and tentativeness themselves, by sharing that their checklist is a starting point and meant to be scrutinized, evaluated, and improved upon. When considering any and all autism interventions, we encourage you to review the checklist provided by Meichenbaum and Lilienfeld because being skeptical and further informed could save you not only money and effort, but more importantly precious time that could be lost on ineffective treatment.

References:

Rosenzweig, S. (1936). Some implicit common factors in diverse methods of psychotherapy. American Journal of Orthopsychiatry, 6(3), 412–415. https://doi.org/10.1111/j.1939-0025.1936.tb05248.x

Citation for this article:

Simard, C., & Celiberti, D. (2021). Review of How to spot hype in the field of psychotherapy: A 19-item checklist and what it means for the autism community. Science in Autism Treatment, 18(9).

Other Related ASAT Articles:

Some Related Media Watch Letters:

Review of How to Spot Hype in the Field of Psychotherapy: A 19-Item Checklist and What It Means for the Autism Community