Editorial Perspective: When to start de‐implementation of interventions: the case of cognitive training for children with <scp>ADHD</scp>

Abstract

Over the last decades, not much has changed in which interventions are most often recommended for school-age children with Attention-Deficit/Hyperactivity Disorder (ADHD). Most clinical practice guidelines advise stimulant medication and behavioral parent training as evidence-based interventions. The short-term efficacy of medication is supported by ample evidence (Faraone et al., 2021), but there are several disadvantages: normalization is rare, long-term effects are uncertain, side effects are frequently reported and many children and their parents are resistant. Behavioral parent training leads to improvements in parenting practices (Dekkers et al., 2022), as well as decreases in conduct problems of the child, but effect sizes are generally only small-to-medium, and it is less clear how effects on the core ADHD symptoms can be interpreted as these typically diminish when raters of the behavior are blinded to the intervention (Daley et al., 2017). Together, this emphasizes the need for innovation with regard to the treatment of children with ADHD. Therapeutic innovations are most likely successful when based on a scientific understanding of the pathogenesis of a disorder. One obvious example of this principle in the field of ADHD is the development of cognitive training. Influential theoretical models of ADHD describe how behavioral symptoms are often underpinned by impairments in executive functioning, and are backed up by considerable empirical support from neuropsychological and neurobiological research (Kasper, Alderson, & Hudec, 2012; Stevens, Pearlson, Calhoun, & Bessette, 2018; Sonuga-Barke, 2002). Cognitive training intuitively derives from these models and involves using computer-based training to target the underlying cognitive deficits (often based on neurocognitive assessment methods of executive functioning) in children with ADHD to achieve behavioral improvements. In general, most cognitive training paradigms consist of daily practice on computerized cognitive tasks for at least 20 minutes during a period of at least 5 weeks, with variations in stimuli and structure (Redick, 2019). The difficulty level of the tasks is continuously adapted to the performance of the child, which is considered crucial for the effectiveness. The most well-known cognitive training paradigm is Cogmed, which specifically targets working memory – one of the executive functions that is on average most impaired in children with ADHD (Kasper et al., 2012; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). Early trials using Cogmed in children with ADHD showed positive outcomes on neuropsychological variables close to the training paradigm as well as on parent-rated ADHD symptoms (Klingberg et al., 2005; Klingberg, Forssberg, & Westerberg, 2002). This led to rapid implementation and commercialization of Cogmed. In the meantime, many other computerized cognitive training paradigms with features similar to Cogmed commenced (e.g., Braingame Brian, RoboMemo, Pay Attention!, BrainTrain, EndeavorRX). Some focus exclusively on training working memory, whereas others target a broader range of executive functions. Commercially, cognitive training has developed into a large industry (Cookson, 2014; Evans et al., 2021), and implementation for ADHD has expanded rapidly over the recent years which is exemplified by the distribution of cognitive training by large commercial parties (e.g., “200,000+ users in 20+ countries” www.cogmed.com, distributed via objective parties such as www.pearsonclinical.co.uk). Despite the initial positive evidence and the wide use of, and advocacy for, cognitive training interventions for children with ADHD, there is currently little scientific evidence for its efficacy in well-controlled trials with placebo or sham control conditions. On the contrary, the scientific evidence not supporting cognitive training as evidence-based intervention for samples of children with ADHD has become very compelling, with at least seven meta-analytic or systematic reviews (Aksayli, Sala, & Gobet, 2019; Cortese et al., 2015; Dovis, Agelink van Rentergem, & Huizenga, 2015; Melby-Lervåg & Hulme, 2013; Melby-Lervåg, Redick, & Hulme, 2016; Rapport, Orban, Kofler, & Friedman, 2013; Sonuga-Barke, Brandeis, Holtmann, & Cortese, 2014), all pointing in the same direction: (a) At best, there is only near-transfer of effects (i.e., improvements on tasks that are highly related to the task being trained), but no far transfer to real-life cognitive, academic, and behavioral functioning; (b) Improvements mainly seem to reflect the expectancy of those involved, which may be related to positive illusory bias effects. Advocates of cognitive training may argue that it is too soon to condemn cognitive training (Redick, 2019), and two options to improve its efficacy are often mentioned. First, some argue that cognitive training may be especially indicated for those children with ADHD that present with most severe cognitive difficulties, instead of offering it as a one-size-fits-all treatment to all children with ADHD. However, a trial in children with ADHD that directly tested whether executive functioning capacity moderated response to cognitive training showed no such effect (Dovis, Maric, Prins, & Van der Oord, 2019). Indirect evidence from a study on a large sample of young adults without ADHD actually suggests the opposite pattern: those with higher working memory capacity improved more from working memory training relative to those with low working memory capacity (Foster et al., 2017). Many other moderators were proposed, but findings are generally pessimistic (see Redick, 2019 for more details). Second, it is argued that most cognitive training paradigms used for children with ADHD are more focused on training the short-term storage/rehearsal aspects of working memory rather than actually targeting the working component of working memory, the central executive (Rapport et al., 2013). As deficits in the central executive are particularly related to real-life functioning of children with ADHD across several domains (Dovis, Van der Oord, Wiers, & Prins, 2013; Rapport et al., 2013), a logical avenue for improvement would be to focus cognitive training more on this central executive instead of short-term storage/rehearsal processes. For the assessment of evidence for an intervention, the Evidence Base Update model by the American Psychological Association is used most often, which consists of a pyramid ranging from level 4 (experimental treatment) to level 1 (well-established treatment) (see Evans, Owens, Wymbs, & Ray, 2018 for an elaborate application of these criteria to psychosocial treatments for ADHD). This framework is very helpful to determine which interventions are well-established and should be implemented, and to determine which interventions are promising but are in need of further scientific testing. This framework, however, does not give answers to the important question about when there is sufficient evidence to conclude that an intervention is not efficacious. To our best knowledge, there are no guidelines to reach such conclusions. The picture is clear that the large majority of the cognitive training paradigms that are currently being distributed commercially for children with ADHD on a large scale are not backed-up by a sufficient evidence base. This does not suggest that further treatment development research is not warranted as we believe that this line of research may one day lead to effective treatments. In fact, there is some evidence that meaningful progress is being made (e.g., Kofler et al., 2018; Singh, Gaye, Cole, Chan, & Kofler, 2022). However, currently, the evidence necessary to support cognitive training as a treatment option to families is lacking. Therefore, our tentative conclusion is that the current state of evidence justifies de-implementation (for more background on how to de-implement, see Norton & Chambers, 2020; Prasad & Ioannidis, 2014), because offering interventions without enough evidence for their effectiveness may harm: Families in need spend money, time and effort on such interventions instead of evidence-based care, and a lack of effects could lead to frustration and could discourage families to seek further help. The authors would like to thank Steve Evans (Ohio University) for his helpful suggestions on an earlier version of this manuscript. T.D. has no potential conflicts of interest. He is supported by a ZonMw grant (# 07290202110010) focused on the implementation of knowledge about effective interventions for ADHD. S.v.d.O. has co-developed a planning-focused and solution-focused treatment and other behavioral treatments but has no financial interest in any of these. She has received research grants from ZonMw and the Research Foundation–Flanders (FWO); she was an advisor of the Dutch ADHD guideline groups and is a member of a working group on ADHD of the Superior Health Council of Belgium.

Topics & Concepts