Jumping to negative conclusions - a case of study-gathering bias? A reply by the developers of metacognitive training (MCT) to the meta-analysis of van Oosterhout et al. (2015)
Metacognitive training (MCT) is a novel therapeutic approach for patients with psychosis. As van Oosterhout et al. (2015) acknowledge in their meta-analysis, few studies on MCT have been conducted so far and a meta-analysis might still be premature at this stage. Van Oosterhout and colleagues report small to medium effects in favor of MCT (Hedges’ g = 0.22–0.31). These effects, which were mainly obtained for group MCT, are in the range of several prior meta-analyses of (individual) cognitive–behavior therapy (CBT) for psychosis (see Introduction of their meta-analysis). However, they report non-significant results and conclude that MCT should not be recommended in routine practice. While they concede that some of the effect sizes bordered (p = 0.054 for positive symptoms) or even achieved significance (p = 0.01 for blinded studies on positive symptoms), the abstract essentially claims a null result. By contrast, a recent narrative review (Moritz et al. 2014) concluded that MCT exerts a small to medium effect, and sometimes even large effect sizes for its individualized variant, termed MCT+. Narrative reviews, particularly when written by the developers of the approach under investigation, are potentially biased, and it is difficult to weigh the importance of different studies without systematic methodological tools. However, another meta-analysis (Eichner, 2015) arrives at similar conclusions, reporting a significant small-to-medium effect in favor of MCT for delusion severity (g = 0.41) and positive symptoms (g = 0.34). Yet another meta-analysis (Jiang et al. 2015) reports significant effects for positive symptoms (as well as for delusions without the study by van Oosterhout et al. 2014). In attempting to reconcile these findings with those reported by van Oosterhout et al. (2015), we believe that the latter study has some potential flaws that we wish to highlight below.