In this blog, Reader in Psychology Dr. Aja Murray shares the details of her Wellcome Trust Mental Health Data Prize winning project to develop a tool to assist researchers in conducting counterfactual analysis.
There’s no doubt that mental health is complicated. Scientists think that a whole range of different factors might influence our mental health, from our genes to our health habits (e.g., sleep, diet, exercise), relationships, and thinking patterns. Finding out which factors to focus on when developing treatments is tough because it’s difficult to know which things truly influence mental health and which ones are merely associated with it but don’t play an active role. Developing and evaluating treatments is expensive and time-consuming and it’s important to know which ones are promising in advance to make sure that resources are best spent. Some factors might also be difficult to test in an experimental or trial design for ethical or practical reasons.
Our Wellcome Trust Mental Health Data Prize project is focused on addressing this challenge. We are developing a digital tool that can help researchers to apply a technique called ‘counterfactual analysis’. Counterfactual analysis allows researchers to test possible ‘active ingredients’ in mental health using ‘observational’ data. In observational data (meaning in this context that it didn’t come from a trial or experiment) it can be difficult to tell which factors are true active ingredients. For example, if social media use is associated with, say, anxiety, is that because it truly impacts anxiety or is it because people who are more anxious have traits or past experiences that also put them at risk of higher social media use?
Counterfactual analysis works by matching people who are similar in their propensity for experiencing a candidate active ingredient but differ in the extent to which they actually experience it. They can then be compared on their mental health outcomes. For example, in our project we are using these types of techniques in data from the Millennium Cohort Study and UK Household Longitudinal Study to test whether reading for pleasure is an active ingredient in mental health. It builds on our earlier work that yielded mix findings and which therefore call for further study to provide more clarity from large-scale, population representative data.
Here’s our process*: In the first step, we find matches between adolescents who – based on background variables such as gender, socioeconomic status, prior mental health etc. – have a similar ‘propensity’ to read for pleasure. Next, we compare these individuals on their mental health outcomes. If those with a similar propensity to read but different levels of reading engagement differ in their mental health, we can conclude that reading engagement is a promising active ingredient for mental health.
Importantly, we’re also involving young people in shaping our project. We’re asking them to tell us what active ingredients they think are important for mental health so that we can target our tool to areas that matter to them. We are also asking them to help us interpret our findings, including getting their views on how the findings might feed into treatments or preventive interventions.
All in all, counterfactual analysis is promising but rare in mental health research and we think that’s a lot to do with a lack of tools to facilitate its implementation and interpretation. The next step for us will be to build a digital tool and associated guidance to help promote more widespread adoption of counterfactual analysis. This, we hope, can accelerate progress in identifying active ingredients in mental health.
*If you are interested in the technical details, you can read our pre-registration here: https://osf.io/wvcmk
About the author
Dr. Aja Murray is a Reader in Psychology at the University of Edinburgh. Her research focuses on developmental aspects of mental health, especially ADHD, internalising problems, externalising problems, and their co-occurrence.