Promoting engagement with a mental health smartphone app: evidence from a randomized field experiment

1 Introduction

Poor mental health among young adults, particularly tertiary students, is a pressing public policy issue. It is estimated that approximately one-third of university students suffer from anxiety and depression (Ibrahim et al., 2013; Ahmed et al., 2023). For students in tertiary education, mental ill-health is associated with reduced academic performance and higher dropout rates (Eisenberg et al., 2009; Richardson et al., 2012), in addition to other personal and social costs.

Our aim is to test interventions to improve the take-up of, and engagement with, apps used to manage students' mental health. Specifically, we consider the efficacy of financial incentives and social influencers. Mental health apps (MHapps) are an attractive tool for improving community mental health as they are low-cost, scalable, and provide timely support. They may also particularly appeal to younger people, who are typically tech-savvy.

Numerous studies have demonstrated the efficacy of MHapps for reducing symptoms of anxiety and depression in randomized trials, which typically use targeted recruitment and closely monitor participants to ensure engagement (for reviews, see Firth et al., 2017a,b; Wu et al., 2021). However, in real-world data, take-up and engagement are often low (Fleming et al., 2018; Baumel et al., 2019). One line of research focuses on how app design can influence engagement, for example, by incorporating persuasive features (i.e., features designed to enhance user experience) and behavioral nudges (Wu et al., 2021). Nahum-Shani et al. (2022) propose a framework for promoting engagement in digital tools that integrates affect, integration, and motivation with attention, context, and translation—referred to as the AIM-ACT framework—to enhance in-the-moment engagement. We focus instead on the effect of incentives outside of the app itself. This distinction is important for two reasons. First, users need to be encouraged to use the app in the first place and then be persuaded by its features. Second, our results speak to what can be done when organizations do not want to develop their own products, noting that there are already between 10,000 and 20,000 MHapps available (Clay, 2021).

While there have been some randomized field studies investigating incentives to use health and consumer apps generally (e.g., Sun et al., 2019; Jeon et al., 2020; Munzert et al., 2021), little evidence exists for MHapps specifically, particularly for those who are not currently using the app or who are not actively interested in it. Existing evidence is mixed. In a pilot study, Boucher et al. (2021) found that paying existing users of the MHapp Happify based on the number of activities completed increased engagement, with higher compensation leading to more engagement. In a large study of participants interested in trying a popular mindfulness app, Shreekumar and Vautrey (2024) found that financial incentives boosted usage among participants. Bear et al. (2025) found that participants in an activity-based reward treatment arm used an MHapp on more days but did not increase the overall intensity of use over the trial period. Qualitative interviews revealed that financial incentives were a motivating factor for some participants. Chang et al. (2023) found no evidence that activity-based financial rewards increased MHapp use in a 1-month trial involving college students.

Financial incentives are appealing due to their simplicity but can be costly, particularly if ongoing payment is required for sustained engagement. Consequently, alternative interventions with minimal or no marginal cost might be required to scale. One promising area is the use of social influencers. Social media has become a popular source of health information (Chen and Wang, 2021), which, while a concern for public health officials, may also present opportunities to positively influence behavior. In the context of the AIM-ACT framework discussed by Nahum-Shani et al. (2022), financial incentives are likely to activate motivation for engagement, while social influencers can increase attention to the app. Celebrity endorsements on Twitter (Alatas et al., 2022) and messaging by socially influential community members (Banerjee et al., 2021) have been shown to improve health literacy and vaccine uptake. However, it remains an open question whether social influencers can similarly enhance engagement with MHapps.

2 Field experiment

Our study received MREC approval from the University of Technology Sydney (project IDs: ETH22-7519 and ETH23-8069).

2.1 Recruitment and participants

On 3 March 2023, we invited all students at the University of Sydney (UTS) (approximately 45,000) to participate in a study on mental wellbeing, with 1,812 students ultimately completing the survey. Students were invited through an email newsletter (see Appendix B) sent to all students enrolled at UTS (i.e., including undergraduate, postgraduate, and other levels) and were given a 1-in-20 chance of receiving an AUD$20 Coles Group gift voucher for their participation.¹ There were no exclusion criteria for participants; since investment in mental health is universally important, we were interested in behavior across the entire student population and did not restrict participation to those with clinical depression or with anxiety.

Students who signed up for the study were asked to complete a short survey (under 10 min) that measured depressive symptoms through the Patient Health Questionnaire (PHQ-9) and anxiety through the General Anxiety Disorder scale (GAD-7).² The median response time was 6.3 min. At the end of the survey, participants were randomly allocated (stratified on sex, first-in-family university attendance, and first-year student status) with equal probability to one of four treatment arms: (i) control, (ii) MHapp treatment (baseline), (iii) MHapp treatment plus financial incentive (financial incentives), and (iv) MHapp treatment plus social media group (social influence). Randomization was performed using the Qualtrics survey platform without enforcing equal group sizes.

2.2 Setting

UTS is a major Australian university with approximately 45,000 students, centrally located in Sydney, Australia's largest city (population approximately 5.5 million). UTS offers various forms of mental health support to students, including access to medical professionals, a student peer-support program, and online self-help guides. It also offers a free confidential counseling service for students that focuses on short-term support, usually involving a limited number of sessions, which can serve as a gateway to more ongoing support through the Australian health care system. At the time of this study, UTS did not have an MHapp program.

2.3 Mental health app

Participants in the treatment arms were given access to the MHapp MoodMission, which was “designed to be an easy-to-use and engaging application to enhance mental health and wellbeing for smartphone users of all ages, adolescents and older, and all mental health statuses” (Bakker et al., 2018b, pp. 499). The structure of the app is very simple. Users are prompted to input how they are feeling by selecting one of three options: “low, flat, or depressed,” “anxious, nervous, or worried,” or “I'm not feeling either of the above,” followed by a brief set of follow-up questions. Based on these inputs, they are given a “mission” to complete. Users can choose from five suggested missions, which are derived from activities supported and promoted by cognitive behavioral therapy (CBT), including behavioral activation, relaxation, mindfulness, physical exercise, and cognitive reframing (Bakker et al., 2018b). Missions are drawn from a large databank of evidence-based strategies to reduce anxiety and improve mood and include activities such as guided meditation, push-ups, or calling a friend. On completing a mission, users provide a short debrief on how their mood has changed and rate the usefulness of the activity. These responses feed into the app's algorithm and help tailor future mission suggestions.

MoodMission was selected because (1) it has a solid evidence base, including positive usability assessments, app engagement ratings, and improvements in mental wellbeing (Bakker et al., 2018a,b; Bakker and Rickard, 2019; Aizenstros et al., 2021), (2) it was developed and tested in Australia, and (3) it is designed to be helpful for a broad spectrum of mental health issues, from everyday mood disturbances to clinical presentations, including depression.

2.4 Treatment arms

Control: the participants completed the survey and were thanked for their time.

Baseline: the participants completed the survey and were given a download code for MoodMission, which provided 5 months of access and covered the AUD$7.99 subscription fee.

Financial incentives: in addition to the download code, the participants were informed that if they downloaded MoodMission, they would receive an AUD$5 Coles Group gift voucher.³ Furthermore, for every “mission” they completed, they would receive a ticket for the major prize draw at the end of the semester (12 weeks), with a maximum of two tickets per week. Completing a mission involves opening the app, answering questions about the current mood, then selecting from one of the five recommended activities and marking it as complete (verification is self-reported). Activities range in level of time commitment, but recommendation sets often include light-touch activities (e.g., stretching, writing out positives, and short meditation). The three prizes were an AUD$500, AUD$200, and AUD$100 Coles Group gift voucher. These incentives were designed to act as extrinsic motivators for continued engagement with the app, potentially helping to overcome the widely documented drop-off that occurs after downloading.

Social influence: in addition to the download code, participants were invited to join an exclusive Instagram group moderated by three student ambassadors (“influencers”). The influencers were current students at the study university with experience in social media engagement. The ambassadors were not celebrities and were unlikely to be known to our participants. Their role was to serve as a relatable set of identities to build enthusiasm for MoodMission and mental health self-care. They created short weekly videos (10 in total, each approximately 1–2 min) detailing their experience with MoodMission and promoting good mental health more broadly. The content of the videos was at the discretion of the ambassadors but generally included personal reflections and promotion of the app.

2.5 Pre-registration and materials

Our study was pre-registered on OSF (https://doi.org/10.17605/OSF.IO/79Z4K). However, due to data limitations, our analysis covers only hypothesis 1 of the pre-analysis plan (PAP).⁴ All the results presented below (using quantitative data) were specified in the PAP unless explicitly stated otherwise. Details of our surveys conducted during the semester are provided in Appendix B, including the text used in our treatment interventions.

3 Data and descriptive statistics

3.1 Data

Information on student characteristics is from our baseline (wave 1) survey. To track app engagement, students in the treatment arms were given unique group codes to use when creating their MoodMission accounts, allowing us to observe how many missions they completed and when.

Across all treatments, 226 out of 1,384 (16%) students were linked to a MoodMission account. An additional 67 voucher codes were used, but no group code was entered, so the student could not be identified. This may be because the student used the app without providing the group code or activated the subscription but then did not go on to use the app. We decided to supplement the user data with self-report data on app downloads, which were collected in each of our three follow-up surveys (568 students in the treatment arms completed at least one follow-up survey). Combining the user with self-report data, there are 325 downloads (or 23%). This exceeds the number of subscriptions activated, presumably because some students downloaded the app but never activated it. Our preferred measure of downloads uses only MoodMission user data, since those students are engaged enough to at least register the app; however, we also provide results combining both user and self-report data for comparison.

3.2 Descriptive statistics

Table 1 shows that approximately 60% of the sample is female and 30% is male (the remainder are non-binary or undisclosed). Participants are 23 years old on average, 73% are under 25 years old, approximately 23% are postgraduate, and 30% are international students, and approximately 33% are the first in their family to attend university. Notably, approximately 36 and 40% of students report symptoms of moderate to severe anxiety and depression, respectively. This is in line with international evidence and highlights the importance of providing mental health support to university students. The last column tests the joint equality of the sample averages and demonstrates that the treatment groups are well-balanced.

Table 1. Descriptive statistics.

Detailed variable definitions are in Table A1. P-values in the last column are for Wald tests on the joint equality of each mean. Missing values are omitted when calculating means in each cell.

While we do not have population benchmarks for all the controls in Table 1 (notably the mental health measures), we know that, in 2022, 50% of UTS students were female, 70% were under 25 years old, 25% were international, and 26% were postgraduate (University of Technology Sydney, 2024). These figures are not greatly different from those in our sample. They are also similar to the national student population, with 57% of students being female and 30% postgraduate (Australian Government Department of Education, 2026).

4 Empirical model

Our baseline estimation equation is given by

Yi=α+β1Tif+β2Tis+ei(1)

where Y_*i* is either a dummy variable for downloading the app or another measure of app usage (e.g., number of missions completed). Tis and Tif are indicators for being in the financial incentives or social influencer treatment arms, respectively. e_*i* is a stochastic error term, and all other terms are parameters to be estimated. Our sample for analysis excludes the control group, as they were not offered the app. The treatment effect is estimated in comparison with the baseline group, which was offered the app for free without additional incentives.

Our main results estimate Equation 1 using a linear regression model with OLS. To explore treatment effect heterogeneity, we estimate models that include interactions between the treatment variables and the following covariates: sex (male/female); first-in-family to attend university; baseline PHQ-9 (above/below median), baseline GAD-7 (above/below median), and student enrolment year (first year/other).

5 Results

Our main regression results are in Table 2. Column 1 shows that being in the financial incentives treatment increases the probability of downloading by 7 percentage points (ppts), compared to a baseline of 14%. Column 2 combines user and self-report data on downloads; this slightly decreases the effect size to 5 ppts, and it is now only significant at the 10% level.⁵ Both columns show no treatment effect for the social influence groups.

Table 2. Regression estimates for the effect of incentive treatment on app engagement.

OLS regression estimates of the effect of treatments on engagement measures. Dependant variables in columns (Dep. vars in cols.) (1) and (2) are an indicator for downloading the app (MoodMission data only) and an indicator for downloading the app (MoodMission plus self-report data), respectively. Dep. vars in cols. (3)–(6) are total missions completed over the trial period (~12 weeks), total missions completed (per week) within 2 weeks, those completed within 6 weeks, and those completed within between 6 and 12 weeks of downloading, respectively. Heteroskedasticity-robust standard errors in parentheses.

^*, ^**, and ^*** indicate significance at the 10%, 5%, and 1% levels, respectively.

Columns 3–6 consider the number of missions completed in total and at different times since downloading. In this context, “missions” are the mental health activities completed in the app, and users can complete as many or as few as they like. In all cases, engagement is significantly higher in the financial incentives treatment. However, engagement is universally low in absolute terms.⁶ For example, over the whole trial, the mean total number of missions completed was 0.17 for the baseline group. Only 7.8% of students completed at least one mission overall in the baseline group, compared with 57% among students who downloaded the app. Similarly, in the first 2 weeks after downloading, users undertook 0.07 missions per week on average in the baseline, and this increased by only 0.05 missions in the financial incentives treatment. There is no evidence that the social influencer treatment increases engagement across any outcome measure. This may wholly be due to only 23 students joining the exclusive social media group, despite several reminders and opportunities to do so (see Appendix B).

Other results are reported in the Appendix. We find no strong evidence of heterogeneous treatment effects across the subgroups we consider (Tables A2–A6). However, we observed that women are more likely to download (9 ppts), as are those with high GAD-7 (11 ppts) and PHQ-9 scores (12 ppts). Our findings are robust to using Tobit regression (Table A7).

6 Insights from qualitative data

To elucidate why app take-up was not high, we analyzed open-response data collected in the first follow-up survey.⁷ Students who had not downloaded MoodMission were asked why not, with N = 198 providing answers out of 213. The main themes in the responses were identified with the assistance of generative AI (GPT-4 Turbo, accessed through the Perplexity platform). Our specific prompts are reported in Appendix C. We uploaded text files with each free-text response on a different line and requested that those responses be summarized into a list of main themes. Two study authors then cross-checked these themes against the individual responses, verifying their appropriateness, and independently categorized each response into one or more themes. There frequencies reported in the next paragraph are the average across the two researchers.

The main reasons students did not download the app (reported in Table 3) included perceived lack of need (25%), lack of time or perceived effort (21%), skepticism or preference for other forms of help (19%), general disinterest or aversion to apps (15%), lack of awareness or forgetfulness (14%), and technical or practical constraints (e.g., insufficient phone storage) (7%). We also obtained responses from 120 participants out of 126 about why they downloaded MoodMission. The main themes included curiosity and general interest (41%), special features or content (3%), mental health improvement (36%), the app being free or the presence of financial incentives (21%), or recommendation or participants' sense of obligation (6%).

Table 3. Example statements from open-ended survey questions (% of responses consistent with theme).

Additionally, seven participants who downloaded MoodMission participated in semi-structured interviews with one of the study authors to discuss their experience using the app. A longer summary of the interviews is provided in Appendix C. All students who reported mental health issues cited these issues as a reason for downloading the app. Other reasons for downloading the app included financial incentives and curiosity about the app's features. Dissatisfaction with some of the app's features and language difficulties (for international students) were cited as reasons for not maintaining usage over time.

7 Discussion

Our results suggest that getting students to engage with MHapps is difficult, despite the high level of mental distress within this population. Across the whole trial, conditional on downloading the app, the median number of missions completed was one. Providing students with the app for free resulted in 14% of them downloading it (22% if self-report downloads were included). Providing them with a small gift voucher for downloading (a negative price) only increased downloads marginally by 5–7 ppts. This modest response is consistent with the findings of Chang et al. (2023) and Bear et al. (2025), and the lack of sustained engagement mirrors empirical patterns in MHapp use Fleming et al., (2018); Baumel et al., (2019), as well as evidence that the effects of financial incentives weaken over time (Shreekumar and Vautrey, (2024).

It is possible that some forward-looking participants opted not to download the app because of anticipated personal conflict if, after the 5-month free period, they felt compelled to continue the subscription (perhaps due to becoming “hooked”). However, this did not emerge as an identified barrier in the qualitative data. We conclude that financial incentives can increase downloads and engagement, but the degree to which people respond to these incentives with greater engagement is low (often nil). As such, these incentives may not provide significant value for money, at least not with the incentive design considered in our study.

We also found that social influence was ineffective. However, this does not mean that this approach cannot work. To our knowledge, our study was the first to use a student ambassador-led social media group as encouragement. In our case, only 23 out of 468 students joined the Instagram group, so the failure may stem entirely from generating exposure for the influencers. Our study highlights the challenges of compliance with social engineering, but future research could trial different approaches to harness influencers and test whether they are more effective. For example, providing training to people with a large existing social media presence has shown promise for influencing mental health messaging (Motta et al., 2024). The opt-in nature of our approach likely reduced its efficacy.

Research on health apps has sometimes focused on the relative importance of intrinsic and extrinsic motivators (e.g., Huang et al., 2024; Bear et al., 2025). Motivation crowding theory (Frey and Jegen, 2001) emphasizes that external interventions (particularly those perceived as controlling) can undermine intrinsic motivation. In contrast, self-determination theory (Ryan and Deci, 2000) distinguishes between controlled and autonomous forms of extrinsic motivation, arguing that well-internalized extrinsic motivation can function similarly to intrinsic motivation when it supports individuals' basic psychological needs for autonomy, competence, and relatedness. Our study was not designed to examine this distinction in motivation, but there are some relevant findings. Similar to Bear et al. (2025), we found quantitatively that extrinsic rewards do motivate engagement, but their effects are small, and from qualitative data, we found that people who engaged with the app often cite intrinsic motivators such as mental health improvement and general interest. People appear willing to download and use MHapps for both intrinsic and extrinsic reasons, but sustained engagement remains a challenge that is not easily addressed through simple financial reward structures.

One somewhat promising finding is that students experiencing mental distress were more likely to download the app and, according to our qualitative data, did so specifically to improve their mental health. Previous research has found that people in worse mental health react less to financial incentives in small-stakes tasks compared to those in better mental health (e.g., Kung et al., 2018), but we found no difference in treatment effects based on mental health status.

Less positively, since very few students used the app more than a couple of times over the trial period, it is unclear whether it provided these students with meaningful benefits. Evidence from clinical trials suggests that MHapps can improve mental health but, in those trials, there is typically a concentrated effort to ensure participants are engaged over the trial period, for example, by recruiting people actively interested in the treatment Firth et al., (2017a),b; Wu et al., (2021). In contrast, our field study is a universal intervention and was offered to the general student population, which, in our view, provides a more realistic perspective on adoption and engagement if, for example, universities were to offer such apps to their students en masse.

Given the challenges to increasing MHapp engagement implied by our findings, this may suggest a limited role for these apps in improving student mental wellbeing in the aggregate, even if they do offer material benefits to the small subset of students who actively use them (as suggested by clinical evidence). However, if apps are offered in a way that addresses the barriers to adoption we have identified, they could potentially make a more meaningful contribution to improving student mental wellbeing.

From a practical perspective, our data suggest that, if apps are to be harnessed as a tool for improving student mental wellbeing, more needs to be done to convince students to engage with the training, and, once engaged, to adhere to the training. Although engagement and adherence strategies showed negligible success in this study, other strategies may benefit from investigation. Barriers to engagement and adherence identified in this study indicate that students may need to be made aware of the value of training and its associated efficacy. Better promotion of this kind may encourage sign-up and training adherence. Findings from Acampora et al. (2022) also support this notion. They demonstrate that providing students with information about the return to good mental health increases willingness to pay for an MHapp. Roth et al. (2024) provide complementary evidence for information provision boosting demand for online therapy and overcoming skepticism about the value of treatment. Marketing materials could be aimed at students' perceived lack of time to complete the training and provide reassurance regarding the effort involved. Reminders could be used to engage those who had forgotten about the app, including text messages, emails, and even phone calls. Future research could evaluate these reminders and other options to address barriers to MHapp adoption.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frbhe.2026.1700973/full#supplementary-material

Supplementary material Supplementary material at the source

Footnotes

Edited by: Michele Belot, Cornell University, United States

Reviewed by: Le (Lyla) Zhang, Macquarie University, Australia

Seema Mehrotra, National Institute of Mental Health and Neurosciences (NIMHANS), India

Tondani Mudau, Africa Unit for Transdisciplinary Health Research North-West University, South Africa

Footnotes

Coles Group is a major supermarket and retail store conglomerate in Australia. Its supermarkets have approximately 29% market share Martin, (2025) and these and other Coles Group stores are prominent in Australia.

See exact details for the questions and scoring in Appendix Table A1.

In practice, participants received an AUD$10 voucher because this was the smallest denomination we could purchase. In our wave 2 survey, we changed this amount to $10 in our invitations to download the app.

We planned to estimate both first and second-order effects of the treatment interventions, including the effect of the MHapp on mental health and learning outcomes. However, while we are strongly powered to estimate first-order effects, lower than expected participation in our baseline survey, high attrition across the semester, and lower than expected take-up of the app have hindered our efforts in estimating second-order effects.

We did not anticipate challenges linking students to the MoodMission data; therefore, the use of self-reported downloads was not specified in our PAP. Remaining columns (on missions completed) only use the data supplied by MoodMission (we did not ask for self-reports on missions completed).

Low engagement is typical of mental health apps. For example, comparing many different apps, Baumel et al. (2019) found a median app retention rate of only 3.9% 15 days after download.

Descriptive statistics for those who answered these questions are in Appendix Table C1.