Usability evaluation of virtual reality-based cognitive intervention for older people: a systematic review

1 Introduction

According to the World Population Prospects 2024, the older population is estimated to reach 2.2 billion in 2070 (United Nations, 2024). The World Health Organization often uses 60 years and older in global ageing and public health contexts (World Health Organization, 2025) while gerontology literature frequently defines the term “older people” as those who are aged 65 years and above (Orimo et al., 2006; Simões et al., 2025). In the context of this study, older people are defined as individuals aged 55 years and above, consistent with prior Human–Computer Interaction research (Knowles et al., 2026; Broekhuis et al., 2019). This age threshold is adopted because age-related changes affecting human–technology interaction may emerge before the conventional definition of the elderly. Besides, older people are more likely to experience aging, which involves the progressive loss of function with increasing age. This may include significant declines in cognitive function among older adults (Ji et al., 2024), also known as mild cognitive impairment (MCI). MCI has been identified as a transitional stage between normal aging and Alzheimer’s disease (AD) (Ai et al., 2025). Furthermore, older people are facing challenges in their daily life, especially during the post Coronavirus Disease 2019 (COVID-19) pandemic, such as a lack of employment opportunities, limited lifelong learning, and financial instability (Monahan et al., 2020). In other words, older people may not be able to enjoy their golden years despite having a longer lifespan. Thus, there is a need to maintain and enhance cognitive capacity in older adults through cognitive interventions (Yun and Ryu, 2022).

There is a growing body of literature that recognizes the importance of cognitive intervention. Cognitive intervention has been identified as a non-pharmacological approach that older people can seek without medication (Forte et al., 2025). There are three types of cognitive intervention for older people, namely, cognitive rehabilitation, cognitive stimulation, and cognitive training (Clare and Woods, 2004; Yun and Ryu, 2022). Cognitive stimulation refers to various group intervention activities that engage older persons, especially to enhance their cognitive and social interaction (Gómez-Soria et al., 2023). Cognitive training is intended primarily as a guided intervention to restore and strengthen an older person’s specific cognitive function through repeated practice (Silva et al., 2024). Cognitive rehabilitation is an individualized intervention used to help an older person to improve his/her functional skills in activities of daily living (Ren et al., 2024). Typically, all cognitive interventions shall be conducted by professional healthcare, and the cognitive performances of older people are measured using conventional methods such as Montreal Cognitive Assessment (MoCA) (Pai and Bankhele, 2020). Previous studies have shown that virtual reality (VR) can be used as a potential technology to support cognitive training (Navarro-Ramos et al., 2025; Bauer and Andringa, 2020), cognitive rehabilitation (Maggio et al., 2025; Maggio et al., 2019), and cognitive stimulation (Gamito et al., 2020).

VR is a technology that simulates a three-dimensional environment which can be similar to or entirely different from the physical world (Felber et al., 2025). VR is also called a virtual environment (Chen et al., 2000) and a virtual world (Doerner et al., 2022). With reference to the Milgram’s reality-virtuality continuum (Milgram et al., 1995), VR is the opposite extreme of reality. The mixed reality is associated with the area between these two extremes. Various mixed reality technologies are being developed, such as augmented reality (Huang et al., 2025) and augmented virtuality (Polydorou, 2021). Compared with mixed reality, VR provides a safe alternative environment where older people can undergo cognitive intervention with minimal risk of injury.

VR also offers several benefits over other technologies. For instance, VR enabled older people to experience presence in different places without being physically present (Kim and Kang, 2023). This is beneficial for those who have limited mobility and financial resources. Older people can also visit historical places through virtual reconstruction (Antoniou et al., 2025). In addition, VR is considered cost-effective (Appel et al., 2020) because it requires only a one-time cost installation, and older people can reuse the VR for various purposes, such as training (Ip et al., 2025), gaming (Cheng et al., 2025), social engagement (Moreira Kares et al., 2025) and others. However, there are certain drawbacks associated with the use of VR. Older people tend to suffer severe cybersickness, especially in virtual environments (Li et al., 2025). This may include nausea, oculomotor discomfort and disorientation. Additionally, older people may be at risk of vision problems due to heavy eye strain when using VR technology (Schaumburg et al., 2025).

Usability evaluation is important in designing and developing VR-based cognitive interventions because poor usability would hamper older people from successfully performing rehabilitation, stimulation, or training tasks. Furthermore, the low motivation and adherence to using VR applications among older people could significantly impact the effectiveness of VR applications in treating older people, including but not limited to people with MCI, AD or other cognitive impairments. Usability evaluation methods can be categorized into five main categories, namely testing, inspection, inquiry, analytical modeling, and simulation (Ivory and Hearst, 2001).

According to ISO 9241-210:2019 (International Organization for Standardization, 2019), usability is the degree to which a technology can be used with effectiveness, efficiency, and satisfaction by specific users in a particular context to achieve an objective. In this paper, technology means VR applications that are developed to support cognitive intervention. The terms “VR application” and “VR-based cognitive intervention” are used interchangeably throughout this paper. Users are the participants who took part in the usability evaluation of VR applications. Effectiveness refers to the accuracy and completeness in achieving the objective, whereas efficiency is defined as the amount of resources used to achieve the same objective. The “satisfaction” term and “user experience” term share almost the same definition (Kurosu et al., 2018), where both terms can be defined as the participant’s perceptions and responses after using the VR applications or anticipated use of VR applications.

Past systematic reviews based on medical and scientific databases revealed that only a small number of research studies evaluated the usability of VR-based cognitive intervention (Tuena et al., 2020; Contreras-Somoza et al., 2021). However, what is not yet clear is the extent to which usability standards and evaluation methods are more suitable for evaluating VR-based cognitive interventions. Most human factor engineers and human-computer interaction professionals are more likely to share their research on usability evaluation in computing databases (Holden et al., 2022). So, this research seeks to obtain secondary data from major computing databases to identify evaluation methods and usability standards applicable to VR-based cognitive intervention, addressing a critical interdisciplinary gap.

The remaining part of the paper is organized as follows: First, Section 2 reviews in detail the available information on existing systematic reviews related to the usability of VR-based cognitive intervention. Section 3 describes the methodology employed for this research. Then, Section 4 presents the research findings, focusing on five main subsections: characteristics of the included studies, the participants involved, the hardware used, the supported context of use, and the usability evaluation of the VR-based cognitive intervention. Section 5 discusses the significance and the implications of the findings for future research. Finally, Section 6 identifies the systematic review limitations, and Section 7 concludes this research.

2 Related works

There are 14 types of review in secondary research, namely, critical review, literature review, mapping review, meta-analysis, mixed studies review, overview, qualitative systematic review, rapid review, scoping review, state-of-the-art review, systematic review, systematic search and review, systematized review, and umbrella review (Grant and Booth, 2009). A systematic review is the best-known review type because other researchers can reproduce the research based on its detailed documentation of the methodology used. The systematic review is a methodology to find primary research evidence for answering a set of pre-defined research questions (Brignardello-Petersen et al., 2025).

Several systematic reviews have been reported in the literature, focusing on different aspects of VR applications. For example, a systematic review identified 17 prior studies on VR-based cognitive training for patients with MCI in electronic databases (Zhong et al., 2021). The review searched the PubMed, Embase, Web of Science, Cochrane Library, Science Direct, and EBSCOhost databases until May 2021. The review established that the VR-based cognitive training improved MCI patients’ global cognitive function significantly from the clinical perspective. Contrary to expectations, the meta-analysis did not find a significant effect of VR-based training on delayed memory, immediate memory, attention and instrumental activities of daily living for MCI patients. The scope of the review was limited to randomized controlled trials using VR-based cognitive training only. Even so, the review failed to address the importance of usability evaluation on these VR applications.

Researchers (Zhu et al., 2022) used CiteSpace software to conduct a bibliometric analysis of 230 publications linked to the VR-based cognitive intervention. Bibliometric analysis is a quantitative method to analyze scientific publications (Donthu et al., 2021). The review searched only one database, Web of Science, from 1995 to 2020. Most publications were issued in the United States of America. The review highlighted that Pugnetti published the first article on a VR application for people with cognitive impairment (Pugnetti et al., 1995). The review also shared the research trend of VR in terms of the influential authors (like Guiseppe Riva and Tarnanas I), hot research topics (such as physical activity, working memory, single-blind, walking, and risk), and future research direction (which included exergame, serious games, spatial navigation, wayfinding, and activities of daily living). Anyway, the usability evaluation of VR-based cognitive intervention was not yet an emerging trend. The main weakness of the review was that only a scientific database was selected.

A recent systematic review (Skurla et al., 2022) suggested that more research studies should address the usability of VR applications for older people. The suggestion was based on 55 journal articles, filtered from PubMed and Web of Science databases between January 2008 and October 2020. The review classified these articles into three main VR categories, namely testing, training, and screening; 25 training articles reported that VR applications improved older people’s cognitive skills after regular use; 21 screening articles provided strong evidence that VR tools were suitable for the diagnosis of older people with cognitive impairment; 15 testing articles revealed that performance using VR applications was correlated with performance of traditional neuropsychological measures. The review also noted that some previous articles fulfilled more than 1 category. The review concluded that extensive research on VR use among older people is required, particularly to ensure the development of usable VR technologies.

Another systematic review (Tuena et al., 2020) conducted an online search on September 2019 and included 25 experimental studies (Brox et al., 2017; Valladares-Rodriguez et al., 2019; Tsai et al., 2012; Nikitina et al., 2018; Sáenz-De-Urturi et al., 2015; Pedroli et al., 2018; Cook and Winkler, 2016; Castilla et al., 2013; Corno et al., 2014; Morán et al., 2015; Vanbellingen et al., 2017; Trombetta et al., 2017; Im et al., 2015; Wüest et al., 2014; Rebsamen et al., 2019; Plechatá et al., 2019; Money et al., 2019; Kiselev et al., 2015; Shubert et al., 2015; Schwenk et al., 2014; Van Beek et al., 2019; Desteghe et al., 2017; Epelde et al., 2014; Fordell et al., 2011; Kizony et al., 2006) from PubMed, PsycInfo, and Web of Science databases. The review showed that different populations of older people had participated in evaluating the usability of non-immersive VR applications. Most previous studies involved healthy older people. In some pilot studies, medical professionals were included to obtain their expert opinions on the VR applications. Most previous studies have reported on the usability of VR applications for motor rehabilitation. Most studies used mixed approaches that combined qualitative and quantitative analyses. Technology acceptance models were commonly used as theoretical frameworks in previous studies. The VR applications were rated as usable and suggestions for improving them were included as well. But the review did not include studies from the computing databases.

To further examine the key research direction on the usability and user experience of cognitive intervention technologies for older people with MCI or dementia, a group of researchers (Contreras-Somoza et al., 2021) reviewed 10 previous studies reported in the Medline, PubMed, Scopus, ScienceDirect, and PsycInfo databases. The online search was limited from 2009 to 2019. The review concluded that older people with MCI need more time and help in order to complete tasks given using cognitive intervention technologies. Half of the past studies (Tziraki et al., 2017; González-Abraldes et al., 2010; Djabelkhir et al., 2017; Boulay et al., 2011; Ben-Sadoun et al., 2016) focused on cognitive stimulation technologies, while the remaining studies (Manera et al., 2015; Kyriazakos et al., 2017; Haesner et al., 2015; González-Palau et al., 2013; Garcia-Sanjuan et al., 2017) concerned on cognitive training technologies. Six previous studies measured both usability and user experience, and the other four past studies measured user experience only. Although facing specific difficulties with interfaces, the older people shared positive experiences through the questionnaire. The review also pointed out that future work should establish standardized tools and a consensus on the usability and user experience terms for evaluating cognitive intervention technologies. The review has not been able to identify any previous study that documents the usability or user experience of cognitive rehabilitation technology. Similar to the above review’s limitation, the review pointed out the need for further investigation into computing and engineering publications.

This paper aims to shed new light on the usability evaluation of a VR-based cognitive intervention conducted by computing professionals. Compared to earlier reviews, this paper seeks to close the research gap by searching and analyzing the highest quality of technical literature in computing databases. The major contributions of this paper are unraveling trends of relevant usability evaluation published in different journals and conferences, identifying evidence of VR-based cognitive intervention for older people, and suggesting future research opportunities.

3 Methodology

Although protocol registration platforms such as PROSPERO are commonly used in healthcare and clinical systematic reviews, formal registration is not a standard requirement in the computing and human-computer interaction research domains (Chakravorti et al., 2025). To ensure methodological rigor and transparency, the authors developed and followed a predefined review protocol throughout the study. This section describes the steps taken to conduct the systematic review in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

3.1 Research questions

The research questions in this review focused on:

• Who are the participants involved in the usability evaluation of VR-based cognitive intervention?
• What are the hardware technologies used in the usability evaluation of VR-based cognitive intervention?
• Which context of use is supported by the VR-based cognitive intervention in usability evaluation?
• How is the usability evaluation of VR-based cognitive intervention carried out?

3.2 Data sources

The Association for Computing Machinery (ACM) Digital Library, and Institute of Electrical and Electronics Engineers (IEEE) Xplore Digital Library were searched as data sources for this systematic review. These digital databases were selected because both contained high-quality, peer-reviewed publications, especially in the field of computing technology. The ACM Digital Library search engine also searched other computing literature in major publisher databases, including Springer-Verlag, Elsevier Science Publishers B. V., John Wiley & Sons, Inc. and others (ACM, n.d.). Since past studies had reported the findings from other medical databases (such as Medline, PubMed, Cochrane, and PsycInfo) and science databases (such as Web of Science, and Scopus), this paper extended the literature by searching publications from computing databases only.

3.3 Keyword

The literature search was conducted in April 2022 using the ACM Digital Library and the IEEE Xplore Digital Library. In the ACM Digital Library, the following search string was applied using the “Anywhere” filter in the ACM Guide to Computing Literature: “usability” AND (“virtual reality” OR “virtual environment”) AND (“mild cognitive impairment” OR “dementia” OR “alzheimer”) AND (“cognitive rehabilitation” OR “cognitive intervention” OR “cognitive stimulation” OR “cognitive training”). The selected search keywords covered the broad progression from MCI to AD so that no potential articles were left out during the search. However, this paper did not cover other types of dementia such as Lewy Body, Parkinson’s disease, vascular dementia, Down syndrome and others. For the IEEE Xplore Digital Library, the same search keywords were applied to both the full-text and metadata fields in the advanced search, with the results limited to journal and conference publications.

3.4 Selection criteria

There were three main phases of selection criteria applied in this systematic review, namely (1) publication identification, (2) title and abstract screening, (3) full-text and quality assessment. First, this systematic review began by ensuring all publications in conferences and journals were downloadable. Second, both authors identified potential publications for this review by screening the publication titles and abstracts. Publications with duplicate titles were identified and removed. If the titles or abstracts were associated with the systematic review, doctoral consortium, non-VR application, or off-topic, then these publications were discarded. If an abstract was considered relevant to VR application and contained empirical evaluation, then this publication would be eligible for full-text assessment. Third, the authors independently reviewed the full texts of the remaining publications using the inclusion and exclusion criteria. A consensus was reached by discussing the discrepancies found and the short list of publications for this systematic review.

The following inclusion criteria were used: (1) these publications reported usability evaluation of VR applications involving older people, (2) these publications also focused on various types of cognitive intervention such as cognitive rehabilitation, cognitive stimulation, and cognitive training, and (3) these publications were written in the English language.

This systematic review excluded publications that involved VR application, but with the following characteristics: (1) did not include a description of usability evaluation, (2) did not involve older people in usability evaluation, (3) did not include cognitive intervention, (4) the same data were reported in another publication, and (5) the content was written in non-English languages.

Subsequently, a quality assessment was conducted to evaluate the reporting clarity, methodological rigor, and relevance of the publications included in this systematic literature review. The quality assessment aimed to ensure that the selected publications provided sufficient and reliable information related to the VR-based cognitive intervention, usability evaluation procedures, and participant population relevant to the research questions of this systematic review. Each selected publication was evaluated using the quality assessment questions presented in Table 1.

Table 1. Quality assessment questions.

A three-point scoring scheme was applied for each quality assessment question: Yes = 1, Partially = 0.5 and No = 0. The total quality score for each publication was calculated by summing the scores obtained from all assessment criteria. Publications scoring above 2 were included in the final review, while publications with scores of 2 or below were excluded from the review. This process helped ensure that only publications with adequate methodological quality and relevance were considered in the analysis. Both authors independently conducted the quality assessment for all shortlisted publications using the predefined evaluation criteria. Any disagreements between the authors were discussed and resolved through mutual consensus. This process helped minimize potential assessment bias and improve the credibility of the quality evaluation.

3.5 Data extraction and synthesis

One author extracted the following data from each included study, namely (1) main characteristics of included studies, (2) primary participants’ characteristics, (3) secondary participants’ characteristics, (4) input and output devices of VR-based cognitive intervention, (5) context of use characteristics, (6) usability evaluation characteristics, (7) the quantitative measures, and (8) the qualitative measures. Due to the varying levels of detail provided in the included studies, both authors discussed them periodically to resolve the disagreements and uncertainties regarding data extraction.

4 Results

The online search resulted in 113 publications, where 49 publications were from the IEEE Xplore Digital Library and 64 publications were from the ACM Digital Library (see Figure 1).

Before screening, 20 publications were excluded because 13 were table of contents, 3 were books, and 4 were not retrievable from digital libraries. Next, the first screening was carried out on the remaining 93 publications using their titles and abstracts. Another 53 publications were excluded due to duplicate publication titles (N = 2), systematic reviews (N = 17), doctoral consortiums (N = 2), off-topic (N = 11), and non-VR applications (N = 21). The subsequent round involved full-text assessment and 32 publications were excluded. The reasons for exclusion were: did not include cognitive intervention (N = 9), did not include usability evaluation (N = 14), did not include older people as evaluation participants (N = 5), and content written in non-English language (N = 2). Two publications (Morán et al., 2013; Zhao et al., 2020) had been classified by the authors as duplicates because the same data were reported in another publication. In the final round of quality assessment, one publication (Tabbaa et al., 2019) was excluded because the participant cohort comprised a mixed-age adult population, including, but not limited to older adults, which did not meet the QA3 criteria (see Table 2).

Table 2. Evaluation of the studies based on the quality assessment criteria.

4.1 Characteristics of the included studies

A total of seven reviewed studies were included in this systematic review (Morán et al., 2015; Guo et al., 2004; Gerling et al., 2011; Yamaguchi et al., 2012; Baker et al., 2020; Graf et al., 2020; Karaosmanoglu et al., 2021). The earliest study identified in this review was published in 2004 (Guo et al., 2004). All reviewed studies involved more than one author, with some having at least three authors. Most of the authors were from Germany (Gerling et al., 2011; Graf et al., 2020; Karaosmanoglu et al., 2021). Two reviewed studies exhibited international collaboration (Guo et al., 2004; Gerling et al., 2011). Five of the reviewed studies were published in the journal, and the remaining were reported at the conference (see Table 3). These two conference papers were classified as full papers based on ACM classification (ACM, 2025). These conference papers contained sufficient methodological information on the usability evaluation of VR-based cognitive intervention.

Table 3. Main characteristics of the included studies.

4.2 Who are the participants involved in the usability evaluation of VR-based cognitive intervention?

There were two categories of participants identified in the reviewed studies, namely primary and secondary participants. The older people who used the VR application during usability evaluation sessions were the primary participants. The secondary participants were those who provided additional information about the primary participants due to their professional or social relationships with them. The characteristics of each participant category are described in the section below.

A total of 100 primary participants took part in the usability evaluation of VR-based cognitive intervention (refer to Table 4). Five reviewed studies had obtained primary participants’ permission through a written informed consent form (Gerling et al., 2011; Yamaguchi et al., 2012; Baker et al., 2020; Graf et al., 2020; Karaosmanoglu et al., 2021). The distribution of primary participants by gender was 41% female, 17% male, and 42% unknown. All reviewed studies involved older participants, and the oldest participant reported was 88 years old. Two primary participants with MCI (Baker et al., 2020) and two primary participants with AD (Yamaguchi et al., 2012) were reported. Although most cases of dementia are related to AD (Leng and Edison, 2021), dementia may be secondary to underlying conditions. Therefore, to reduce the risk of misclassification bias, participants with dementia were classified under other impairment categories. There were 44 primary participants with other impairments such as 13 mild dementia (Gerling et al., 2011; Baker et al., 2020; Karaosmanoglu et al., 2021), 1 moderate dementia (Karaosmanoglu et al., 2021), 1 dementia (Baker et al., 2020), 13 visual impairment (Graf et al., 2020), 1 hereditary spastic paraparesis (Baker et al., 2020), and 15 decrements in physical abilities (Gerling et al., 2011). The remaining 52% of primary participants were considered healthy older people. Only 3% of primary participants used VR technology and 18% used computers. The number of primary participants per study ranged from 4 to 32.

Table 4. Primary participants’ characteristics.

AD, Alzheimer’s disease; COM, computer; D, dementia; DP, decrements in physical abilities; F, female; H, healthy; HSP, hereditary spastic paraparesis; M, male; MCI, mild cognitive impairment; MD, mild dementia; MoD, moderate dementia; N/A, not available; VI, vision impairment; VR, virtual reality; WOI, with other impairment.

Four reviewed studies involved secondary participants during the usability evaluation of VR-based cognitive intervention. In this review, these secondary participants are divided into two main sub-categories, namely, health professionals and administrative staff (see Table 5). The health professionals are made up of highly skilled individuals who work in the medical field such as rehabilitation experts, physiotherapists, clinical care coordinators, and nursing staff (Guo et al., 2004; Gerling et al., 2011; Karaosmanoglu et al., 2021; Baker et al., 2020). The administrative staff comprises individuals who support the daily operations of an aged care organization for older people, including care workers, center managers, and leisure and lifestyle assistants (Baker et al., 2020; Karaosmanoglu et al., 2021). Sometimes, the administrative staff is assigned an assistant role to tailor the VR application to the primary participant’s needs.

Table 5. Secondary participants characteristics.

CCC, clinical care coordinator; CM, centre manager; CW, care worker/giver; HP, health professional; LLA, leisure and lifestyle assistant; N/A, not available; NS, nursing staff; PT, physiotherapist; RE, rehabilitation experts.

4.3 What are the hardware technologies used in the usability evaluation of VR-based cognitive intervention?

The VR-based cognitive intervention hardware can be divided into two main groups, namely input devices and output devices (Anthes et al., 2016). The input devices comprise various controllers, tracking sensors, and microphones, while the output devices comprise visual screens, head-mounted displays, and audio, as shown in Table 6.

Table 6. Input and output devices for VR-based cognitive intervention.

C, computer/desktop; L, laptop; TV, television; OR, oculus rift; OG, oculus go; DG, data glove; HVPE, HTC Vive Pro Eye; VI, Valve Index; M, mouse; TS, tracking sensors; TC, touch controllers; TDPC, three-dimensional printed camera; WC, web camera; G, gripper; NWR&BB, Nintendo Wii Remote and Balance Board; Y, yes; N, no.

Five reviewed studies utilized visual display screens in VR-based cognitive intervention such as computers, laptops, televisions, and others (Guo et al., 2004; Gerling et al., 2011; Yamaguchi et al., 2012; Karaosmanoglu et al., 2021; Morán et al., 2015). Apart from that, four reviewed studies enabled participants to be more immersive in the simulated environment through head-mounted displays like Oculus Rift, Oculus Go, HTC Vive Pro Eye, and Valve Index (Guo et al., 2004; Graf et al., 2020; Karaosmanoglu et al., 2021; Baker et al., 2020). Besides that, four reviewed studies supported audio output in the VR-based cognitive intervention (Yamaguchi et al., 2012; Graf et al., 2020; Karaosmanoglu et al., 2021; Baker et al., 2020).

Seven reviewed studies encouraged participant interaction using various controllers in VR-based cognitive intervention such as a mouse (Yamaguchi et al., 2012), a data glove (Guo et al., 2004), a gripper (Morán et al., 2015), touch controllers (Baker et al., 2020), the Nintendo Wii Remote, the Nintendo Wii Balance Board (Gerling et al., 2011), the Oculus Go Controller (Graf et al., 2020), and a three-dimensional (3D) printed camera (Karaosmanoglu et al., 2021). Two reviewed studies employed web cameras and tracking sensors to track the participants’ motions such as head, body, hand (Baker et al., 2020), and arm positioning (Morán et al., 2015). Only one reviewed study used a microphone, in which each participant was required to speak aloud what he/she was going to do (Yamaguchi et al., 2012).

4.4 Which context of use is supported by the VR-based cognitive intervention in usability evaluation?

Table 7 presents the details of the context of use for each study. One reviewed study implemented a non-task-oriented approach, in which participants explored freely within commercial VR content (Baker et al., 2020). The reviewed study explored VR applications as a potential cognitive intervention, with an exploration duration of 1 h. The participants were given the opportunity to explore both indoor and outdoor environments.

Table 7. Context of use characteristics.

CI, cognitive intervention; CR, cognitive rehabilitation; CS, cognitive stimulation; CT, cognitive training; I, individual; G, group; NTO, non-task-oriented; SRILM, self-recorded instructional learning method; TO, task-oriented; WILM, written instructional learning method.

In contrast, the remaining six reviewed studies adopted a task-oriented approach to designing VR content and were classified into different types of cognitive intervention. Two studies stressed cognitive rehabilitation. The tasks mainly dealt with the activity of daily living where participants were asked to complete a virtual cooking task (Guo et al., 2004; Yamaguchi et al., 2012). Cognitive rehabilitation took place in indoor-based environments such as kitchens. During cognitive rehabilitation, the participants were also required to repeat the same virtual cooking task several times.

Meanwhile, the game concept was commonly used in cognitive training and cognitive stimulation. During cognitive training, these participants were required to play a simple game involving walking, counting, pointing (Gerling et al., 2011), or memory tasks (Graf et al., 2020). The participants would spend about 30 min in the virtual forest, an outdoor environment. For cognitive stimulation, the participants were asked to play either a daily living task (Morán et al., 2015) in the indoor based environment or the reporter role with a photo-taking task (Karaosmanoglu et al., 2021) in the outdoor-based environment. The game lasted between 20 and 45 min.

All reviewed studies supported individual use with each participant using the VR-based cognitive intervention individually. Moreover, two reviewed studies demonstrated that the VR-based cognitive intervention was also effective in a group setting. It is evident that one reviewed study allowed three co-located participants to play the game simultaneously (Gerling et al., 2011), and another allowed other co-located participants to watch a participant play the game on a large television screen (Karaosmanoglu et al., 2021).

4.5 How is the usability evaluation of VR-based cognitive intervention carried out?

Table 8 summarizes the usability evaluation characteristics reported in the seven reviewed studies, including the data collection type, location, and usability evaluation methods. Five reviewed studies collected both qualitative and quantitative data types (Guo et al., 2004; Gerling et al., 2011; Yamaguchi et al., 2012; Graf et al., 2020; Morán et al., 2015), but the other two reviewed studies asserted qualitative data only (Karaosmanoglu et al., 2021; Baker et al., 2020). Four reviewed studies conducted usability evaluations in a natural setting such as an aged care facility or participant’s home (Gerling et al., 2011; Graf et al., 2020; Karaosmanoglu et al., 2021; Baker et al., 2020), and two reviewed studies involved an experimental setting (Guo et al., 2004; Yamaguchi et al., 2012).

Table 8. Usability evaluation characteristics.

ACF, aged care facility; E, experiment setting; FG, focus groups; FSS, Flow Short Scale; GEQ, Game Experience Questionnaire; I, interview; IO, indirect observation; ISOQ, ISO-Norm Questionnaire 9241/10; L, qualitative methods; N/A, not available; O, observation; PANAS, positive and negative affect schedule; PEQ, Post Experiment Questionnaire; PH, participant’s home; PM, performance measurement; PQ, Presence Questionnaire; SSI, semi-structured interview; SSQ, Simulator Sickness Questionnaire; T, quantitative methods; TAM, Technology Acceptance Model Questionnaire; TUI, Technology Usage Inventory; VRI, Visual Rating Instrument.

Among these seven reviewed studies, two usability evaluation methods were found, mainly testing methods and inquiry methods. Only four reviewed studies utilized testing methods where the participants’ performances were measured (Gerling et al., 2011; Guo et al., 2004; Yamaguchi et al., 2012; Morán et al., 2015).

Inquiry methods were commonly used including questionnaires, observations, and interviews. There were ten different questionnaire types utilized during usability evaluation such as game experience questionnaire (Gerling et al., 2011; Graf et al., 2020), simulator sickness questionnaire (Graf et al., 2020; Yamaguchi et al., 2012), presence questionnaire (Graf et al., 2020; Yamaguchi et al., 2012), flow short scale (Graf et al., 2020), technology acceptance model questionnaire (Morán et al., 2015), technology usage inventory (Graf et al., 2020), ISO-norm questionnaire 9241/10 (Gerling et al., 2011), visual rating instrument (Baker et al., 2020), post-experiment questionnaire (Guo et al., 2004), and positive and negative affect schedule (Graf et al., 2020).

Six reviewed studies applied observation techniques during the usability evaluation session (Gerling et al., 2011; Yamaguchi et al., 2012; Morán et al., 2015; Graf et al., 2020; Karaosmanoglu et al., 2021; Baker et al., 2020). One of these seven studies used an indirect observation by reviewing video recordings from an earlier usability evaluation session (Morán et al., 2015). After the participants used the VR-based cognitive intervention, four reviewed studies used interview methods to get the participants’ feedback on their experiences, such as verbal interview (Graf et al., 2020; Yamaguchi et al., 2012), semi-structured interview (Baker et al., 2020), and focus group (Karaosmanoglu et al., 2021). Only one study used a semi-structured interview at the beginning of the usability evaluation (Baker et al., 2020) to gather each participant’s background, such as past experience with other technologies and length of stay/work in an aged care facility.

Usability can be measured through both qualitative and quantitative data. Two quantitative measures were identified in the reviewed studies: effectiveness and efficiency. Effectiveness was measured in the form of perseveration errors made by the participants (Yamaguchi et al., 2012) and the score achieved by the participants (Morán et al., 2015; Gerling et al., 2011; Yamaguchi et al., 2012). The completion time (Guo et al., 2004; Yamaguchi et al., 2012) and additional useless actions (Yamaguchi et al., 2012) were considered as efficiency. In addition to these two usability measures, VR-based cognitive intervention was evaluated through user perceptions by collecting self-reported responses via rating scales and opinion-based items (Lim, 2013). For instance, perceived usability, immersive experience, hedonic experience, safety and physical comfort, behavioral acceptance and universal user experience. Table 9 presents quantitative measures from the usability evaluations in the reviewed studies.

Table 9. The quantitative measures.

ABS, absorption; ACC, access(ibility); ACQ, acquisition; ANT, anticipation; ACT, activity; AFR, afraid; AUA, additional useless actions; CCN, concern; CHA, challenge; COM, competence; CON, concentration; CT, completion time; ENJ, enjoyment; EYE, eyestrain; FAT, fatigue; FLO, flow; GD, general discomfort; HEA, headache; IMM, immersion; INTF, interfere; INV, involvement; JOY, joy; NA, negative affect; PA, positive affect; PE, perseverations errors; PRE, presence; SCE, skepticism; SCO, scores; SMP, smooth progress; SP, spatial presence; TEN, tension; PANX, perceived anxiety; PEOU, perceived ease of use; PITU, perceived intention to use; PRL, perceived realism; PUE, perceived user experience; PUSA, perceived usability; PUSE, perceived use(fulness); UPS, upset; VER, vertigo.

The reviewed studies also presented five major user experience issues in the form of qualitative data (see Table 10). Certain user experience issues demonstrated conceptual overlap across multiple user experience dimensions. However, each issue was categorized according to the primary experiential or interaction-related concern emphasized in the original study findings.

Table 10. The qualitative measures.

5 Discussion

The present paper reviews empirical research on the usability evaluation of VR-based cognitive intervention for older people. A total of seven studies were selected from the IEEE Xplore and ACM Digital Library, as this review aimed to examine the usability of VR-based cognitive interventions from the perspective of computing professionals. This systematic review aimed to answer four research questions addressing four main components of usability evaluation: participant (who?), VR-based cognitive intervention hardware (what?), the context of use (which?), and evaluation method (how?). The review found that: (1) the usability evaluation primarily involved healthy older people; (2) head-mounted display and controllers were commonly used as VR-based cognitive intervention hardware; (3) most VR contents were designed for individual usage with the task-oriented approach; (4) both qualitative and quantitative data were collected in usability evaluation; (5) the observation method was the most used, and (6) most of the usability issues concerned about behavioral acceptance, safety and physical comfort.

VR-based cognitive intervention should consider that most older people were novice users because they did not have experience using computers and VR. 40% of the participants stayed at aged care facilities. The older people spent most of their time in these facilities and did not have the opportunity to engage in outdoor activities. Consistent with findings from previous review (Tuena et al., 2020), healthy older people were frequently recruited in usability evaluations of VR-based cognitive interventions prior to deployment among cognitively impaired populations. Within the framework of complex intervention development, usability evaluation is generally regarded as part of the feasibility stage in the Medical Research Council framework (Skivington et al., 2021), as it focuses on assessing system acceptability, interaction performance, and implementation-related issues prior to formal clinical evaluation. This feasibility stage provides a safer, more controlled environment to identify usability challenges, interaction barriers, and interface design limitations before involving individuals with cognitive impairments. In contrast, older adults with cognitive impairments were more commonly involved in advanced evaluation stages, including clinical evaluations, controlled trials, longitudinal studies, and real-world deployment evaluations, where the VR-based cognitive interventions had undergone further refinement and validation. Besides dealing with MCI, AD or other cognitive impairments, some older people had physical limitations such as movement confined to a wheelchair, wearing medical glasses, and poor dexterity with hand controllers. The failure to address this issue may lead to participants’ withdrawal from VR-based cognitive intervention sessions (Baker et al., 2020). Thus, VR-based cognitive intervention should be designed to be portable, lightweight, easy-to-learn, and outdoor-based virtual environments so that older people would likely use these VR applications for a long-term period.

Another significant finding of this review is that various VR applications have been developed for cognitive stimulation, cognitive rehabilitation, and cognitive training. Only cognitive rehabilitation required the older people to repeat the same activity of daily living. Measuring the performance of older people using the VR application is an essential indicator of the usability of VR-based cognitive intervention. Surprisingly, older people were tasked with playing the game-based VR application during cognitive training and cognitive stimulation. For cognitive training, the VR applications were designed to engage older people (either individually or in groups) and at the same time, they helped to strengthen a particular cognitive function. For cognitive stimulation, the VR application supported group settings where older people took turns as players and shared their VR experiences with others. It is also worth noting that one reviewed study took a different approach during cognitive stimulation for older people by reusing one existing VR application designed specifically for physical rehabilitation (Morán et al., 2015). Besides being a game-oriented experience, it is important to note that older people enjoy exploring the VR environments without completing any specific task.

Most of the VR-based cognitive interventions employed controllers and head-mounted displays, but only a single individual was able to interact with virtual objects in an immersive environment. In addition, a secondary visual display was used during VR-based cognitive intervention either for a trial purpose (Guo et al., 2004) or to share the VR content with secondary participants such as health professionals, administrative staff, and other older people (Karaosmanoglu et al., 2021). VR applications may be redesigned to allow multiple persons to join the immersive virtual environment together (Afifi et al., 2021) at the same time, especially during cognitive stimulation. Older people may feel isolated and disconnected from the rest of the world when each person is asked to experience the virtual environment individually. The immersive collaborative environment would promote social communication and interaction among co-located older people by allowing each person to feel the other’s presence in the virtual world. Additionally, family members and loved ones may engage with older people by viewing the virtual environment together. Therefore, future studies on the immersive collaborative environment for older people are recommended.

The most striking observation to emerge from this review is that none of the reviewed studies make use of existing usability questionnaires to gather subjective responses from older people, such as Questionnaire for User Interaction Satisfaction (Harper and Norman, 1993), Software Usability Measurement Inventory (Kirakowski and Corbett, 1993), Post-Study System Usability Questionnaire (Lewis, 1992), and System Usability Scale (Brooke, 1996), which is a rather unexpected outcome. This omission is unexpected, as standardized questionnaires provide reliable and comparable measures of user satisfaction, ease of use, and overall interaction experience, which are particularly important when evaluating VR-based cognitive interventions for older adults.

Beyond core usability considerations, the reviewed studies indicated that VR-based cognitive interventions are frequently assessed across multiple user experience-related dimensions, including immersive experience, hedonic experience, safety and physical comfort, behavioral acceptance and universal user experience. These findings suggest that usability in VR-based cognitive interventions extends beyond traditional efficiency and effectiveness measures to encompass broader experiential and human factors dimensions. Accordingly, there is a clear need for a more standardized and comprehensive usability evaluation framework tailored specifically to VR-based cognitive interventions to ensure consistency and comparability across studies.

Table 9 shows that three of the reviewed studies measured both usability and user experience after older people tried the VR application. Besides usability, the user experience was one important factor influencing older people’s acceptance of VR applications. Previous research revealed that the poor usability was closely associated with a negative user experience (Raita and Oulasvirta, 2014). However, a new finding from this review was that most participants (including both older people and health professionals) in the reviewed studies had a high level of tolerance for the poor usability of VR applications, such as difficulty using the controller. A possible reason may be that most participants paid much more attention to the usefulness than to the usability of VR applications for cognitive intervention. In particular, the older participants thought the VR application was usable as reflected in both questionnaire measurement and the interview after trying the VR applications. Thus, they might tolerate the poor usability of the VR application to achieve the “expected” treatment goal.

The review results suggest that there are missing comprehensive studies on usability evaluation of VR-based cognitive intervention in computing research, which are consistent with a few studies reported in previous systematic reviews, such as 10 studies (Contreras-Somoza et al., 2021) and 25 studies (Tuena et al., 2020). The main reason accounting for the lack of a comprehensive study could be the lack of a suitable battery of usability evaluation kits for the VR-based cognitive intervention. Given older people’s declining and varied cognitive capacities, the VR-based cognitive intervention needs to be highly adaptive to suit the older people, including but not limited to people with MCI, AD or other cognitive impairments. The existing usability instruments mainly focus on the users with normal motor and mental capacities. Some researchers modify these usability instruments to measure the usability of VR applications for older people (Gerling et al., 2011; Graf et al., 2020). Most usability evaluations had small sample sizes because six out of seven reviewed studies had fewer than 20 participants. Therefore, the effectiveness and efficiency of these modified usability instruments need to be tested in large-sample studies.

Depending on their intended use and regulatory context, some VR-based cognitive interventions may fall within the scope of medical-device regulations, particularly when they are designed for therapeutic or clinical purposes. In such cases, the design and implementation of these VR applications should comply with the existing usability guidelines for medical devices, such as IEC 62366–1:2015 (International Electrotechnical Commission, 2015). The most unexpected finding from the analysis was that none of the reviewed studies specifically evaluated the usability of VR applications with respect to safety-related aspects. Although one reviewed study employed six questionnaires that measured different aspects of user experiences (Graf et al., 2020) and some reviewed studies indirectly identified safety issues among older people during evaluation (Karaosmanoglu et al., 2021; Baker et al., 2020), the risk associated with user errors, hazardous use scenarios, and safety failures during interaction with the VR application was not addressed. The reason could be twofold. First, since the use of VR applications for cognitive intervention is still at its trial stage, researchers and clinicians might not treat it as a formal medical device. Second, the reviewed VR applications generally provided a safe environment for older people. Thus, the usability evaluation did not cover all guidelines required.

From a medical-device usability perspective, identifying and mitigating hazardous use scenarios is essential to ensure that VR-based cognitive interventions can be safely implemented among older people. This consideration is particularly important because older adults may face an increased risk of adverse events during interaction with VR applications. For example, participants experienced neck discomfort due to the weight of the HMD (Karaosmanoglu et al., 2021), while prolonged usage contributed to fatigue and physical strain. Environmental and hardware-related hazards were also observed, such as HMD cables stretching around swivel chairs, which created potential tripping or entanglement risks. In addition, older people with limited digital literacy may have difficulty operating handheld controllers, thereby increasing the likelihood of interaction errors and unsafe use. These findings indicate that VR-based cognitive interventions should not only prioritize usability and user experience but also incorporate safety-oriented design considerations.

6 Limitation of the review

Several limitations to this systematic review need to be acknowledged. First, the present review only searches for the two major computing databases. Since the review focuses on the usability evaluation of a VR-based cognitive intervention led by computing professionals, the results may not capture findings from usability evaluations led by other professionals, such as experts in human factors and psychology. Future research could explore similar studies in other domains. Second, this paper includes a small sample of reviewed studies. Although this review adopted an objective and replicable search strategy to identify relevant evidence, there may have been studies that used “user experience” rather than “usability.” Since these studies may not be included in this review, the present findings may be limited. Third, the literature search was completed in April 2022. Consequently, the review does not include studies published subsequently. The findings should be interpreted within the context of this temporal limitation, as emerging evidence may expand the current understanding of the usability of VR-based cognitive interventions for older people and complement the findings reported in this review. Lastly, studies on usability evaluation of VR applications for cognitive diagnosis were excluded because these VR applications are not designed for treatment purposes. Some previous studies have explored VR applications as cognitive diagnostic tools to assess older people’s cognitive abilities and identify evidence of impairment (Tsai et al., 2021; Yeh et al., 2012).

7 Conclusion

This paper provides an overview of usability evaluation trends published in different computing journals and conferences, identifies evidence of VR-based cognitive intervention for older people, and suggests future research opportunities. The findings suggest several observable trends within the reviewed studies. For example, most participants involved in the evaluations were healthy older people, head-mounted display and controllers were frequently used as VR hardware, most VR contents supported a task-oriented approach for individual use, and the observation was the most used method in usability evaluation. However, these observations are based on a relatively small and heterogeneous body of literature. Therefore, they should be interpreted as indicative rather than conclusive.

The reviewed studies suggest that usability may be an important consideration in the design and implementation of VR-based cognitive interventions. Several studies reported that usability challenges may contribute to user frustration and potentially affect engagement with cognitive intervention. At the same time, the available evidence remains limited, and further empirical research is needed to better understand how usability may influence user experience, long-term adoption, and intervention outcomes. Overall, the findings highlight the need for continued research on usability evaluation in VR-based cognitive interventions and may serve as a useful reference for computing researchers and practitioners developing future VR applications for older adults.

Generative AI statement

The author(s) declared that Generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Serban Georgica Obreja, Polytechnic University of Bucharest, Romania

Reviewed by: Hermine Lenoir, Assistance Publique Hopitaux De Paris, France

Vítor J. Sá, Polytechnic Institute of Porto, Portugal