Investigation of algorithmic facilitation vs. self-led learning in complex skill development for operational training

1 Introduction

Training for high-consequence industries, such as aviation and spaceflight, relies heavily on expert trainer facilitation to ensure trainees are mastering the correct skills at the appropriate speeds. Commercial pilots engage in multi-year training that requires extensive instructor facilitation during ground school, simulator training, and flight training. Even during simulator-based training sessions, significant reliance is placed on the instructor to guide the trainee to meet learning objectives and fill fidelity gaps of the simulator. In their 2018 paper, Myers and colleagues posited that the simulator does not do the training, but should instead be treated as a tool that an instructor can use to support skill mastery (Myers et al., 2018). In simulation learning environments for the spaceflight industry, astronauts also rely on expert trainers to facilitate learning (Dempsey and Barshi, 2018; Pieters and Zaal, 2019). As aviation and spaceflight industries rapidly grow, though, the desire to train larger cohorts of people is growing commensurately. Without a change in the paradigm of human-led facilitation, this growth will require industries to invest in extensive cohorts of instructors. Further, during long duration exploration missions (LDEM), such as a mission to Mars, communication delays of up to 40-min round trip will prohibit trainers from providing hands-on or real-time training facilitation. These mission constraints create a need for automated and innovative training methods that can be used in-flight to mitigate expected skill decay (Barshi et al., 2020; Manzey et al., 2000; Pieters and Zaal, 2019).

Algorithms can facilitate training without the need for direct personnel support by providing dynamic difficulty adjustment (Hunicke, 2005; Xue et al., 2017; Zohaib, 2018). In this method, the challenge of a task is algorithmically modified to match the skill of the user. The process of dynamic difficulty adjustment has been explored in gamified education as a way to improve learning outcomes by increasing engagement, immersion, and flow (Hamari et al., 2016). Engagement is defined by Rieber et al. as a learning environment where the learning goals are intrinsically interesting, bolstering the student’s self-motivation to participate in the learning process (Rieber et al., 1998). Immersion is the perception of being enveloped and interacting with the environment (Witmer and Singer, 1998). Flow is particularly important when using an algorithm for task learning to keep users motivated. The theory of flow (Csikszentmihalyi, 1990) states that players remain engaged in a game when they can travel through a flow channel defined by an optimal mapping of game challenge to player expertise. Through this channel, according to the challenge-point hypothesis, the game is neither too hard (frustrating) nor too easy (boring) (Guadagnoli and Lee, 2004). Computer-based learning games that use dynamic difficulty adjustment to challenge users based on their individual game play ability show improved motivation and learning when compared to incremental difficulty adjustment (Sampayo-Vargas et al., 2013). Despite the broad research on dynamic difficulty adjustment, many studies focus on single-dimension tasks (Cameirão et al., 2010; Flegal et al., 2019), rather than complex operational tasks, where the bottleneck of simulators and facilitators is greatest. Recent work by the authors began to address this gap by investigating important aspects of algorithmic facilitation for complex, operationally relevant skill acquisition (Verniani et al., 2024).

Virtual reality (VR) and other simulation platforms play a crucial role in training for high consequence industries, providing an immersive and realistic environment where trainees can safely practice potentially dangerous operations. VR platforms also introduce automated facilitation capabilities to reduce the need for extensive support personnel, such as the 175-person software support team and 200-person hardware support team who maintained command and lunar module training tools during the Apollo program (Woodling et al., 1973). Further, these kinds of training environments can enable automated facilitated training by directly integrating algorithms into immersive simulations, which some have called for as a critical path to the future of education (Psotka, 2025). While there is a demonstrated need to enable learning of complex skills in an automated way, the best approach by which to facilitate skill acquisition utilizing VR has yet to be determined.

An alternative to algorithmic difficulty adjustment is self-facilitation. Self-regulated learning (SRL) allows a trainee to shape their own trajectory of learning by self-adjusting difficulty levels. SRL is a growing topic of investigation, especially in the era of online learning platforms, allowing students to learn independent of facilitators. The effectiveness of SRL is hypothesized to be modulated by how information is processed on the path to learning (Winne, 2001) and individual differences in motivation to achieve a goal (Meece, 1994). The goals a student or trainee may use to motivate approaching success while learning are often dichotomized as either “mastery-oriented”, where the objective is to increase competence and understanding of a task, or “performance-oriented”, where the objective is focused on demonstrating ability or appearing skilled to external evaluators (Meece et al., 2006).

A person’s tendency to display one type of achievement goal over another is often attributed to innate balances of intrinsic vs. extrinsic motivation (Lee et al., 2010). SRL and motivation studies show large individual differences in how a learner may interact with a training environment, demonstrating a need to account for motivational differences when assessing facilitated vs. self-led training. Some authors argue that performance-orientation results in motivation loss over time, in part due to reliance on external rewards to fulfill needs for extrinsic motivation (Deci et al., 1999). In contrast, intrinsic motivation has been shown to have strong correlations to self-regulation of learning and the use of cognitive strategies to improve engagement (Pintrich and De Groot, 1990). However, a “multiple goal” perspective that blends both performance and mastery achievement goals has been posed to foster valuable intrinsic motivation and improve learning outcomes (Barron and Harackiewicz, 2001).

The impact of self-led vs. instructor facilitation has been investigated in limited contexts. For example, in CPR training, self-instruction is often deemed equivalent in outcome to instructor-led training (Hsieh et al., 2016). One study found individuals reported scores directly after training were significantly higher in the instructor-led group, as were ratings of self-assessed knowledge and willingness to act (Bylow et al., 2019). However, this study found that instructor facilitation improves skill acquisition, but not retention, a result that has been supported by similar studies (Einspruch et al., 2007; Roppolo et al., 2007). Some improved learning outcomes have been seen in self-led video training compared to instructor-led courses immediately after training (Lynch et al., 2005). While these studies share insight on the influence of the instructor-led vs. self-led training in a high-consequence scenario, CPR training is often procedural to ensure providers administer relatively simple steps to save lives. Similarly, adaptive training has shown performance improvements for other procedural tasks, such as completion of aviation check-lists (Aguilar Reyes et al., 2023). Further, these studies do not inform the use of automated facilitated training and its impact on complex skill acquisition.

In intelligent tutoring systems (ITS) research, facilitated learning is investigated while taking into account the learner’s skill. An analysis of trends in technology education has shown ITS to be one of the most rapidly growing fields within the domain of education (Huang et al., 2025). A review by VanLehn found that ITS may narrow the gap with human tutors, as opposed to automated, fixed adaptive, answer-based, and static tutoring (VanLehn, 2011). This review highlights, though, that these systems are most typically applied to traditional classroom-based learning settings, rather than simulations of operational multi-skill environments. Regardless, the research on aspects of ITS, such as student motivation yielding improved learning outcomes, provides motivation and translation of similar approaches to operational task training (Yuce et al., 2019).

A gap remains to directly compare performance outcomes in self-led vs. automated facilitated learning of a complex task requiring multi-dimensional integration of skill. The objective of this research is to investigate the importance of automated facilitated learning in the mastery of complex skills in a high consequence industry. The task used for this research is a simulated Mars entry, descent, and landing (EDL) scenario (Verniani et al., 2024), providing a mission-relevant and complex operational skill that requires consistent motivation and engagement. We hypothesized that for these highly motivated individuals:

• During training, individuals in self-led training conditions will demonstrate greater mastery of the task than those in the algorithmic facilitation group.
• During evaluation, individuals trained using algorithmic facilitation would outperform individuals who were self-led; and
• Algorithm facilitation would result in less frustration and more appropriate challenge-skill balance, resulting in higher flow and confidence than self-regulated or static training conditions.

2 Methods

This research was classified as IRB-exempt by the IRB at the University of Colorado Boulder (IRB #23-0345) and the IRB at the United States Air Force Academy (Protocol FAC20230034E).

2.1 Subjects

Subjects were recruited from USAFA cadets. This was done to recruit from a population that would likely be highly motivated to perform the selected task as well as to have somewhat more consistent prior knowledge on a piloting task. USAFA admission is restricted to highly qualified individuals who demonstrate both academic excellence and leadership. The traits that are looked for in potential cadets often overlap with those sought after in Astronaut candidates-as demonstrated by the 40 NASA astronaut selectees who received undergraduate education at USAFA (U.S. Air Force, 2023). Cadets at USAFA experience operational training and become familiar with air and spacecraft systems. As such, recruitment for this study specifically targeted USAFA cadets. Inclusion criteria were limited to subjects who had either a United States Air Force affiliation or greater than 5 h of experience piloting an aircraft.

A total of n = 45 subjects were recruited for testing. In a previous study of adaptive and nonadaptive VR, an effect size of 1.46 was observed (Jeon, 2013). This corresponds to greater than 80% power at alpha = 0.05 with n = 9 subjects per group. However, in our lab’s experience using this EDL scenario we have observed high variance and lower effect sizes, potentially due to the complexity of the EDL task. As such, we expanded the sample size to n = 15 subjects per group to achieve sufficient statistical power. This corresponds to a minimum detectable effect of 0.48. A summary of subject demographics can be found in Table 1.

TABLE 1. Participant demographics.

n/N (%); Mean (SD).

Pearson’s Chi-squared test; Kruskal–Wallis rank sum test; Fisher’s exact test.

2.2 EDL task

The EDL task, implemented in VR, required subjects to complete three consecutive subtasks within a single trial to safely execute a landing on Mars (Verniani et al., 2024). by calculating the average distance between the subject-selected landing site to each site of scientific interest, D S, and comparing that value to the average distance associated with the most optimal landing site on the map, D O. A crash was scored if the subject failed to select a landing site within the allotted time (8 s) or if they selected a landing site over hazardous terrain. If no site was selected, the computer automatically selected a site allowing the trial to progress. The difficulty of the LS subtask was modified by increasing terrain complexity as well as the number of scientific sites and their distribution on the map. by calculating the root-mean-squared-error (RMSE) between their joystick inputs (P S for pitch, and R S for roll) and the guidance cue (P G for pitch, and R G for roll), where N represents the samples taken over the MC period. A crash was scored if the subject ran out of fuel before reaching their landing site. When an MC crash occurred, the simulator was unable to score the LS (since the score for LS is based on terrain steepness scored on touchdown) and TD tasks, resulting in a score of 0 for the entire trial. The difficulty of the MC subtask was modified by increasing the magnitude, directional complexity, and frequency of wind disturbance on the vehicle. the vertical speed of the spacecraft (change in altitude, a, divided by change in time, t) at the moment when the spacecraft contacted the surface. A crash was scored if the subject ran out of fuel before reaching the surface. TD difficulty was modified by limiting the amount of fuel available to perform this subtask.

• Landing site selection (LS), where subjects assessed a topographic map to select a landing site within a constrained time frame that optimized distance to sites of scientific interest (all sites were equally important) while avoiding hazardous (steep) terrain. Success on this subtask was scored using Equation 1:

S L S = D S − D O D O (1)

• 2. Manual Control Piloting (MC), where subjects executed roll and pitch maneuvers to align their vehicle flight path with a provided guidance cue navigating towards the landing site they selected, requiring the subject to actively null against wind to remain on course. Success on this subtask was scored as shown in Equation 2:

S M C = ∑ i = 1 N P S − P G 2 + R S − R G 2 N (2)

• 3. Terminal Descent (TD), where the subject used a hand thruster to minimize vertical descent speed on touchdown using a limited amount of fuel. Success on this subtask was measured by Equation 3:

S T D = Δ a Δ t (3)

Each of the subtasks had 24 possible difficulty levels (1–24). At the end of each trial, the subject was presented with simple performance feedback rating their performance as “Poor”, “Adequate”, or “Excellent” based on how their score fell on a rating scale determined through pilot testing and previous experimentation (Verniani et al., 2024). The display interfaces used by the subject in performance of the EDL task are visualized in Figure 1.

2.3 Training phase procedures

At the start of the session, all subjects watched a training video that oriented them to task objectives and interactions. Before beginning training trials, knowledge of interface interactions such as which buttons to press on the joystick was verified by a member of the research team. Subjects also completed a demographic survey.

Subjects were randomly assigned to one of 3 training conditions: facilitated training with an adaptive algorithm, self-led training with subject self-selection of difficulty, or static difficulty training representing the baseline condition. In each condition, subjects completed 30 consecutive trials of training with the EDL task within a single session, performed using an Oculus Quest 2 head mounted display for VR presentation. Regardless of the training condition, subjects were instructed to strive to reach “Excellent” ratings on all three subtasks each trial. However, their experience differed in facilitation style.

In the algorithm-facilitated condition, subjects experienced a 2-up, 1-down (2U1D) training algorithm which modulated difficulty based on task performance. In this algorithm, two consecutive “Excellent” ratings were needed for a subtask difficulty to step up by one level. A single “Poor” rating caused a step down in difficulty for that subtask. An “Adequate” rating maintained the subtask at its current difficulty level. This algorithm was applied to each subtask independently, allowing difficulty levels to diverge based on individual performance. This algorithm was created and validated in a previous experiment that utilized the EDL task (Verniani et al., 2024) and was selected for use after testing demonstrated that this algorithm method resulted in the highest asymptote of skill achieved during 30 trials of training. Subjects in this condition began their training at level 12, as it is in the middle of the difficulty scale, such that any individual had the maximum range to drop if the performed poorly (0) or to progress if they performed well (24).

In the self-led training condition, subjects were provided an assessment of their performance across each of the subtasks, and then shown a screen at the end of each trial displaying their current difficulty level along with a slider that allowed them to select which difficulty level they would like to experience on the next trial. They selected only one difficulty level, which was applied to all three subtasks. Subjects were not given any guiding feedback on how they should choose a level-they were told that this selection was up to them, but they had the same objective of reaching “Excellent” ratings on all three subtasks in a given trial.

In the static control condition, all 30 trials were completed at difficulty level 12 out of the 24 possible difficulty levels, regardless of performance. This group served as the control condition for the experiment.

For all subjects, a break was taken after 10 trials of training, dividing the training portion into 3 sections. During this break, subjects removed their VR headset and took the Flow Short Scale survey (Engeser and Rheinberg, 2008). Following their 30 trials of training, all subjects transitioned into a period of evaluation.

2.4 Evaluation phase procedures

Prior to beginning evaluation trials, subjects completed an additional survey about their motivation. The validated Reduced Instructional Materials Motivation Survey (RIMMS) (Loorbach et al., 2015), based on the ARCS model of motivation (Keller, 2010), was slightly modified to remove specific references to “text-based instructional materials”, to match the context of VR training used here. The original RIMMS questions and the rewritten version used in this study are directly compared in Appendix A. Two additional questions were asked to examine differences in motivational tendencies that may influence behavior, specifically in the self-led training condition. The questions were opposites of one another, asking subjects about their motivation in selecting a difficulty level, where subjects rated the following statement as “not true” to “very true” on a 5-point Likert scale.

• “It is more important for me to feel challenged than it is for me to get a good score”
• “It is more important for me to score highly than it is for me to experience the most challenging version of the task”

After answering this survey, subjects were informed that they were entering an evaluation period, where they were asked to perform the task to the best of their ability. Unknown to the subject, the difficulty level during evaluation was static at level 18 of 24. Critically, level 18 was skipped in both the algorithm facilitated and self-led training, meaning no subject had experienced this specific level during the training phase. Level 18 was chosen as the evaluation level as the midpoint between 12, the initial level, and 24, the highest possible difficulty, and was confirmed via pilot testing as an appropriate evaluation level. Subjects completed 5 trials of evaluation in the VR environment, using the same joystick and procedures as the training phase. At the end of this evaluation, one final Flow Short Scale was completed to assess flow during evaluation.

2.5 Statistical methods

Significance was evaluated at the α = 0.05 level and all associated assumptions were evaluated. For each hypothesis, one way ANOVA was used to evaluate differences between each training condition, with Tukey’s HSD for post-hoc comparisons. When data failed to meet parametric assumptions, a non-parametric Kruskal–Wallis rank sum test was used with Dunn post-hoc comparisons with Holm adjustment for multiple comparisons. Statistical analysis was performed in R, and tables were generated using the gtsummary package (Sjoberg et al., 2021).

2.5.1 Training phase analysis

The primary analysis to answer Hypothesis one regarding task mastery is an assessment of subject’s crash rate, or the number of training trials out of 30 where at least one subtask was scored as a crash. This was compared between training groups. Scoring a crash in any single subtask is considered task failure as it represents the inability to successfully complete the integrated task.

To put the crash rates into context, we also assessed training difficulty during these 30 trials to assess how facilitation style impacted average training difficulty. The average difficulty level experienced during training was computed for each subject according to Equation 4. LS, MC, and TD subtask difficulty on a given trial i was averaged to provide a trial average difficulty and further averaged over their 30 trials to provide the average training difficulty for that subject.

A v e r a g e T r a i n i n g D i f f i c u l t y = 1 30 ∑ i = 1 30 L S d i f i + M C d i f i + T D d i f i 3 (4)

The average training difficulty for two subjects in the self-led training group were identified as a moderate outlier, as both data points fell beyond 1.5 times the IQR below the first quartile or above the third quartile. These data points were removed only from training difficulty analysis. The subsequent data met assumptions of normality and equal variance. Average training difficulty was analyzed using Student’s two-sample t-test to compare algorithm and self-led training conditions. These average difficulties were also assessed for significance against the static difficulty group through one-sample, two-tailed t-test against the null hypothesis that µ = 12 (the baseline difficulty level).

2.5.2 Evaluation phase analysis

Integrated skill was used as the primary metric to evaluate performance in the Evaluation phase to answer Hypothesis 2. It is defined as the skill across the entire task (composed of all three subtasks). Raw performance (which is determined as described previously for each subtask separately) is transformed to be a continuous value between zero and one, such that zeros correspond to crashes, the threshold for “Poor” performance is anchored to 0.25, and the threshold for “Excellent” performance is anchored to 0.75. These anchor points were selected to be uniformly offset from the max and min of the scale, and based on our desire to balance algorithm responsiveness and skill mastery (i.e., such that the algorithm changed difficulty levels commensurately with the level of skill required in the adjacent level. This was tuned during pilot testing before the algorithm was finalized. We computed a linear transformation of the raw performance metrics, and then applied the error function. These values are then summed to get the integrated task skill metric (ranging from 0–3). A five-trial average of integrated skill was calculated to assess differences in achieved skill during the evaluation phase. For analysis and data visualization, the integrated skill metric as calculated was represented as a percentage of the maximum possible integrated skill metric at the difficulty level used during the evaluation trials. Since there would only be one opportunity to successfully perform a Mars landing in an operational environment, we further evaluated performance on the first trial as a primary assessment of Hypothesis 2. Finally, the evaluation crash rate, or the number of trials out of 5 where at least one subtask was scored as a crash, was compared between training groups.

2.5.3 Motivation and flow analysis

To answer Hypothesis 3, the RIMMS survey is assessed by four sub-metrics in attention, relevance, confidence, and satisfaction, as well as overall motivation, which averages responses to all survey questions. The Flow survey was analyzed as another primary assessment of Hypothesis 3 for differences in time and condition during the training trials. One subject in the static training condition was removed from Flow analysis after being identified as a moderate outlier, as all three of their responses fell beyond 1.5 times the IQR below the first quartile. The subsequent data set met parametric assumptions and was analyzed using a two-way repeated measures ANOVA to test the null hypothesis that flow during training was equal between all three training conditions over each of the three response timepoints. Additionally, motivational differences between the training groups were analyzed to put the RIMMS and Flow results into context. Motivational tendencies were compared between groups by examining responses to the two additional questions described in Section 2.4.

3 Results

Demographic data seen in Table 1 shows no significant difference in demographic distributions between groups for any parameter including sex, age, previous flight experience, or previous VR experience.

3.1 Training phase results

Self-led training subjects experienced significantly higher average difficulty during training than those in the algorithm group (Mean difference = −4.99, 95% CI [-7.60, −2.38], t (25) = −3.93, p = 0.0006), as seen in Figure 2A. In this figure, dashed lines denote difficulty level 12 (the difficulty level experienced throughout training for static subjects) and level 18 (the evaluation difficulty level) for comparison.

The algorithm group did not show a significant difference in average training difficulty compared to the static condition (M = 11.31, 95% CI [9.54, 13.07], t (14) = −0.84, p = 0.4). However, the self-led training group demonstrated a significantly higher average training difficulty level (M = 16.30, 95% CI [14.14, 18.45], t (11) = 4.39, p = 0.001) with an average difficulty level of 16.3. Subjects in the algorithm group experienced significantly fewer total crashes during training than subjects in the self-led group (Kruskal–Wallis χ² (2) = 6.35, p = 0.042, η H 2 = 0.103), as seen in Figure 2B, a relationship that was driven primarily by the manual control piloting subtask (Kruskal–Wallis χ² (2) = 9.50, p = 0.0086, η H 2 = 0.179). Subjects in the static difficulty condition showed high variance in crash rates (M = 7.25, SD = 5.93), with no significant comparisons to either algorithm or self-led conditions.

3.2 Evaluation phase results

The 5-trial average performance during the evaluation phase was significantly different across training groups (ANOVA F (2,42) = 5.86, p = 0.006, η p 2 = 0.218), seen in Figure 3A. Tukey post-hoc analyses demonstrated that algorithm subjects performed at significantly higher average levels of integrated skill than static trained subjects (p = 0.004). In comparison to the self-led training condition, the higher average result approached, but did not reach, significance (p = 0.097). When considering evaluation performance on just the first trial, a similar relationship was seen where the highest performance was observed in algorithm trained subjects. The results in just this first evaluation trial approached, but did not reach, significance (ANOVA F (2,42) = 2.437, p = 0.1, η p 2 = 0.104).

A significant difference in crash rates was seen between training conditions (Kruskal Wallis χ² (2) = 6.8, p = 0.033, η H 2 = 0.114) as seen in Figure 3B. Static difficulty subjects were seen to experience significantly higher rates of crashes than both self-led (Dunn’s z-test statistic = 3.18, adjusted p = 0.003) and algorithm subjects (Dunn’s z-test statistic = 3.25, adjusted p = 0.003).

3.3 Motivation and flow results

Analysis of the RIMMS survey found that the highest average motivation during learning tended to be experienced by subjects in the algorithm training condition across all four sub-metrics as well as the overall motivation score. However, only the sub metric of confidence reached statistical significance (Kruskal-Wallis rank sum test, p = 0.048, H = 0.973). Modified RIMMS survey results are summarized in Table 2.

TABLE 2. Modified RIMMS.

Mean (SD).

Kruskal-Wallis rank sum test. Bold in the table indicates statistical significance p < 0.05.

No significant differences were seen between training groups in response to the question “It is more important for me to feel challenged than it is for me to get a good score” (Kruskal–Wallis, p = 0.8, H = −0.04). Similarly, no difference was seen in response to the inversely worded question “It is more important for me to score highly than it is for me to experience the most challenging version of the task” (Kruskal–Wallis, p = 0.7, H = −0.04).

Figure 4 shows self-reported flow over the training trials. Algorithm-trained subjects and static training subjects showed positive trends over time, while self-led subjects showed decreased trends in flow over time. There was no significant interaction between time and condition (ANOVA F (3.4,69.4) = 1.623, p = 0.187, η p 2 = 0.07) nor a significant effect of time (ANOVA F (1.7,69.4) = 1.563, p = 0.219, η p 2 = 0.04). However, differences in condition approached significance (ANOVA F (2,41) = 3, p = 0.06, η p 2 = 0.128). Pairwise comparisons revealed the self-led group was lower than the algorithm group at the last timepoint (adjusted p = 0.02).

4 Discussion

This study investigates how adaptive facilitation may enhance both training effectiveness and learner motivation for complex tasks. Participants who trained with an algorithm-facilitated approach not only demonstrated the highest skill mastery during evaluation, but also reported greater confidence in their abilities and increased flow.

4.1 Training effectiveness and learning outcomes

Higher difficulty in training did not necessarily translate to better skill acquisition, contrary to our first hypothesis. Participants in the self-led training condition voluntarily trained at more challenging levels but did not outperform the algorithm-trained group during evaluation. This suggests that progressive, tailored facilitation of difficulty may be more effective than self-regulated facilitation of learning complex skills, even when self-regulated learners are motivated to seek challenging experiences.

In training, subjects in the algorithm condition spent their 30 trials of training at the lowest average difficulty level of any group, significantly lower than those in the self-led training condition. In contrast, subjects in the self-led training condition chose to train at much higher difficulty levels, on average much closer to the level 18 evaluation difficulty level. The higher difficulty levels selected by self-led subjects likely contributed to the observation of a significantly higher incident rate of crashes than their algorithm counterparts, demonstrating unstable task performance and reduced ability to follow the fundamental procedures associated with the task, particularly in the manual control (MC) subtask. Further, the self-led individuals reported decreased flow late in training compared to the automated facilitated group.

Despite the higher challenge in training, subjects in the self-led condition did not demonstrate greater mastery of the task during evaluation of performance compared to the other groups. As a group, the algorithm training condition experienced a collective average difficulty of 11.3, a slightly lower average than the difficulty level 12 experience of the static condition. The highest individual subject average in the algorithm group was level 16.1 across their 30 trials of training, even slightly lower than the group average difficulty of the self-led training group (16.3). However, the algorithm training group showed the highest task mastery during evaluation, with integrated skill metrics significantly higher than the static difficulty group (p = 0.004) and nearly significant in comparison to the self-led training condition (p = 0.097), partially confirming our second hypothesis. Even though their training experiences were consistently at a lower difficulty level, algorithm trained subjects were better prepared to perform a more challenging version of the task in evaluation. This suggests that facilitation may play a role in complex skill mastery, which was not experienced by the other two groups.

As expected, the static difficulty training approach proved to be the least effective method of learning. Subjects in the static training condition struggled to adapt to the sudden increase in difficulty between training and evaluation, demonstrating significantly higher rates of crashing during evaluation (p = 0.033) where 69% of the static group (11/16 subjects) experienced at least one crash trial out of the five evaluation attempts. One static subject experienced 4 out of 5 trials with a crash rating, a behavior not seen in any other training condition. On average, subjects in the algorithm condition had a more substantial increase in difficulty level between training and evaluation, but this did not result in higher rates of crashes during evaluation. This demonstrates the benefit of algorithm-facilitated training in mastering the core procedures needed to execute a trial without crashing when compared to the static group.

In our study, the adaptive algorithm partially served the role of a human-tutor by using difficulty level as the mechanism by which to focus a subject’s learning onto the subtask where skill development was most needed. In Azevedo et al., they note the complex responsibilities of the human tutor in observing and guiding learning (Azevedo et al., 2008). In many modulated learning environments, such as systems modeled from Bloom’s mastery theory of learning (Bloom, 1971), progression is predicated on completion of discrete tasks rather than incremental adjustments based on performance. A notable exception is a study by Reyes et al. that leveraged machine learning classification of operator workload states during pilot training to adapt the difficulty of the environment based on physiological data. They incorporated 9 difficulty states, but while they showed promising trends in performance-based outcomes, these did not reach statistical significance, nor did they compare against self-led training (Aguilar Reyes et al., 2023). A novel contribution is made in this research by providing and evaluating an automated tool to allow for effective facilitation without a human operator for complex operationally relevant tasks. More work remains, though, to enable the kind of specific, tailored feedback a human facilitator may provide, such as explaining errors, teaching strategies, and diagnosing misunderstandings.

Algorithm subjects tended to show the highest average report of motivation on the RIMMS across all four metrics (relevance, attention, confidence, and satisfaction) as well as overall motivation, though only reaching significance for the metric of confidence (p = 0.048), partially confirming our third hypothesis. The confidence metric includes responses to the following questions:

• “As I trained, I was confident that I could achieve the training objective of an all-excellent trial.”
• “After training, I feel confident that I can achieve all-excellent ratings during evaluation.”
• “The level of challenge during training helped me be confident in my ability to succeed during evaluation.”

Significantly higher reports of agreement to these statements among subjects in the algorithm training group suggest the value of algorithm facilitation in improving confidence to perform a complex test. Notably, the self-led training group tended to have the lowest average reports of confidence. It is important to note that no significant differences were seen in motivational tendencies among groups (p = 0.8 and p = 0.7), thus there was no reason to believe that subjects in the self-led group were more likely to choose high levels of difficulty based upon motivation, instead of focusing on task mastery, than subjects in the algorithm or static conditions. Yet, when left to facilitate their own training, subjects in the self-led group tended to choose higher difficulty levels and did not master the task to the level of the algorithm training group. These findings are consistent with a 2016 study that examined bystander willingness to perform CPR following conventional facilitated instruction and video self-instruction (Uhm and Kim, 2016). The results of the study found that students who experienced facilitated training had significantly higher scores for willingness to perform CPR, reflecting the value of facilitation in supporting the confidence needed to perform a skill. Our results observed in RIMMS confidence metric ratings are consistent with this, where complex operational skills like EDL also benefit from facilitated learning in supporting confidence to successfully perform the task. This is important as we consider the confidence required to transition to aerospace settings when subjects perform these tasks outside of the simulator for the first time, or in the case of a Mars landing, the first time ever in human history.

These findings are further reinforced by this study’s flow results. Participants in the self-led group reported decreased flow over time, reaching statistical significance at the last training time point. Participants in the facilitated algorithm and static difficulty groups showed positive trends in flow over time, partially confirming our third hypothesis. In the literature, dynamic difficulty adjustment has been shown to increase a user’s confidence in their ability to be successful, which increases the likelihood that they explore new strategies of performance and encourages learning when interacting with a virtual environment (Constant and Levieux, 2019). We note that our findings here are likely due to differences in facilitation, rather than differences in how difficulty progression was implemented between the self-led and algorithmic groups. Subjects in the self-led group adjusted difficulty for all subtasks simultaneously, with difficulty set to the same level. The algorithmic group’s difficulty for each subtask was specific to their performance on that dimension, allowing them to progress independently. In our prior work, though, when algorithmic facilitation progressed with the subtasks in lockstep with one another, flow was increased (Verniani et al., 2024). Thus, these two studies taken together give confidence to the flow results being driven by facilitation, rather than difficulty progression. The interplay between flow and motivation is important to address in future work to ensure algorithmic facilitated training is effectively designed for high-consequence complex operational task training.

4.2 Implications for training design

These results inform the broader potential of adaptive training systems in aviation, space operations, and other complex, high-consequence fields. The findings of improved performance, confidence, and flow for the algorithm-driven approach suggests that automated, and potentially AI-powered, facilitation can serve a role similar to human instruction by dynamically adjusting training levels based on real-time performance indicators. This aligns with previous research on adaptive training in fields such as aviation, CPR instruction, ITS, and classroom learning where facilitated learning often outperforms self-regulated approaches. For these kinds of tasks, expanding the use of VR as a modality for education and training may be important as it enables immersion and simulation of the real-world context (Psotka, 2025).

Importantly, the findings highlight a potential gap in traditional training models: While self-led training may seem appealing, it does not guarantee better skill retention or mastery. The study suggests that without guided difficulty adjustments, learners may push themselves too hard, reducing efficiency and leading to unstable performance patterns. Future training systems should prioritize structured adaptability ensuring that difficulty aligns with an individual’s actual learning trajectory rather than their perceived challenge threshold.

4.3 Limitations and future work

This research strove to overcome limitations in prior work to ensure highly-motivated and consistent background knowledge in the subject population, as would be consistent with implementation in operational contexts. However, this also yielded a limitation in the homogeneity of our subjects. Additional work should strive to investigate the implications of these findings in alternative simulations and application paradigms. We postulate these results may generalize to a similar cohorts of individuals who in jobs that are highly screened (i.e., competitive placement) and exhibit strong intrinsic and extrinsic motivation, but additional analysis should be performed on alternative cohorts before these findings can generalize more broadly. Further, this study does not investigate long-term retention of training, skill durability, or transfer of training to novel tasks which may also limit its generalizability and promise for use in practical application. Ongoing work in our lab is investigating these important aspects needed to better understand the utility of algorithmic training facilitation.

Future work exists in exploring the role of feedback in automated facilitation. The design used in this study only utilized qualitative feedback (Poor, Adequate, or Excellent on each subtask) to encourage trainees to strive for general improvement instead of fixating on the minimum exact score or behaviors to reach Excellent. However, advanced feedback could be explored as a method of increasing facilitation through automation, narrowing the gap with how a human trainer may facilitate training. This may require the use of more complex learning algorithms that can recognize and respond to areas of weakness that are otherwise indeterminate from the coarse data collected in this task. For example, it was observed that some subjects were overly cautious during TD subtask performance, repeatedly running out of fuel before touchdown and resulting in a crash. It often took multiple crashes for the subjects to try a new approach. This behavior could be recognized in the data through observing multiple consecutive trials where subjects have low average descent speeds during the TD phase and run out of fuel when at high altitudes above the surface. This is an area explored by others (Yahyaeian et al., 2026), and is an important area of future work. An algorithm that identifies this behavior may suggest that a subject try letting the spacecraft fall to the surface at a faster rate before reducing the speed for a safe landing.

5 Conclusion

The results of this study contribute evidence of the critical importance of facilitation in learning of complex skills, even when training is performed by a highly motivated population. With an increasing need for additional aviation and spaceflight personnel, there is a desire to reduce the burden on training facilitated by people. During a long-duration mission like a transit to Mars, facilitated training is a challenging task as communication delays prohibit direct facilitation from mission control. It may be postulated that highly-motivated individuals may achieve effective training without facilitation. The results of this study, though, demonstrate that facilitation is a critical requirement for the mastery of complex tasks. Adaptive algorithms are a promising capability of future training systems to master complex skills and build confidence.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2026.1892103/full#supplementary-material

Supplementary material Supplementary material at the source

Footnotes

Edited by: Gabriela Kiryakova, Trakia University, Bulgaria

Reviewed by: Justin Maximilian Mittelstädt, German Aerospace Center (DLR), Germany

Jorge A. Gonzalez-Mendivil, Monterrey Institute of Technology and Higher Education (ITESM), Mexico

Amir Yahyaeian, Embry-Riddle Aeronautical University Worldwide and Online, United States