Resident stress levels during simulated patient encounters

The emergency department (ED) is a stressful clinical environment. Stress activates the sympathetic nervous system (SNS), which leads to physiologic responses such as an increase in heart rate (HR) and heart rate variability (HRV). Studies have shown a relationship between HRV and cognitive performance. As a training tool, simulation attempts to mimic real-world conditions, including the reproduction of physiologic stress reactions in learners. In this pilot study, we sought to assess the physiologic indicators of resident stress induced by simulation, whether measurement of these parameters was feasible during simulation, and whether this stimulation during simulation would lead to cognitive performance improvements. A wearable device was used to measure HR, HRV, and electrodermal activity (EDA) at two American College of Graduate Medical Education (ACGME) accredited emergency medicine (EM) residency programs during a simulation scenario. All residents at the participating sites were eligible to participate. A standardized simulation protocol was utilized. Before and during the scenario, participants completed a cognitive test, and the time of completion was noted. We calculated and reported descriptive statistics on HR, HRV, and EDA. For the IBI data, HRV was also analyzed using three methods: standard deviation, root mean square of successive differences, and PNN50. Twenty-six residents participated, including 7 postgraduate year 1 (PGY1), 8 PGY2, and 11 PGY3. Further, 11 (42.3%) participants were females, 15 (57.7%) were male, and the mean age was 30.7 years. The mean HR range was 59.11–117.46. The average percentages of time the HR was above 120, 130, and >160 were 2.475, 0.88, and 0.041%, respectively. HRV showed that the mean standard deviation of the interbeat interval (IBI) was 87 ms. EDA showed a trend toward increasing throughout the scenario. In this pilot study, simulation-induced physiologic indications of stress were observed as evidenced by increased HR, HRV, and EDA. Measurement of these parameters was feasible and can help to tailor simulation for maximum educational effectiveness. Cognitive testing performance showed improvement in time to completion.

Stress is pervasive and negatively impacts emergency physician performance (1). Simulation allows residents to practice skills and mitigate the effects of stress on performance in a safe fashion (2). It is important to assess whether residents are exposed to the same quantity and quality of stress during simulation as on shift. In this study, we observed how adequate simulation is at inducing and replicating stress experienced while on shift in the emergency department (ED).

With stress, the sympathetic nervous system (SNS) activates the “fight-or-flight” response, manifesting as tachycardia, pupil dilation, and gastrointestinal inhibition (3). Stress's cognitive effects include decreased processing and memory (4). Tachycardia is a surrogate marker for stress and gives a moment-to-moment indication of the severity and duration of stress (5). These physiologic changes can negatively impact performance. Medical students and young physicians need to develop strategies to perform under stress.

In prior studies observing tachycardia during a resident's shift, the average heart rate (HR) for emergency medicine (EM) residents on shift was 87.7 beats per minute (bpm), but there was an average of 11.3 sustained peaks of at least 3 s where the HR was 130–159.9 bpm, with a total of approximately 10 min spent per shift in this HR range. Only 6.5% of shifts examined did not have peaks above 130 bpm (6). This indicates there are several moments during shifts in which a severe range of stress is induced. We attempted to induce stress by creating a challenging clinical scenario, with interruptions for a cognitive test. We measured the physiologic signs of stress with a wearable device and attempted to determine the educational effectiveness of the simulation by assessing performance on a cognitive test before and during the simulation.

In this pilot study, we assessed resident stress levels during a simulation using physiologic measurements and evaluated the feasibility of measuring these physiologic derangements during a simulated patient encounter. We hypothesized that simulation would be a valuable teaching tool but that it would not recreate the actual stress seen on a shift, that it would be feasible to measure these physiologic changes during a simulation, and that there would be signs of cognitive increase as evidenced by decreased duration of time for completion of cognitive tests.

Study design and setting

Validated cases, scoring sheets, and a validated cognitive test (Trail Making Test) were identified through a literature review. Standardized scripts with set event time intervals were created (7). The study was performed in the area where all didactic simulations occur at Louisiana State University Health Shreveport and Baylor College of Medicine. Institutional Review Board approval was obtained at each institution.

Selection of participants

It was explained to participants that the project's data were blinded to staff and faculty and that participation status in no way affected evaluations or standing. Potential subjects were invited to ask questions privately. Residents voluntarily agreed or refused to participate.

Interventions

All study participants received orientation and training about the Empatica E4 monitor bracelet and cognitive test (CT). A unique study number was assigned to each participant and used on Empatica's site. The device was placed on participants, and their study ID was registered on Empatica's website. The monitor was turned on to begin recording and was worn throughout the simulation.

At a predetermined point, when stress was likely to be high, another CT was administered. The start and end of this test was timestamped on their monitor. The resident then finished the simulated case. The simulations were done from January 3, 2018 to August 21, 2021.

Outcomes

The primary outcome was a physiologic change in residents associated with stress during simulation cases. Secondary outcomes were the feasibility of measuring physiological stress during the simulation and cognitive test performance change under stress.

Initial data

Data were collected from 26 subjects using Empatica's E4 wristband. The measures taken included HR, temperature, electrodermal activity (EDA), and blood volume pulse. Each of these measures was time-series data that included a Unix timestamp indicating start times and a frequency measure (in hertz). Also measured was the interbeat interval (IBI), which had a column of time measures instead of frequency measures and a column of values that corresponded to each time measure.

Data analysis

The statistical analysis included finding the mean minimum HR, mean maximum HR, HR variability (HRV), and mean minimum and mean maximum EDA. Also obtained were mean minimum and mean maximum temperatures and general statistical descriptions of each of these measures.

Additional analysis was done on IBI using various methods to determine IBI values. A formula (IBI = 60/HR) was implemented to determine the IBI based on HR. These give not actual IBI values but averaged values at a sample rate of 1/s. Based on the IBI data, HRV was also analyzed using three methods: standard deviation, root mean square of successive differences (RMSSD), and PNN50. Analysis was performed on HR, EDA, temperature, and IBI data within each segment of time between two time marks. Two groups were studied for the cognitive testing portion: those for which the timeframe of the second cognitive event was marked only and those for which the timeframe for both the first and second cognitive events were marked. These two are considered separately, with the second group being a subset of the first.

Statistical significance testing was conducted to determine relationships among the independent variables age, gender, and postgraduate year (PGY) level, and the dependent variables HR, EDA, and temperature. Demographic information was available for 21 test subjects, and the following dependent variable statistics were used when making comparisons: mean, median, min, max, and standard deviation. For example, we considered the relationship between age and mean HR for each subject or the relationship between gender and minimum temperature. The first and last 15 s of each record for HR, EDA, and temperature were truncated to correct for misleading values associated with the startup and shutdown of the device. When looking at the gender set of relationships, the statistical test used was the Wilcoxon rank-sum test. For the age and PGY sets, Spearman correlation was used.

The average age was 30.7 (28–41), with 54.5% male. PGYs were as follows: PGY1: five residents (22.7%), PGY2: six residents (27.3%), PGY3: 10 residents (45.5%), and >PGY5: one resident (4.5%). Three residents (13.6%) had prior residency training, and two residents (9.1%) worked in Emergency Medical Services prior to residency. No residents were former law enforcement, had served in the military, or worked the night before the simulation.

HR, IBI, HRV, and stress index (SI) were assessed first. The mean minimum HR was 59.111 beats per minute (bpm; SD = 7.451 bpm). The mean maximum HR was 117.468 bpm (SD = 22.671 bpm). The average mean HR across subjects was 88.489 bpm (SD = 8.558 bpm; Table 1). The mean minimum IBI was 0.533 s, and the mean maximum IBI was 0.922 s (Figure 1). The average of the mean IBI values was 0.695 s. HRV had a minimum SD of the IBI data of 0.036 s (36 ms), while the maximum SD was 0.153 s (153 ms). The mean SD of the IBI data was 0.087 s (87 ms), with a median of 0.087 s (87 ms). The SD across all SD measures was 0.029 s (29 ms). SI was calculated for each simulation, both overall and while doing the cognitive test. Normal stress index numbers (50–150) were used for comparison. Eight simulations had an overall SI below the normal range and two simulations above the normal range. The remaining 16 had a SI within the normal range (Table 2).

EDA and temperature were also assessed. The mean minimum EDA was 0.078 μS (SD = 0.193 μS). The mean maximum EDA was 2.664 μS (SD = 3.221 μS). The average mean EDA was 1.205 μS (SD = 1.713 μS; Table 3). The mean minimum temperature was 29.58 °C (SD = 3.71 °C). The mean maximum temperature was 33.81 °C (SD = 1.90 °C). The average mean temperature across subjects was 31.91 °C (SD = 2.60 °C).

Relationships between the independent variables age, gender, and PGY level and the dependent variables HR, EDA, and temperature were assessed. Across all tests, the following are the only results with a p -value of < 0.05: Age – Mean EDA: p -value = 0.015, coefficient = −0.52, Age – Median EDA: p -value = 0.004, coeff = −0.59, Age – Min EDA: p -value = 0.026, coefficient = −0.48.

The other two statistical measures between age and EDA (max and SD) had a p -value close to 0.05 (0.055 and 0.056, respectively), and similar coefficient values (−0.42 for both). Finally, PGY and mean heart rate had a p -value of 0.066 and a coefficient of 0.41. All other p -values were > 0.08.

In this study, we set out to assess the feasibility of measuring physiologic signs of stress during a simulation and whether the simulation stimulated physiologic stress to the same extent as actually being on shift and to pilot a novel technique for measuring stress during simulations. Multiple individuals underwent HR monitoring as a surrogate measure for stress induced by the simulation scenario. While a prior study focused on events occurring on shift, this study delved into training methods, specifically simulations (1). In this study, we saw that residents experienced fewer physiological signs of stress during a simulation compared to being on shift. It was shown that residents spent about 29 s with a HR above 120 bpm, which correlated to about 2.5% of total study time, with a fraction above 130 and 160 bpm. The fraction of the time spent with a HR above 120 during a clinical shift was found to be 4.94%, almost double the stress time percentage in this study (1). Although this shows that residents experience stress during a simulation, it appears to occur in a more controlled fashion compared to clinical cases of shorter duration and lower severity. Stress-generated hormones act in concert to increase memory in some circumstances and lead to an adaptive stress response (2, 3). How much hormonal activation is optimal for enhancing memory is unknown. Is enough stress induced by a simulation to induce this adaptive stress response? Given that increased stress in response to new experiences can lead to decreased immediate performance, residents need to train themselves to face these scenarios using a simulation that gives a similar stress response (4).

What is the best way to prepare residents for the stress of clinical practice? The thought behind performing a simulation is that it represents a realistic enough experience to allow residents to practice and modulate stress without adverse patient outcomes (5). It may be the threat of bad patient outcomes that creates the stress response in real patient encounters, making it difficult to produce this kind of stress in a simulation. Residents appear to be less stressed during standard simulation scenarios when comparing physiological responses seen in a simulation vs. those during a shift (1). With repeated stress, the level of response to that stress diminishes. The most common mechanism is proposed to be habituation, but other mechanisms exist for this decrease in response (6, 8). The appropriate levels and amount of repetition of stress to accomplish habituation are unknown. Repeated stress causes diminished memory formation and difficulties in memory retrieval, with likely effects on learning and performance, and the appropriate repetition pattern is undefined (9, 10). These effects may be related to intrinsic factors in each person, such as gender roles and extraversion vs. introversion, and it is unknown how simulation can be made most effective for all learners (11, 12). Simulation shows decreased physiologic signs of stress response and, potentially, learning, with repetition of simulation scenarios leading to the question of how long a simulation may remain effective for learning (13). Some simulation studies using modified simulation techniques found increased immediate performance that declined to the same level as traditional simulation teaching methods. Mixing different simulation methods may be beneficial for making simulations more effective for learning over longer periods of time (14). Despite all the unknown factors in learning under stressful conditions, simulations should not be discounted as an important adjunct to learning safe practice (15).

Numerous physiologic indicators were assessed in this study. HR was the physiologic factor used as a surrogate for stress reactions in the residents. It is known that acute stressors activate the SNS, which leads to an elevated resting HR in preparation for the “flight-or-fight” response. Some studies, however, have shown that elevated HR does not correlate well with perception of stress or performance (16). Using physiologic measures of stress, such as HR, can be unreliable as other states can cause changes in these parameters and are difficult to measure (17). We did find increases and peaks in HR that corresponded to times of increased stress in the simulation scenario; however, the increases were not consistent from person to person and not as sustained or reaching the same levels as seen in studies looking at these variables in clinical settings (1).

This unreliability of HR as a marker of stress has led to the study of HRV as a more reliable indication of stress and ability to perform (18). The degree of variability in the HR provides information about the function of nervous control on the HR and how the heart responds to this. This is analyzed by looking at the IBI, which is the time between heartbeats and is not the same from beat to beat, as it is influenced by the person's sympathetic and parasympathetic nervous systems (19). As the HR increases with sympathetic stimulation, the IBI decreases. Variability from beat to beat is normal and expresses the heart's ability to respond to multiple physiological and environmental events (e.g., breathing, physical exercise, mental stress, hemodynamic and metabolic changes, and sleep and orthostatism) and compensate for disease-induced disorders (20, 21). HRV can be used to measure the sympathetic and parasympathetic function of the autonomic nervous system (ANS) and the degree of integration of the areas of the prefrontal cortex, brainstem, and peripheral nerves and how much control they have over the ANS (18, 20). The HRV seen in our study was found to be a median of 87 ms with a minimum SD of 36 ms and a maximum SD of 153 ms. The average age of the residents in this study was 31 years old. When HRV was considered across a sample size of almost 25,000 subjects, the average HRV was 78 ms for 25-year-old and 60 ms for 35-year-old subjects in the morning readiness phase of the day (22). This is interesting as it shows increased HRV in our participants for age-matched baseline, which is generally associated with decreased stress, higher levels of baseline cardiovascular fitness, or greater resilience to stress. This is difficult to interpret, however, as HRV is a complicated relationship between autonomic branches and must be interpreted in the context of each participant's situation (23, 24). In this light, interpreting the elevated HRV in this study as it pertains to stress becomes more difficult. It would seem to suggest that simulation does not caused increased stress, as stress tends to lower HRV. Another possible cause of this inconsistency from recorded HRV data in our participants is higher than normal fitness levels at a baseline, which was not recorded. A recent systematic review looked at HRV and its ability to measure stress specifically in medical professionals, and it found HRV to be a reliable measure of stress and even recommended its use for monitoring appropriate stress during simulation (25).

The SI is a quantitative measure of physiologic stress derived from HRV data (26). In our study, there were only two studies above the normal SI and eight below the SI, but the majority were in the normal range. This likely means most of the participants' autonomic nervous systems were functioning well under stress, keeping the SI in the normal range.

EDA is the change in the conductance of the skin as the SNS is stimulated and, in turn, stimulates sweat glands, leading to increased sweat and skin conductance, which can be measured to assess sympathetic activation and, in turn, stress (27). We see a general increase in skin conductance over the course of the simulations, but none of the values were above the normal 1–20 μS. We know this represents SNS activation, but it is unclear what this elevation in EDA means in terms of student learning (28). Temperature was assessed, but the maximum recorded temperature and the mean temperature were below normal body temperature, so there was likely ambient temperature interference, so the data were not reliable or used for drawing any conclusions.

The analysis of the CT during the simulation showed similar patterns of increase in indicators of stress and a shortened time to completion from CT1 to CT2. The mean HR showed a general increase across the simulation scenario, with decreases seen during CT2. The increase in EDA was also seen in relation to the CT, with peaks seen during CT2 and decreasing thereafter. This is interesting as these two indicators of stress seem to contradict each other. This shows signs of increased stress over the base stress induced by simulation. This could be an effective method to increase pressure during a simulation and potentially induce learning. The decreased time to completion of the CT showed that stimulation did in fact occur, but it is unclear whether the repetition of the same test relied on stimulation of the short-term memory of the placement of letters and numbers, doing the test, or actual cognitive increase for the testing.

The strength of this study relies on the physiologic responses from the body, which are objective and not dependent on participants' perception of stress. It also provides a moment-to-moment glimpse of physiologic derangements that occur with stress (18, 29). It must be taken into account, however, that numerous other factors come into play in learning and stress that cannot yet be measured directly, and the exact interpretation of the effects these physiologic signs of stress have on learning is uncertain (30, 31). These physiologic responses can also be accurately compared to data that were acquired on shift by the same device model. While some studies have considered performance metrics related to simulation, this study focused mainly on the physiologic response to simulation (32). Using the methods outlined in this study, these ideas can be expanded into larger sample sizes and extended into different means of monitoring.

Limitations

There was a lack of variability in the simulated clinical scenarios. The cases chosen were intentionally felt to be higher stress compared to other scenarios. As scenarios and their induction of stress vary among people, this was difficult to control for. It may be beneficial to investigate which cases are most distressing to residents and use those cases to prepare residents for success with future training. Another limitation was that the data collected from the device were incomplete for some measures, with suspected interference from ambient temperature and subject movement. Incomplete data for IBI were statistically derived and then used in the calculation of HRV, possibly leading to less reliable HRV data. Only physiologic signs of stress were assessed, which give a limited picture of the stress response in a participant. A central methodological limitation is that there was also no direct comparison of the same study participants between simulated stress and clinical stress, so previously completed work with different participants was relied on as an indirect comparison. This study also evaluated only resident participants. It is unknown whether these same changes would be seen in a more experienced learner group. The participants were screened for prior work experience and other factors that might have affected the physiologic response to the simulation, but they were not screened for behaviors like smoking, energy drink intake, or medications such as beta blockers that could affect the heart rate.

Simulation is a valuable tool within residency education, giving residents exposure to rare situations and performing uncommon procedures (33). Thoughtful adjustments are needed to prepare residents through simulation for the stress that is experienced during patient care. Further study needs to address repetition, timing, and individualization for maximum benefit using combined multimodal stress assessment techniques. This pilot study demonstrates a novel and effective method for measuring the physiologic indicators of resident stress induced by simulated patient scenarios. Measurement of these parameters during simulation was feasible.