Striving toward ecological validity in hearing science: evaluating a variety of hearing assessments

1 Introduction

Approximately 37.5 million Americans report some degree of hearing difficulty (Blackwell et al., 2014). However, traditional hearing assessments like the audiogram often fail to fully capture the real-world challenges these individuals face. More than 10% of people with normal audiogram results still struggle to understand speech in noisy settings (Tremblay et al., 2015). This disconnect between measures like the audiogram collected in laboratory environments and real-world experiences necessitates more sensitive, accessible assessment methods. A key step toward this goal is ensuring that hearing assessments possess strong ecological validity.

Ecological validity in hearing science is defined as the degree to which research findings reflect real-life hearing-related function, activity, or participation (World Health Organization, 2021), and the call for improved ecological validity in hearing science is not new. For years, researchers have recognized the limitations of traditional, lab-based assessments that rely on simplified stimuli and sterile environments, which often fail to capture the complexity of real-world listening challenges, and four key aims to advance this are: (1) better understanding the role of hearing in everyday life, (2) supporting the development of improved procedures and interventions, (3) facilitating improved methods for assessing and predicting ability to accomplish real-world tasks, (4) and enabling more integrated and individualized care (Keidser et al., 2020).

To move toward these goals, it is essential to examine how current assessment tools align with the realities of everyday hearing. Audiological assessments include, at minimum, an otoscopic examination of the external auditory canal and an audiogram assessment for frequencies between 250 and 8,000 Hz (Baiduc et al., 2013). To show higher sensitivity to early signs of hearing loss and represent more realistic communicative signals, assessments of speech reception thresholds and speech discrimination tasks in quiet and/or speech-in-noise intelligibility tasks are often included as well (Bhatt et al., 2022). Nonetheless, ecological validity is not met as these tasks are assessed in quiet or stationary background noise, in a highly controlled environment, and without visual information of the talker (Tharpe, 2004). Efforts to improve ecological validity have focused on using more naturalistic stimuli, incorporating background noise that more closely mimics everyday environments (Hey et al., 2022). Gathering additional self-report questionnaires about people's experience of functional hearing in everyday situations [e.g., Hearing Handicap Inventory (Cassarly et al., 2020), Speech, Spatial, and Qualities of Hearing Scale (SSQ—Gatehouse and Noble, 2004)] affords a more realistic view of the individual's communicative functioning and the relationship to their environment, and helps inform on individual needs for intervention or further assessment (Cox, 2003). Unfortunately, the individual variability in these reports is often linked to biases, such as recall bias (Verbrugge, 1980) and has been found to be associated more closely with certain personality traits than actual hearing performance (Kawaguchi et al., 2019). The use of ecological momentary assessment to capture subjective listening experiences in real-time has been identified as an important tool to overcome these limitations (Holube et al., 2020).

Others have used virtual and augmented reality to create immersive and controllable, yet realistic, listening environments (Mehra et al., 2020).

To advance ecological validity in hearing assessment, this work systematically examined four key factors that may influence real-world hearing experiences: (1)availability of visual information (auditory-only vs. audiovisual cues), (2) assessment focus (audibility, intelligibility, comprehension, and vocal effort), (3) interactive nature of communication (individual vs. dyadic/familiar partner scenarios), and (4) naturalness of the environment (controlled lab vs. simulated real-world settings). By structuring our study and subsequent analyses around these factors, we aimed to identify which aspects of assessment most closely align with individuals' real-world hearing challenges and self-reported experiences. We hypothesized that measures incorporating audiovisual cues, comprehension tasks, interactive communication, and more naturalistic environments would show greater sensitivity to hearing difficulties than traditional audiometric tests.

2 Methods

2.1 Study design

This within-subjects correlational study included adults who met the following criteria: (1) native English speakers; (2) normal hearing or mild to moderate symmetric hearing loss, as measured by audiogram; (3) optional history of hearing aid use; (4) availability for a single 2–3 h data collection session on location; and (5) preferably, the ability to bring a familiar communication partner.

Each participant completed a comprehensive battery of assessments. These included an audiogram and otoscopy, retrospective and real-time self-report questionnaires, and a speech-in-babble (SIB) task administered in both auditory-only (A-only) and audiovisual (AV) modalities. In addition, familiar dyads participated in naturalistic conversation tasks (to measure vocal effort) and podcast listening tasks (to assess listening comprehension), both conducted in quiet and in background babble noise. Throughout these tasks, we systematically examined how objective performance measures and subjective ratings were influenced by assessment type, modality, environment, and noise condition.

2.2 Participants

Data were collected from 26 English-speaking adults (13 female, 13 male), aged 50–79 years (mean: 67). Most participants attended with a familiar communication partner, resulting in 11 dyads and four individuals paired with a research assistant. Hearing status ranged from normal to moderate loss (Figure 1): 11 had normal hearing (4-PTA ≤ 20 dBHL), 13 had mild loss (4-PTA between 21 and 40 dBHL), and two had moderate loss (4-PTA between 41 and 50 dBHL). Seven participants regularly used hearing aids, however none wore them during the study (Table 1).

Table 1. Overview of the 26 participants, with their conversational partner (Dyads), the relationship between partners (Relationship), the 4-PTA (dBHL), Age, Gender and whether (1) or not (0) they used hearing aids (HA) at home.

The study involves human participants, and procedures were reviewed and approved by Reality Labs Research (RL-R) Research Review Committee, including Environmental, Hazards, and Safety, Hardware Safety, Legal, Ethics, and Privacy reviewers. Participants provided written informed consent under a protocol that was approved by the Institutional Review Board (https://www.advarra.com/review-services/institutional-review-board-services/) for the collection of data and use of their data in publications. Participants were compensated for their time.

2.3 Stimuli and materials

Participants completed a 2–3-h session in a lab-based and simulation-based setting (Table 2). Both environments were created within one of the Reality Labs Research buildings in Redmond, WA, USA.

Table 2. Overview of the lab-based and simulation-based measurements, broken up by task type, environment, modality, SNR, target stimulus and measurement.

2.3.1 Measurements in lab-based environment

The lab-based measurements were conducted in two rooms. Basic audiological assessments were performed in a standard soundproof booth, while the SIB-task took place in a separate laboratory environment (Figure 2, left).

Pure tone air conduction thresholds were measured for both ears at frequencies ranging from 0.25 to 8 kHz, using a calibrated Grason-Stadler Audiostar Pro audiometer with IP30 insert earphones. For all analyses involving audiometric data, we used the four-frequency pure-tone average of the best ear (4-PTA) calculated as the average of air conduction thresholds at 0.5, 1, 2, and 4 kHz. This measure will be referred to consistently as 4-PTA throughout the manuscript.

In the sound booth, participants also completed two retrospective self-report hearing questionnaires on an iPad: the Hearing Handicap Inventory–Revised (HHI-R; Cassarly et al., 2020) and the Speech, Spatial, and Qualities of Hearing Scale (SSQ-12; Gatehouse and Noble, 2004). These questionnaires are designed to capture how individuals experience speech and environmental sounds in daily life. The HHI-R consists of 18 questions, each scored as yes (4), sometimes (2), or no (0), resulting in a total score between 0 and 72, where higher scores indicate greater perceived handicap. The SSQ-12 comprises 12 items across three subscales (Speech, Spatial, and Qualities) with each item rated from 0 (a lot of difficulty) to 10 (no difficulty).

The SIB task was completed in a second laboratory environment with five calibrated Genelec 8330A loudspeakers. Each of the four noise speakers was set to 57 dB LAEq, with speech presented at −3 dB SNR relative to the combined four-talker babble. The SNR of −3 dB was selected to avoid ceiling and floor effects, based on a pilot study with six individuals with thresholds in the normal to mild range. Stimuli were delivered via Max software (Cycling '74, 2024) on a MacBook Pro and routed through an RME Fireface UFX II interface. Target sentences (Basic English Lexicon (BEL); Calandruccio and Smiljanic, 2012) were presented from the front loudspeaker, accompanied by a monitor (Sunfounder 133″ 1,920 × 1,080) displaying either the talker's video (AV) or a fixation star (A-only). Sentences were recorded in-house by a female talker producing Lombard speech (to represent more natural conversational speech-in-noise environments; Marcoux et al., 2022) and processed to 60 dB SPL (See Supplementary Figure S1 for more details). Four noise loudspeakers, positioned at 45-degree angles, relative to the participant, played calibrated podcast fragments (from Bloomberg Podcasts) featuring both male and female talkers (Figure 2). Each trial used a unique combination of 4/150 different podcast channels, with fragments starting 500 ms before and ending 500 ms after the target speech. The assignment of podcasts to loudspeakers was randomized for each trial to minimize adaptation to talker timbre by location, ensuring that no more than four talkers were heard simultaneously, creating a 4-talker babble. Noise and sentence order, as well as loudspeaker assignment, were randomized for each participant. Each participant completed six randomly selected lists (25 sentences each) via jsPsych (de Leeuw et al., 2023). Each list took about 4 min to complete.

2.3.2 Measurements in simulation-based environment

In a simulated living room (Figure 2, right), familiar dyads completed two tasks: conversation and listening comprehension. Each task lasted 4 min and was conducted in both quiet and noise (four-talker babble) conditions. For the conversational context, the target stimulus was the partner; for the listening task, it was a podcast conversation (selected from elllo.org; see Supplementary Table S1), presented at 60 dBSPL. Two podcasts (out of four) were randomly assigned per dyad and condition. A researcher was present but not involved after providing instructions to enhance participant comfort. Noise was generated using 60 four-minute podcast snippets (from 15 Bloomberg Podcasts; Supplementary Table S1), presented as four-talker babble from loudspeakers at 45°- angles. Each channel featured a different talker, and all noise was calibrated to −25 dB RMS. Five Genelec 8330A loudspeakers were individually calibrated to 57 dB LAEq (combined 60 dB LAEq), with the speech signal set at 0 dB SNR relative to the babble. Stimuli were presented via Max (Cycling ‘74) on a MacBook Pro, routed through an RME Fireface UFX II interface. The 0 dB SNR was chosen based on pilot data indicating that −3 dB (like in the lab environment) was too challenging for participants with mild to moderate hearing loss. Conversations were recorded (Slint Professional Lavalier Microphones + iPad). After each podcast, participants answered four multiple-choice listening comprehension questions (from elllo.org; see Supplementary Table S2) via Qualtrics (iPad).

Throughout all tasks (both lab- and simulation-based), participants provided real-time self-reports on speech understanding, voice pleasantness, and listening effort using a Lab.js tool via a local http server on an Android smartphone (Samsung Galaxy S22). Ratings were collected twice per condition (midpoint and end: six prompts: three questions, two times), using a 1–5 color-coded scale (red = poor/hard, blue = good/easy), with higher scores indicating better understanding, greater pleasantness, and lower effort.

2.4 Procedures

After providing informed consent, participants underwent audiological assessment in a sound booth, unless they had completed this assessment at the lab within the previous 6 months. The research audiologist performed otoscopy and pure tone audiometry using a modified Hughson-Westlake method (Carhart and Jerger, 1959). Participants also completed the HHI-R and SSQ-12 questionnaires.

The order of the tasks was counterbalanced per dyad. Half of the dyads completed the lab-based assessments before moving to the simulation-based environment, the other half completed the simulation-based environment first. Within the simulation-based environment, the order of the tasks (conversation first vs. podcast first) was also counterbalanced across dyads.

When participants moved to the SIB test environment, they were seated facing a loudspeaker and screen. After instructions and practice trials, they completed six blocks of the SIB task (one randomized BEL sentence list per block (out of 20 lists), 25 sentences per list). For each sentence, participants repeated what they heard, and responses were scored for four key words per sentence (maximum score per list: 100). The word had to be completely correct before a point would be assigned, for example when the participant would repeat “book” when the target word was “books,” this would be scored as incorrect. Responses for the SIB task were scored by the experimenter in real-time. All participant responses were recorded. For a randomly selected subset of five participants, one randomly selected SIB list was re-scored by the original researcher after study completion. This revealed full agreement with the initial scoring, and therefore no further re-evaluation was conducted.

Real-time self-report measures of speech understanding, voice pleasantness, and listening effort were collected via smartphone at the midpoint and end of each condition. Participants answered an ordinal scale, from 1 to 5. The use of the phone app was explained to the participant via a paper booklet. The paper booklet represented every possible screen and question. The participant would press “play” on the app at the beginning of a condition. The app would then show a timer for 2 min. After 2 min the first question would show (“Understand speaker well?”). If the participant did not notice the question, the researcher directed the participant to the app. While answering, the speech intelligibility task was paused. The participant was instructed at the beginning of the task to always prioritize repetition of a sentence before starting the real-time self-report measure. At the end of a condition the participant answered all questions again.

When dyads moved to the simulation-based environment, depending on group assignment, they began with either conversation or podcast listening. During conversations, participants were encouraged to talk freely, with prompts available if needed, and were reminded that content was not of interest to us. Real-time self-report measures were collected similarly to the lab-based tasks. For the dyad conversations, vocal effort was measured as the talker's voice intensity (Wu et al., 2018), recorded via a microphone attached to their clothing. The researcher was blinded to the participant and the dyad when scoring this measure.

For podcasts, participants listened and then completed a digital listening comprehension questionnaire (four 4-AFC questions) after each podcast, with real-time self-reports collected at the midpoint and end of each condition. Scores were automatically assigned via the tablet.

2.5 Analysis

Although participants completed six blocks of the SIB task, the conversations, and the podcasts, this manuscript will only discuss two out of six. The results from the four other conditions involving research technology are considered confidential and are subject to Meta's internal review and approval processes. As such, these data are not presented in this manuscript. The two reported conditions were selected a priori as the core, non-proprietary assessments for this study. Therefore, the conclusions drawn in this manuscript are based solely on the reported conditions, and generalizability to the withheld proprietary conditions cannot be assumed.

All tasks were scored by the same researcher. All analyses were conducted in R Studio (2023.06.1). Regression analyses used the nlme package, and ordinal data were analyzed with the ordinal package. Plots were generated with ggplot. For all tasks, summary statistics (min, max, mean) were calculated to describe performance ranges.

We evaluated associations of our hypotheses using two complementary approaches. First, we computed Pearson/Spearman correlations to provide a descriptive, model-agnostic summary of the marginal association. Second, we fit linear mixed-effects models to quantify the association while accounting for repeated measures (random intercepts for participant; random slopes where appropriate) The lme results constitute the primary inferential analysis.

Pearson's correlation was used for numerical data, and Spearman's rank correlation for ordinal data.

All continuous variables were standardized (scaled and centered with scale()), and categorical variables were sum contrast coded to facilitate interpretation of interaction effects.

Model 1: lme (SIB ~ 4-PTA ^* Modality + (1|Participant/Round)) examined the relationship between 4-PTA and SIB task performance across A-only and AV modalities. After confirming normality of residuals, a linear mixed-effects model was used, with participant as a random intercept and assessment order (Round) nested within participant. Both main and interaction effects of hearing threshold and modality were assessed.

Models 2 and 3 analyzed podcast listening comprehension scores (0–4, based on four 4-AFC questions per podcast). Separate models were used due to multicollinearity between 4-PTA and SIB scores. Both main and interaction effects of hearing performance and noise condition were tested, with participant as a random intercept.

Model 2: lme (Listening Comprehension Score ~ Noise Condition ^* 4-PTA + (1|Participant)).

Model 3: lme (Listening Comprehension Score ~ Noise Condition ^* SIB + (1|Participant)).

Due to human error during the recording process, the audio quality for several participants was compromised, resulting in usable data of the dyad conversations for 16 participants (8 dyads). Some data preparation was required to analyze these data. Following Wu et al. (2018), a researcher identified five pairs of adjacent segments (≥2 s each) per 4-min recording: speech-plus-noise vs. noise-only (noise condition), or speech vs. quiet (quiet condition). SNR was calculated as the difference in dBRMS (measured in Audacity Team, 2023) between speech and noise segments, averaged across five pairs. Vocal effort was defined as the difference in SNR between noise and quiet conditions.

For real-time self-report ratings (1–5 scale, ordinal), each question was presented twice per condition, yielding two data points per question per participant. Cumulative Link Mixed Models (CLMM, ordinal package) assessed the effects of question type, modality (A-only/AV), and test location (lab/simulation), with sum contrast coding for categorical variables. The overall model: clmm(Self-report Rating ~ Question Type ^* Modality ^* Test Location + (Repetition of Response | Participant)), included participant and response repetition as random effects. To simplify interpretation, separate models were run for each question, and results are reported as log odds ratios (OR).

3 Results

3.1 Speech intelligibility as a measure of hearing performance

The SIB task demonstrated substantial variability in performance across participants (2%−98% correct; mean = 64%, SD = 25.97%), confirming its suitability for individuals with hearing thresholds from normal to moderate loss (see Figure 1).

Our primary analysis used a linear mixed-effects model (Model 1) to test effects of hearing threshold (4-PTA) and modality (A-only vs. AV) on SIB performance. The model confirmed significant main effects of hearing threshold (t_{1, 24} = −6.33, p < 0.001) and modality (t_{1, 24} = 5.56, p < 0.001), with better performance with lower hearing thresholds and when visual cues were present. No interaction was found, indicating that the benefit of visual cues was consistent across hearing thresholds.

As a descriptive complement, SIB scores were strongly negatively correlated with 4-PTA in both modalities (A-only: r = −0.76, t = −5.78, p < 0.001; AV: r = −0.75, t = −5.62, p < 0.001; Figure 3). Consistent with these effects, mean performance was lower and more variable in the A-only condition (mean = 55.2%, range = 2%−97%, SD = 26.4%) than in the AV condition (mean = 71.0%, range = 23%−98%, SD = 23.4%).

3.2 Listening comprehension as a measure of hearing performance

Our primary analysis used linear mixed-effects models (see Methods) to test the effects of listening condition (quiet vs. noise) and hearing performance (SIB or 4-PTA) on listening comprehension. The models showed significant main effects of noise (4-PTA-model: t1,73 = −9.12, p < 0.001; SIB-model: t1,7₂ = −8.64, p < 0.001) and hearing performance (4-PTA-model: t1,₂3 = −3.15, p < 0.01; SIB-model: t1,7₂ = 2.84, p < 0.01), as well as a significant noise × hearing-performance interaction (4-PTA-model: t1,73 = −4.49, p < 0.001; SIB-model: t1,7₂ = 3.85, p < 0.001), indicating that the impact of background noise on comprehension depended on hearing ability. Consistent with this, background noise substantially reduced comprehension overall (mean 3.04/4 in quiet vs. 1.68/4 in noise; t1,7₂ = −8.64, p < 0.001), and this reduction remained significant even among participants with clinically normal hearing (t_{1, 24} = −4.11, p < 0.001), demonstrating the sensitivity of the measure.

As a descriptive complement, comprehension in quiet did not correlate with hearing measures (4-PTA or SIB; Figure 4c, dashed line), consistent with the suprathreshold nature of the task. In contrast, comprehension in noise was strongly associated with hearing measures (Figure 4c, solid line): 4-PTA (r = −0.61, t = −5.28, p < 0.001), SIB A-only (r = 0.66, t = 4.22, p < 0.001), and SIB AV (r = 0.64, t = 3.97, p < 0.001).

3.3 Vocal effort as a measure of hearing performance

To assess vocal effort, we quantified the change in talker voice intensity from quiet to background-noise conditions. Participants increased vocal intensity in noise by an average of 6 dB (SD = 3.02; t = 8.05, p < 0.001). As a descriptive analysis, we examined whether the magnitude of this intensity increase was associated with individual hearing ability (Figure 4d). The intensity change was not significantly correlated with 4-PTA (r = 0.25, t = 0.95, p > 0.10), SIB A-only (r = −0.23, t = −0.88, p > 0.10), or SIB AV (r = −0.40, t = −1.64, p > 0.10). Together, these results suggest that although participants reliably increased vocal intensity in noise, individual differences in this response were not strongly related to hearing performance.

3.4 Self-report as a measure of hearing performance

3.4.1 Retrospective questionnaires

At the start of the lab session, participants completed the SSQ-12 and HHI-R to capture baseline self-reported hearing difficulties.

SSQ-12 total scores ranged from 2.78 to 8.93 (mean = 6.16, SD = 2.08), with 23/26 participants reporting at least some hearing difficulty. We examined associations between SSQ-12 subscales and lab-based hearing measures (Figure 4a). The Speech and Spatial subscales were not significantly correlated with objective performance. In contrast, the Qualities subscale (covering ease of listening and perceived clarity/naturalness/identifiability of everyday sounds (Noble et al., 2013), showed moderate correlations with 4-PTA (r = −0.40, t = −5.32, p < 0.01) and SIB-AV (r = 0.45, t = 4.28, p < 0.05), but only a weak, non-significant correlation with SIB A-only (r = 0.28, t = 2.46, p > 0.10).

HHI-R scores ranged from 0 to 54 (median = 16; mean = 19.77). Associations with objective measures were weak and non-significant for 4-PTA (r = 0.19, t = 2.33, p > 0.10) and SIB A-only (r = −0.29, t = −2.58, p > 0.10), with a moderate but non-significant correlation with SIB-AV (r = −0.33, t = −2.59, p > 0.10; Figure 4b).

Overall, correspondence between retrospective self-report (SSQ-12/HHI-R) and objective lab-based measures was limited.

3.4.2 Real-time self-report

Real-time ratings of speech understanding, listening effort, and voice pleasantness were collected in both lab-based and simulation-based environments (Figure 5). Effects were evaluated using ordinal models (reported as log (OR)).

3.4.2.1 Speech understanding

In the lab, ratings were typically rated as “3” (average), with higher ratings in the AV modality (Figure 5, top left). Predicted probabilities showed a shift toward high ratings (4–5) in AV and toward low ratings (1–2) in A-only. Modality significantly improved ratings in both environments (lab: log(OR) = 1.7, p < 0.001; simulation: log(OR) = 7.5, p < 0.001). Comparing environments within each modality, there was no lab–simulation difference in A-only; however, within AV, simulation-based ratings were significantly higher than lab-based ratings (log(OR) = 5.6, p < 0.001).

3.4.2.2 Listening effort

In the lab, listening effort was generally rated high (4–5) in both modalities but was significantly lower in AV than A-only (log(OR) = −1.4, p < 0.01). This listening effort reduction for AV was even larger in the simulation-based environment (log(OR) = −5.2, p < 0.001). As with speech understanding, there was no lab–simulation difference within A-only, whereas within the AV modality, listening effort was rated significantly lower in simulation than in the lab (log(OR) = −4.2, p < 0.001).

3.4.2.3 Pleasantness

Pleasantness ratings did not differ significantly by modality in the lab (log(OR) = 0.89, p > 0.10), but in the simulation-based environment AV was rated more pleasant than A-only (log(OR) = 6.1, p < 0.001). Within the A-only modality, the podcast talker was rated less pleasant than the SIB talker (log(OR) = −1.4, p < 0.01), while the opposite was true in AV (log(OR) = 3.8, p < 0.001).

Overall, the conversational context was rated as most pleasant, least effortful, and associated with the highest speech understanding, with the smallest between-participant variability.

3.4.2.4 Associations with objective measures

Correlations (Spearman's ρ; see Table 3) showed that the question “Understand speaker well?” was most strongly associated with objective measures (4-PTA, SIB, listening comprehension). Notably, change in vocal intensity was strongly correlated with listening effort (ρ = −0.62, p < 0.05), but not with speech understanding, suggesting that increased vocal effort reflects perceived effort rather than reduced understanding. Correlations between real-time and retrospective self-report were limited, with only a significant association for listening effort in the lab-based AV SIB task.

Table 3. Table of Spearman rank correlation coefficients of the responses to the real-time self-report questions (speech understanding, listening effort, pleasantness speaker) in quiet and in noise for different test scenarios (lab-based vs. simulation-based and A vs. AV) with other assessed (non-self-report and self-report) tasks.

Overall, the real-time self-report questions correlated best with the SIB performance, and especially rating of speech understanding was most correlated with other tests.

p < 0.05 = ^*, p < 0.01 = ^**, p < 0.001 = ^***.

Significant values are bolded.

In summary, the lab-based AV SIB task was the strongest predictor of self-reported speech understanding and listening effort, followed by the A-only simulation-based task in noise (listening comprehension of the podcasts). Pleasantness ratings were not informative for hearing performance in any condition.

4 Discussion

4.1 Study goals

This study aimed to identify which factors of everyday listening situations influence hearing experiences. We compared traditional measures (4-PTA, A-only SIB, SSQ-12, HHI-R) with measures that more closely align with real-world experiences of the listener (AV SIB, listening comprehension, vocal effort, real-time self-report) in 26 adults (ages 50–79) with varying hearing abilities. These manipulations targeted: the presence of visual information, the focus of assessment (audibility, intelligibility, comprehension, and vocal effort), the interactive nature of communication (individual vs. dyad), and the naturalness of the environment (lab-based vs. simulation-based), and hypothesized that measures incorporating audiovisual cues, comprehension tasks, interactive communication, and more naturalistic environments would show greater sensitivity to hearing difficulties than traditional audiometric tests.

4.2 Increase in ecological validity

We introduced four-talker babble as background noise in all tasks assessing speech intelligibility (A-only and AV SIB), listening comprehension, and vocal effort. This approach was designed to mimic real-world environments, where individuals are often exposed to multiple simultaneous speech sources (e.g., in restaurants). Unlike traditional speech recognition tasks that use steady-state, co-located noise, our use of four distinct, spatially separated podcast conversations as maskers allowed participants to leverage cues such as voice continuity, spatial location, prosody, and talker characteristics. This more complex noise environment provided an assessment of speech understanding designed to better approximate real-world listening conditions, especially for our diverse group of older adults with varying hearing abilities (Brungart et al., 2001; Darwin and Hukin, 2000; Souza et al., 2007; Theunissen et al., 2009; Vermiglio et al., 2019).

To further enhance ecological validity, the target speech in the SIB task was recorded while the talker was exposed to background noise, inducing the Lombard effect (Lane and Tranel, 1971). This adjustment better reflects natural conversational speech in noisy settings and facilitates more direct comparison between the SIB task and dyadic conversation, where the Lombard effect is also expected to occur.

The inclusion of real-time self-report measures allowed us to capture both speech understanding and listening effort. This is particularly valuable, as listening effort does not always scale linearly with intelligibility, especially for individuals with hearing impairment, and may reflect listening-related fatigue (Alhanbali et al., 2017).

4.3 Visual information: impact on hearing assessment

Adding visual cues (AV SIB) significantly improved speech perception compared to auditory-only (A-only SIB), consistent with prior work (Sumby and Pollack, 1954). We initially hypothesized that the presence of AV speech cues, more specifically the benefit over speech recognition with only A-only cues, would interact with hearing thresholds of the participants (based on the 4-PTA), as hearing-impaired individuals were expected to extract more information from the visual signal or were able to use the information more efficiently (Puschmann et al., 2019). Interestingly, the AV SIB task did no better job at explaining hearing performance as measured by the 4-PTA than the A-only SIB task, and thus the benefit of visual cues did not interact with hearing thresholds. This discrepancy with earlier literature may be explained by our use of spatially separated maskers, which could have enabled selective auditory attention regardless of hearing status (Varghese et al., 2012).

Nonetheless, the presence of visual information did uniquely contribute to our evaluation of hearing performance. The AV SIB task, but not the A-only SIB task, significantly correlated with the retrospective self-report SSQ-12 Qualities subscale, which represents experiences of clarity, separation, and identification of speech and sounds in everyday environments (Akeroyd et al., 2014). The presence of visual information may contribute to a more global, qualitative perception of the listening experience; for example making the interaction feel more natural and less effortful. Real-time self-reports of speech understanding and listening effort during AV SIB also strongly correlated with objective (SIB, 4-PTA) measures as well as real-time self-reports of listening effort with the retrospective self-report questionnaires (SSQ-12, HHI-R), suggesting that both the experience (real-time self-report) as the actual speech intelligibility (SIB performance) better reflects both real-time and retrospective self-evaluations of the listener's hearing experience, when visual information is present.

These results thus support the conclusion that incorporating AV cues into assessments of speech intelligibility enhanced the correlation with (both real-time as retrospective) self-report hearing experiences. Future work can determine whether implementing the AV modality in a clinical SIB task can also improve fitting of hearing correction tools (like hearing aids), as there often seems to be a mismatch between the “ideal” hearing aids settings (based on 4-PTA and auditory SIN tasks) and the individual's experience/preference.

4.4 Assessment focus: audibility, intelligibility, comprehension, and vocal effort

A brief (4-min) podcast listening comprehension task in noise correlated strongly with both 4-PTA and (A-only and AV) SIB performance, and real-time self-reports during this task predicted objective hearing measures. An interesting aspect of this approach is that the participant was asked to listen and then respond to four multiple-choice questions about the content, rather than repeating speech fragments as in more classic speech intelligibility tasks. These findings suggest that comprehension-based tasks, especially in noise, are sensitive to hearing difficulties even in those with normal 4-PTA. However, it is of note that Sommers et al. (2011) did not find a correlation between audibility and listening comprehension until the age of 65. Therefore, it may be worth investigating whether these promising findings hold up for a younger population in future studies.

We measured increase in vocal intensity during conversation in noise and expected a larger change of intensity in the voice of individuals who struggle more with background noise, as it is known that we start speaking louder when background noise is introduced (The Lombard effect: Lane and Tranel, 1971). We also expected the louder the noise the more present the effect would be (Bottalico et al., 2022). However, this is not what we observed. The measured increase in vocal intensity did not reliably index hearing loss, as the effect was similar across hearing abilities. However, increased vocal effort was strongly associated with higher self-reported listening effort, indicating that this measure may help identify individuals experiencing listening difficulties not (yet) captured by standard audiological tests.

4.5 Interactive communication: familiar partner effects

Conversing with a familiar partner (simulation-based AV) was expected to improve speech intelligibility (Domingo et al., 2020). In our task, where the majority of the participants (22/26) conversed with a familiar partner, it indeed led to higher ratings of speech understanding and even pleasantness, and lower listening effort, compared to other scenarios. This was supported by the finding that the real-time self-report rating of listening effort during the conversational scenario significantly correlated with the vocal effort. However, neither self-report nor objective measures (vocal intensity change) in this context provided additional insight into hearing function beyond classic tasks. Thus, while real-world listening conditions and participant experience improved, diagnostic value is rather limited to representing the listening experience.

4.6 Naturalness of environment: lab vs. simulation-based

Comparing sound-attenuated rooms (lab-based), and slightly more reverberant environments like a living room (simulation-based), we found that the specific task manipulation (e.g., A-only vs. AV, comprehension vs. vocal effort) rather than the environment itself determined the correlation with hearing experience. Thus, increasing environmental naturalness alone did not enhance assessment sensitivity.

This finding highlights the complexity of striving toward ecological validity and suggests that no single factor, whether environment, modality, or interactivity, is sufficient on its own (Verbeke et al., 2024). Rather, a combination of factors may be necessary to fully capture the challenges of real-world listening. Our study represents a step toward these goals rather than a full validation of everyday functioning. The simulation-based environment, while designed to be more naturalistic than a traditional sound booth, was still a controlled experimental setting, and the tasks still not fully replicate the unpredictability and variability of everyday listening situations. Therefore, while our findings provide preliminary evidence that assessments designed to better approximate real-world listening conditions may offer additional insights into hearing difficulties, further research is needed to determine whether these assessments provide qualitatively different information that translates to improved clinical outcomes.

4.7 Limitations

While this study incorporated several intentional manipulations to better approximate real-life listening experiences, there are important limitations to consider. Notably, we did not systematically measure or control these manipulations across all assessment types, which may influence the comparability of our results, between tasks and with existing literature.

All self-report ratings were collected during brief lab-based and simulation-based sessions, rather than in participants' everyday environments. This differs from established real-time self-report protocols, which often involve longer-term, in-situ monitoring (Holube et al., 2020). Despite this, our adapted real-time self-report measures of speech understanding and listening effort did show strong correlations with objective measures and collecting self-reports during controlled tasks may also offer practical advantages, such as improved compliance and motivation compared to at-home monitoring (Smeds et al., 2021).

Another limitation of our study is the reliance on the 4-PTA as the primary auditory predictor. More recent evidence does suggest that high-frequency thresholds above 8 kHz may be more strongly related to speech-in-noise performance than conventional audiometric measures (Monson and Trine, 2023; Motlagh Zadeh et al., 2019; Yeend et al., 2019). Our reliance on 4-PTA may therefore underestimate the full impact of hearing loss on real-world listening challenges, and future studies should incorporate these measures to provide a more comprehensive assessment of the relationship between hearing thresholds and performance assessments designed to better approximate real-world listening conditions.

Furthermore, our study did not explicitly control for demographic and health-related factors such as age-related cognitive differences, education, socioeconomic status, or physical and mental health. All of these might have contributed to increased variability in our measured speech-in-noise performance and subjective hearing measures (Adachi and Paul, 2024; Pronk et al., 2013).

4.8 Clinical applicability

While our findings suggest that assessments designed to better approximate real-world listening conditions may provide valuable information about individuals' hearing challenges, it is important to consider the practical feasibility of implementing such assessments in routine audiological practice. Our test battery was rather long (2–3 h).This may limit the scalability of such approaches in typical clinical settings, where time constraints and resource availability are significant considerations. However, certain elements of our protocol could be more readily integrated into existing clinical workflows. Specifically, the use of AV stimuli, real-time self-report measures, and recordings of dyadic conversations represent feasible additions that do not necessarily require extensive infrastructure or time investment.

5 Conclusion

This study demonstrates that integrating assessments designed to better approximate real-world listening conditions, such as AV speech intelligibility, listening comprehension in noise, and real-time self-report measures, improves the assessment of hearing ability compared to traditional clinical tools alone. The addition of AV cues not only enhances objective performance but also strengthens the alignment between test results and individuals' self-reported hearing experiences. Real-time self-reports of speech understanding and listening effort, collected during simulation-based listening scenarios, offer insights into hearing difficulties that may be missed by retrospective questionnaires or 4-PTA.

Our findings support the adoption of assessment methods designed to better approximate real-world listening conditions in both research and clinical practice. By bridging the gap between laboratory measures and real-world communication, these approaches enable a more accurate understanding of hearing challenges and can inform more personalized interventions.

Furthermore, this work shows that measures that more closely resemble the listener's daily experience help us to better understand the role of hearing in everyday life, while still maintaining a controlled environment.

Generative AI statement

The author(s) declared that generative AI was used in the creation of this manuscript. AI tool was used to help with grammatical improvements of the manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fauot.2026.1772008/full#supplementary-material

Supplementary material Supplementary material at the source

Footnotes

Edited by: Nicola Quaranta, University of Bari Aldo Moro, Italy

Reviewed by: Maxime Perron, University of Toronto, Canada

L. C. Goh, University of Malaya, Malaysia