Evaluation of research productivity among academic neuro-ophthalmologists using the relative citation ratio

1 Introduction

Measures of research productivity have important implications for decisions regarding faculty hiring, promotion, and allocation of research funding (1, 2). While commonly utilized bibliometrics have attempted to objectively and accurately gauge research output, many have been criticized as an oversimplification of scholarly activity, often conflating various independent measures into a single value (3, 4). Numerous metrics, such as the h-index, are heavily influenced by the career duration, therefore disadvantaging younger researchers or those who have experienced career interruptions, such as women taking breaks for parental leave (3). Furthermore, many suffer from a lack of field standardization, which hinders the comparability of authors working across different specialties of varying sizes and scopes (4–6).

In light of these shortcomings, the National Institutes of Health (NIH) introduced the relative citation ratio (RCR), an article-level metric aimed to better quantify research impact and productivity (7). The RCR is determined by dividing an article’s citation rate (total citations per year) by the expected citation rate, which is based on the performance of co-cited articles benchmarked against NIH R01-funded publications. This allows for standardization of an article’s impact relative to others in its field, with an average NIH RCR value of 1. Unique to the RCR is its incorporation of a co-citation network, which facilitates field standardization and thus enables direct comparisons of impact across publications at both inter- and intra-field levels (7, 8). This metric has been validated in a dataset of over 88,000 publications, demonstrating consistency with expert opinion of research quality (7).

Mean and weighted RCR values are calculated by averaging and summing, respectively, all article-level RCR scores for an individual (7). The mean RCR (mRCR) assesses an author’s overall impact within their field, independent of time-dependent factors such as total publication or citation counts. The weighted RCR (wRCR) considers an author’s total productivity by factoring in number of publications. By separating measures of impact and productivity into distinct metrics, the RCR provides a unique opportunity for a more nuanced approach to bibliometric analyses.

To date, no bibliometric analyses have been conducted among academic neuro-ophthalmologists. Application of the RCR to other academic ophthalmologic subspecialties has provided insight into gender, career length, and academic rank differences in research productivity, however, the field of neuro-ophthalmology lacks similar investigation (9–12). Given the absence of prior bibliometric analyses in academic neuro-ophthalmology, this study was designed as an exploratory analysis to characterize research impact and productivity using the RCR and evaluate associations between faculty characteristics and bibliometric measures.

2 Materials and methods

2.1 Inclusion and exclusion criteria

A list of all ophthalmology residency programs accredited by the Accreditation Council for Graduate Medical Education (ACGME) in the United States was accessed in May 2022 (https://apps.acgme.org/ads/public/). The department websites of all 125 ACGME-accredited programs were searched to identify faculty members who completed a neuro-ophthalmology fellowship following ophthalmology or neurology residency. A total of 103 ophthalmology programs were found to have at least one fellowship-trained neuro-ophthalmologist. All listed faculty that met the criteria were included. Faculty demographics, including gender, residency type, acquisition of a Doctor of Philosophy (PhD), academic rank, and career duration, were collected using department websites, Doximity, and other publicly available professional sources. When not explicitly stated, gender was assigned only when there was consistent identification across institutional and professional sources; no faculty members required exclusion because gender could not be determined. Academic ranks included assistant professor, associate professor, professor, and “other”, which included titles with qualifiers, such as “clinical” or “adjunct”; titles such as “instructor”, “lecturer”, or “staff”; and otherwise, unspecified titles. Career duration was determined by using residency start year and further stratified faculty into the following categories: ≤1980, 1981-1990, 1991-2000, 2001-2010, >2010. For faculty members who obtained a medical degree internationally, residency start year was defined as the beginning of their first neurology or ophthalmology residency. The study was deemed exempt from Institutional Review Board review.

2.2 Bibliometric analysis

The NIH iCite Website (https://icite.od.nih.gov/) was accessed between October 2023 and January 2024 to collect publication counts, mRCR, and wRCR for each neuro-ophthalmologist. Only PubMed-indexed publications and citations starting in 1980 are included in the iCite database. Before 2002, authors’ first and middle names are represented as initials in PubMed. Thus, faculty members who began their residency before 2002 were searched using first and middle initials, if applicable, along with their last name. Those who began residency training in 2002 or later were searched using their full first and last names. For faculty with hyphenated or parenthesized last names, both names and the full hyphenated name, if applicable, were utilized for indexing. Publications were then consolidated in these instances to yield aggregate values. This occurred for 7 faculty (2.7%). Individual publications were reviewed with attention to ophthalmology, neurology, and basic science adjacent journals to remove those from authors of similar names from other medical specialties or scientific fields. For authors with names generating exceedingly high numbers of publications that could not feasible be reviewed manually, publicly available sources, including departmental websites, ResearchGate, ORCiD, and Doximity, were used for cross-referencing publications.

2.3 Statistical analysis

Data were compared by gender, career duration, academic rank, residency type, and acquisition of a PhD. Due to the skewed distribution of data, the results are presented as the median and interquartile ranges (IQR). Univariate and multivariate ordinary least squares regression were used for subgroup analyses of influencing publication count, mRCR, and wRCR, including gender, type of residency completed, PhD status, academic rank, and career duration. The significance threshold was set at 0.05. To account for potential heteroskedasticity, all linear regressions were estimated using heteroskedasticity-consistent standard errors (HC3). Chi-squared goodness-of-fit test was used to determine whether proportions of academic title and residency start year differed between genders. Statistical analyses were performed using IBM SPSS Statistics for Windows, version 27 (IBM Corp., Armonk, N.Y., USA).

3 Results

The analysis included a cohort of 256 academic neuro-ophthalmologists from 102 institutions (Table 1). The median number of publications was 26 (IQR, 8.25-84.50) (Table 2), the median mRCR was 1.23 (IQR, 0.78 – 1.68) (Table 3), and the median wRCR was 31.48 (IQR, 7.61 – 99.94) (Table 4).

Table 1. Demographics of neuro-ophthalmology faculty (n = 256).

Other includes instructors, lecturers, titles with qualifiers such as “clinical”, “adjunct”, or “affiliate”, and faculty not otherwise specified.

Table 2. Publications by gender, residency, PhD acquisition, academic ranking, and residency start year.

P values generated using univariate and multivariate ordinary least squares regression. Bold values indicate statistical significance at the <0.05 significance level.

Other includes instructors, lecturers, titles with qualifiers such as “clinical”, “adjunct”, or “affiliate”, and faculty not otherwise specified.

Table 3. Mean RCR by gender, residency, PhD acquisition, academic ranking, and residency start year.

P values generated using univariate and multivariate ordinary least squares regression. Bold values indicate statistical significance at the <0.05 significance level.

Other includes instructors, lecturers, titles with qualifiers such as “clinical”, “adjunct”, or “affiliate”, and faculty not otherwise specified.

Table 4. Weighted RCR by gender, residency, PhD acquisition, academic ranking, and residency start year.

P values generated using univariate and multivariate ordinary least squares regression. Bold values indicate statistical significance at the <0.05 significance level.

Other includes instructors, lecturers, titles with qualifiers such as “clinical”, “adjunct”, or “affiliate”, and faculty not otherwise specified.

3.1 Gender

The study cohort was predominantly male (60.9%). Male faculty exhibited greater publication counts (17.5 [IQR, 8.0-41.5] vs. 34.0 [IQR, 11.0-109.0]; p = 0.0002) and wRCR scores compared to females (15.63 [IQR 7.03-59.01] vs. 45.93 [IQR 9.90-158.90]; p = 0.012). No difference in mRCR score was observed between genders (1.30 [IQR 0.78 to 1.56] vs. 1.18 [IQR 0.80 to 1.87]; p = 0.07). Multivariate analysis did not demonstrate a significant independent influence of gender on publication count (12.3 [-6.6 to 31.1], p = 0.20 [relative to female]), mRCR (0.06 [-0.22 to 0.35], p = 0.65 [relative to female]), or wRCR (21.84 [-21.44 to 65.12], p = 0.32 [relative to female]).

The distribution of academic titles significantly varied by gender (X² [3, N = 256] = 24.70, P < 0.0001), with female faculty being underrepresented in higher academic ranks (Figure 1A). The distribution of residency start year also differed significantly by gender (X² [4, N = 256] = 40.93, p < 0.0001) with fewer female faculty starting their careers in earlier decades (Figure 1B).

3.2 Residency type

The majority of neuro-ophthalmology faculty completed an ophthalmology residency (69.1%) compared to those who completed a neurology residency (26.6%) or both ophthalmology and neurology residencies (4.3%) (Table 1). Univariate and multivariate analysis demonstrated no statistical differences in publication number and wRCR when comparing type of residency completed. However, completion of a neurology residency was associated with higher mRCR than completion of an ophthalmology residency on univariate (-0.52 [-0.98 to -0.07], p = 0.025 [relative to neurology]) and multivariate analysis (-0.58 [-1.07 to -0.10], p = 0.018 [relative to neurology]).

3.3 PhD acquisition

30 (11.7%) faculty members held a PhD degree. No significant differences in publication count, mRCR or wRCR were demonstrated on univariate or multivariate analyses in faculty holding a PhD relative to those without a PhD.

3.4 Academic rank

The most prevalent academic rank in this cohort was professor (28.9%), followed by “other” (27.7%), assistant professors (25%), and associate professors (18.4%). The academic title of professor was associated with greater publication counts (p < 0.0001), mRCR (0.040), and wRCR (p < 0.0001) relative to the reference of assistant professor on both univariate and multivariate analyses. The academic title of associate professor was associated with greater publication counts (p = 0.0006) and wRCR (p = 0.022) relative to the reference of assistant professor on both univariate and multivariate analyses.

3.5 Career duration

The greatest number of faculty started their residency between 2001-2010 (24.2%), followed by >2010 (22.3%), 1991-2000 (19.9%), 1981-1990 (19.1%), and ≤1980 (14.5%). Faculty starting residency between 2001 and 2010 (p = 0.0002) and after 2010 (p < 0.0001) demonstrated significantly fewer publications relative to those starting their careers on or before 1980 on univariate analysis, however, no significant differences were observed on multivariate analysis. No significant difference in mRCR was observed between cohort of career start dates on univariate or multivariate analyses. Faculty starting residency between 2001 and 2010 (p = 0.0001) and after 2010 (p < 0.0001) demonstrated significantly lower wRCR relative to those starting their careers on or before 1980 on univariate analysis. No significant differences were observed on multivariate analysis.

4 Discussion

To our knowledge, this is the first bibliometric analysis of academic neuro-ophthalmologists using the National Institutes of Health (NIH) Relative Citation Ratio (RCR). By separately evaluating publication count, mean RCR (mRCR), and weighted RCR (wRCR), this study provides a comprehensive assessment of both research productivity and article-level influence within the academic neuro-ophthalmology workforce. These findings establish benchmark bibliometric values for the specialty and provide a foundation for future investigations of academic scholarship, faculty development, and workforce trends.

The distinction between mRCR and wRCR is an important feature of the RCR methodology. Mean RCR reflects the average influence of an author’s publications independent of publication volume, whereas weighted RCR incorporates both article influence and scholarly output, representing cumulative research impact. Accordingly, these metrics should be interpreted as complementary rather than interchangeable measures of research performance. Across the cohort, academic neuro-ophthalmologists demonstrated a median mRCR of 1.23, exceeding the NIH benchmark value of 1.0 and indicating that, on average, publications authored by neuro-ophthalmologists receive greater citation influence than the typical R01 NIH-funded publication. Simultaneously, the median weighted RCR and publication count demonstrate substantial cumulative scholarly productivity across the specialty.

Among the variables examined, academic rank demonstrated the strongest and most consistent association with research productivity. Professors exhibited significantly greater publication counts, mean RCR, and weighted RCR than junior faculty, even after adjustment for career duration and other covariates. These findings are consistent with previous bibliometric studies across ophthalmology and other medical specialties and likely reflect the cumulative nature of scholarly activity, the importance of sustained research productivity in academic promotion, and increased opportunities for mentorship, collaboration, and research leadership that accompany advancement through academic ranks (2, 13–17).

Completion of a neurology residency was independently associated with higher mean RCR, whereas publication count and weighted RCR did not differ significantly by residency background. Because weighted RCR incorporates publication volume and represents cumulative research influence and, these findings should not be interpreted as evidence that neurology-trained faculty have greater overall scholarly impact. Rather, the observed association suggests differences in the average citation influence of individual publications, the underlying reasons for which remain uncertain. Future investigations evaluating research focus, publication type, funding mechanisms, collaborative networks, and institutional characteristics may help clarify this relationship. We have intentionally avoided attributing this finding to differences in fellowship training or clinical practice patterns, as these factors were not evaluated in the present study.

Although faculty were identified through ACGME-accredited ophthalmology residency programs, the residency training distribution within our cohort closely parallels that reported in a recent national demographic study of United States neuro-ophthalmologists, which included members of the American Academy of Ophthalmology, North American Neuro-Ophthalmology Society, American Neurological Association, and American Academy of Neurology (18). That study reported that 67.6% of neuro-ophthalmologists completed ophthalmology residency, 25.0% completed neurology residency, and 7.4% completed both, compared with 69.1%, 26.6%, and 4.3%, respectively, in our cohort. The close agreement between these independent datasets supports the representativeness of our study population despite identification through ophthalmology residency programs. Nevertheless, future bibliometric analyses incorporating neurology residency programs may provide an even more comprehensive characterization of the academic neuro-ophthalmology workforce.

Gender differences were similarly clarified after adjustment for potential confounders. While male faculty demonstrated greater publication counts and weighted RCR on univariate analyses, these associations were no longer significant following multivariable adjustment, suggesting that differences in scholarly productivity were largely explained by other factors, including academic rank and career duration, rather than gender itself. The significant differences in academic rank and residency cohort distributions observed between males and females (Figure 1) further support the interpretation that career stage, rather than gender itself, largely explains the unadjusted differences in scholarly productivity. At the same time, women remained substantially underrepresented among professors, with only 21.6% of faculty holding the rank of professor being female. Because this cross-sectional study cannot determine the mechanisms underlying this disparity, our findings should not be interpreted as evidence of inequities within the promotion process alone. Rather, the observed distribution may reflect multiple contributing factors, including historical workforce demographics, differences in career stage, institutional promotion practices, mentorship opportunities, and other structural influences that warrant further investigation. Encouragingly, female representation increased substantially among more recent residency cohorts, suggesting continued progress toward a more balanced academic workforce.

The present findings have practical implications for academic departments and faculty development. Objective bibliometric measures are increasingly incorporated into decisions regarding recruitment, promotion, mentorship, and research funding. By establishing specialty-specific benchmark values for publication count, mean RCR, and weighted RCR, this study provides a contemporary reference against which academic neuro-ophthalmologists may compare scholarly activity while recognizing that bibliometric measures represent only one component of academic success. Clinical excellence, education, mentorship, administrative leadership, and service remain essential contributions that are not captured by citation-based metrics.

Several limitations merit consideration. As with all studies utilizing the NIH iCite database, only PubMed-indexed publications published from 1980 onward are included, potentially underestimating scholarly output among investigators with earlier careers start dates. The RCR does not account for publication type, authorship position, or individual contribution to multi-author publications. Review articles, consensus statements, and large multicenter clinical trials may achieve substantially different citation patterns than original investigations or case reports despite representing different forms of scholarly contribution. Despite careful manual review of publications and consolidation of authors with hyphenated or parenthesized surnames, errors related to author name ambiguity or undocumented name changes remain possible. In these instances, articles were reviewed on a case-by-case basis and cross-referenced with available resources, however, the opportunity for error remains. As women are more likely to undergo a name change during their career, this may result in a disproportionate underestimation of their research productivity.

In conclusion, academic neuro-ophthalmologists demonstrate high levels of scholarly productivity and article-level research impact. Academic rank was the strongest independent predictor of cumulative scholarly output, while completion of a neurology residency was associated with higher mean RCR but not greater cumulative research impact. Women demonstrated research productivity comparable to men after adjustment for career duration and academic rank, although they remain underrepresented in senior academic positions. Together, these findings establish benchmark bibliometric values for academic neuro-ophthalmology and provide a framework for future investigations examining scholarly development, workforce diversity, mentorship, and academic advancement.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Ali Hamedani, University of Pennsylvania, United States

Reviewed by: Eric D Gaier, Harvard Medical School, United States

Leanne Stunkel, Washington University in St. Louis, United States