Beyond traditional paradigms: a bibliometric and systematic content analysis of quantitative and qualitative methodological innovations in organizational psychology

1 Introduction

Organizational psychology is at a really important point in its research. For a long time, the main evidence for ideas in the field has been what people report in questionnaires at a single point in time, followed by linear regression or ANOVA to analyze the data. And while this approach did give us important, basic theories about motivation, leadership, and how jobs are structured, the way it assumes we find things out doesn't really fit with how organizations are these days. Today's workplaces are defined by management by algorithms, hybrid or fully remote working, teams that change and blur, short-term “gig” work, and a constant stream of information being created by digital activity, all of which are constantly changing, influencing each other, and very much tied to their situation (Kellogg et al., 2020). So, the main issue for research is not just about better techniques; it is about using what we know to understand the real world. Organizational psychology has a growing number of increasingly complex methods for investigating phenomena. These include dynamic structural equation modeling (DSEM), exponential random graph models, reflexive thematic analysis, studying digital interactions as an anthropologist (digital ethnography), understanding language with computers (natural language processing), and building computer simulations of how people act (agent-based modeling), but we don't have a good, carefully checked plan for using them on questions about organizations (Crawford and Kelemen, 2023; Hamaker et al., 2018).

And this problem is not just for academics to discuss. When researchers look at things that change over time, like how safe people feel in a team, how an organization's values are built up, or how leaders get to be leaders and only use a single point in time for their data, they get conclusions that are seriously misleading (McCormick et al., 2020). Problems like people answering questions in similar ways regardless of the question, it being hard to tell what causes what, and applying what is true for a group to everyone all make these limitations even worse. This can lead professionals to make choices at work that aren't based on solid evidence (Aguinis et al., 2016; Mitchell et al., 2019). Also, because organizations are collecting huge amounts of data about what people do, how they communicate, and how well they perform, if organizational psychology does not use really advanced computer methods, it will lose out to data scientists and management consultants who do not have the same understanding of the fundamental theories of the field (Leavitt et al., 2021).

There is a big difference between having good research methods available and researchers using them. For example, Hamaker et al. (2018) showed that DSEM is a much better way to look at how many differences exist within a single person, compared to older, unchanging “latent variable” methods. However, by 2022, less than 4% of research based on proof in organizational journals used DSEM (Gabriel et al., 2019). And Snijders (2011) explained that “stochastic actor-oriented models” are particularly well-suited to understanding how networks and behavior change together in organizations. Yet, reviews show that under 6% of organizational network research uses this method (Lazega and Snijders, 2022). In qualitative research, Braun and Clarke (2021, 2022) significantly changed how thematic analysis is conducted. They now view researchers' consideration of their own influence as a useful part of understanding, not a problem for ensuring the results are correct. But Pratt et al. (2020) found that most qualitative studies in organizations still treat researcher self-awareness as merely a step to be completed, rather than as a substantive part of the analysis itself. What makes this problem even bigger is that there aren't many instances of combined (mixed-methods) research in which the numbers and the words are genuinely woven together, rather than just presented side by side (Buchanan and Bryman, 2022; Denzin and Lincoln, 2022).

What is more, when people have reviewed the methodology in organizational psychology, they've either concentrated on newer numerical techniques (Bauer and Curran, 2023; McNeish and Wolf, 2023) or improvements to qualitative methods (Saldaña, 2021; Strauss and Corbin, 2022). They have not brought the two ways of thinking about research together. Also, we haven't developed thinking on the moral aspects of new methods in organizational studies, especially the challenges of keeping people's data confidential when obtaining information from their digital activity and the fairness of the use of algorithms (Kellogg et al., 2020; Orlikowski and Scott, 2021). The goal of this research is to carefully examine and summarize the literature on new approaches in organizational psychology from 2020 to 2025. This will be done through a broad statistical analysis of academic databases and a detailed, systematic review of published content; importantly, we won't be asking people to participate in the study directly. We are specifically trying to measure how many publications and how many citations are related to more complex statistical and non-numerical methods in organizational psychology, to find the main groups of ideas and what we don't know in the literature on method innovation, to thoroughly consider the philosophical, real-world and moral compromises of these newer approaches, and to offer suggestions for choosing and combining methods that are backed up by proof.

2 Literature review

2.1 Quantitative breakthroughs: modeling dynamics, networks, and non-linearity

Looking back over the last 5 years, the biggest step forward in measuring phenomena in organizational psychology has been the development of methods to examine what's going on within a single person over time, using frequent measurements and a method called DSEM (Hamaker et al., 2018). Normally, in organizational psychology, we compare people to one another based on their average levels of a given attribute. This inevitably overlooks how things change for each person over time and how those changes affect what happens at work. In fact, Curran et al. (2014) showed that how job demands and how people are doing at work go up and down within each person in a predictable way, but you wouldn't see that in a simple snapshot or in a typical long-term study. DSEM, commonly implemented using specialized statistical platforms such as Mplus, enables researchers to examine how people differ from each other generally, how their behavior at one point in time influences their behavior soon after, and how different things relate to each other as they're measured many times within the same people, usually by asking them about their experiences throughout the day (Gabriel et al., 2019; Sonnentag, 2015).

At the same time, social network analysis has become more sophisticated in both its thinking and its use of computing power. Exponential random graph models (ERGMs) and their extensions for looking at changes over time, stochastic actor-oriented models (SAOMs) mean researchers can now model the processes of social interaction—things like giving and getting back, the tendency for friends to have friends in common, and people with similar characteristics being linked that determine how formal and informal networks in organizations are created and change (Snijders, 2011). Lazega and Snijders (2022) point out that SAOMs are particularly useful for investigating how advice networks, with whom people collaborate, and their friendships develop alongside behaviors such as knowledge sharing, innovation, and people leaving, all of which are central to organizational psychology. The benefit of SAOMs over simply surveying who knows whom is not just statistical; they shift the researcher's role from describing a network at a single point in time to explaining how the network develops as people make choices within the existing structure (Clauset et al., 2015).

Computational methods and machine learning are a third area of new development. Grimmer et al. (2022) have developed a robust approach to treating text as organizational data, enabling the analysis of large volumes of a company's culture as expressed in internal emails, performance evaluations, policy documents, and company-wide announcements. Leavitt et al. (2021) suggest that applying supervised machine learning to data about how people act and communicate can reveal hidden patterns of problems in an organization, like early signs of a harmful culture or a spreading loss of motivation, that you wouldn't find with questionnaires or traditional analysis of written content. However, they also wisely caution that without a strong theoretical basis and a critical approach, machine learning could simply become a very complex way of finding patterns that exacerbate unfairness already present in past data within an organization. Agent-based modeling (ABM) takes this even further by letting researchers create simulations of how organizational things happen from basic rules of individual behavior; this provides a strong way to test ideas about how teams work, how organizations bounce back from difficulties, and how cultures change, ideas which can't be tested with actual data alone (Kozlowski and Chao, 2012).

2.2 Qualitative innovations: rigor, reflexivity, and the digital turn

Three developments that go together have really changed how we do qualitative work in organizational psychology: a new approach to analyzing themes in data, the acceptance of doing ethnography (being an observing researcher) in digital spaces, and carefully using “process tracing” to show how things cause other things in the study of single organizations. Braun and Clarke's reflective thematic analysis (from 2021 and 2022) is a huge departure from the old idea of ensuring that different researchers coding the same thing agree with each other (which comes from a way of thinking about research that assumes reality is hard and measurable). Instead, it is about the researcher's own views, beliefs, and experience, used as useful parts in interpreting the data, not things to be gotten rid of as “bias.” Lots of people have praised this new way of looking at things as correcting the way thematic analysis had become almost like counting, but others (Pratt et al., 2020; Rossman and Rallis, 2022) believe that when we use many research methods, we need clearer lines to show what each one is doing.

Digital ethnography has emerged specifically because of changes in organizations and the extent to which their social interaction now happens on computers. As more and more of what people in organizations say to each other, do together, and do to get power happens on things like Slack, Microsoft Teams, Zoom, and company social media, the traditional “place” where ethnography is done a limited, all-at-once, and real-time view has broken up into something spread out, not happening at the same time, and controlled by the platform (Orlikowski and Scott, 2021). Spreitzer et al. (2017) suggest that digital ethnography, which can be done by watching organizational data on platforms without getting involved, by observing people in virtual spaces, and by analyzing what digital “footprints” people leave, lets you look at the hidden parts of organizations that you cannot see with regular fieldwork. It is very useful for understanding how working from home, or a mix of home and office, creates new expectations about when people are available, how they present themselves, and how they subtly try to influence others (Hardy and Maguire, 2015).

Process tracing, taken from political science and used to examine single organizations, addresses the long-standing criticism that qualitative case studies don't really allow you to draw firm conclusions (Shepherd and Sutcliffe, 2011). It does this by carefully checking if the cause-and-effect relationship you think is happening leaves evidence at each point in the way you think it works. This turns a case study from a simple example into one that can explain why something happened, even when you are only looking at one instance. This is in line with “realist evaluation” (especially the work of Pawson and Tilley, which examines context-mechanism-outcome) and means that organizational researchers should explain how and why something works, and in what situation (Nag et al., 2007). Qualitative longitudinal research (QLR) takes this further by looking at how meanings, who people are and what they do, change over long periods of time; it gives a sense of process and development that you don't get from qualitative work at one point in time, or from looking at numbers over time (Saldaña, 2021).

2.3 Mixed-methods advancements: genuine integration over methodological pluralism

The main criticism of using both numbers and words (mixed methods) in organizational psychology is that, in the past, it has generally involved doing quantitative and qualitative work side by side, with only a brief mention of how they relate in the discussion. However, more recent approaches are changing that by integrating multiple methods into the core of the research design. Qualitative Comparative Analysis, or QCA, uses Boolean logic and set theory to show this kind of merging; it lets researchers identify which combinations of conditions are sufficient to make something happen in an organization, rather than assessing the overall impact of many different factors. Unlike standard regression, which assumes causes and effects are additive, QCA can handle complex causation. It does this through equifinality (the idea that many different routes can lead to the same result) and asymmetric causation, both of which are pretty much expected given how organizations work.

Another way to integrate is through mixed-methods social network analysis: it combines quantifying network-level measures like centrality, density, and structural holes with examining what people in the network think about their connections. Humphrey and Aime (2014) showed that this addresses a major issue with networks studied purely with numbers: they can't distinguish between connections that look the same on a map but are, in reality, very different in how people experience them. And if you add brief descriptions (vignettes) to studies where people report their experiences at multiple time points, you can add details of the actual events to the numbers, charting how someone is changing. Gabriel et al. (2019) and Morrison (2011) suggest that this is especially useful for examining emotional labor, moral worries, and threats to someone's sense of self in organizations. Perhaps the most elaborate strategy currently developing is a sequence of machine-learning and survey-based steps. Here, computer analysis of text within an organization is used to generate initial research ideas, which are then tested by asking a representative group of people. Grimmer et al. (2022) and Leavitt et al. (2021) have done work in this area.

2.4 Methodological trade-offs and ethical implementation

When organizations use more sophisticated psychological techniques, they aren't simply objective or morally neutral. Each choice of method is based on a particular idea of what organizations are, how a researcher relates to the people they study, and the research goal. The conflict between how much research gets in the way of what's really going on (ecological validity) and how much the research affects the situation is especially strong with intensive longitudinal designs and passive sensing. Asking employees for information many times a day using experience sampling yields highly detailed data about each person, but it can be a hassle for them and may change what you're looking at. Wearable devices and “sociometric badges” (these devices can capture interpersonal proximity, vocal interaction patterns, and movement behavior) provide a level of detail about behavior we have never had before, but bring up significant worries about watching employees, their independence, and the power differences in a company that control who gets their data collected and who has their data collected about them.

The balance between how well you can understand something and how complex it is also hugely important. Machine learning using organizational data can predict outcomes much more accurately than traditional statistical approaches, but because it is often a “black box,” it is hard to find the theoretical meaning central to this area of study. Grimmer et al. (2022) argue that this is not an unsolvable problem: you can use tools to explain individual predictions, such as SHAP values and LIME, without sacrificing the model's complexity. However, understanding a single prediction well doesn't mean the overall theory makes sense for everyone, and turning a good prediction into a proper theoretical understanding is not something methods clearly show us how to do. When we think about ethical standards for fairness in algorithms in organizational research, we need to think about not only unfairness in the results of the model, but also unfairness that was already present in the older data used to create the algorithm—and standard ethics review boards were not and were not created to deal with this (Aguinis et al., 2021; Fisher et al., 2016).

3 Methodology

3.1 Research design

The study used numbers and data to examine actors, specifically by counting and analyzing publications, along with a careful, organized review of what those publications say. This study got its data from past research available in three major academic databases: Web of Science, Scopus, and APA PsycINFO. Because the researcher did not collect information directly from people, this study did not need to obtain permission from an Institutional Review Board (or consider people's informed consent). The researcher was interested in the academic publications themselves (journal articles, reviews, book sections, and conference papers), not in the researchers or people at the organizations involved. This is standard when analyzing existing data and in “meta-science,” where the goal is to identify and understand trends across all research conducted (Briner and Denyer, 2023; Chadegani et al., 2013).

3.2 Database search and inclusion criteria

The researcher did a careful search of databases in January 2026 and had already planned the search terms. These terms linked topics such as organizational psychology, organizational behavior, industrial-organizational psychology, and management to different research approaches: dynamic structural equation modeling, social network analysis, machine learning, digital ethnography, reflexive thematic analysis, agent-based modeling, qualitative comparative analysis, natural language processing, and the experience sampling method. The researcher only looked at publications in English between January 1st 2020, and December 31st 2025. After removing duplicates, the database search returned 31,482 documents. From those, the researcher kept 14,617 for a more in-depth look. To be included, something had to be a proper study (or a review of studies) published in a peer-reviewed journal and focus on organizational behavior or psychology. The study did not include work solely on clinical, educational, or health psychology that did not relate to organizations, or outlines of presentations (conference abstracts) that were not full articles.

The researcher used the title, abstract, and keyword fields with Boolean combinations such as (“organizational psychology” OR “industrial-organizational psychology” OR “organizational behavior”) AND (“dynamic structural equation modeling” OR “social network analysis” OR “machine learning” OR “digital ethnography” OR “reflexive thematic analysis” OR “agent-based modeling” OR “natural language processing” OR “experience sampling”). Filters included peer-reviewed publications, English language, publication years 2020–2025, and document types restricted to journal articles, reviews, and book chapters. Deduplication involved DOI matching, title-similarity screening, and manual verification across Web of Science, Scopus, and PsycINFO. Bibliometric normalization included author-name harmonization and keyword standardization.

A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow chart was used in this study to transparently illustrate how the researcher identified, screened, and selected the final set of studies for the review. PRISMA was used because it allows readers to see exactly how many records were found at the beginning and how many were removed at each stage, making the selection process reproducible and trustworthy. Hereunder is a clear breakdown of the PRISMA diagram. Step-by-step flow used in this study:

• Records identified

• ° n = 31,482
• ° This is the total number of records (articles, studies, etc.) retrieved from the database searches at the very beginning of the review.

• 2. Duplicates removed

• ° n = 7,964
• ° Duplicate records (the same study appearing in multiple databases) were removed.
• ° Remaining records: 23,518

• 3. Records screened

• ° n = 23,518
• ° These 23,518 unique records were screened (usually by reading titles and abstracts) to determine whether they were relevant to the research question.

• 4. Records excluded

• ° n = 8,901
• ° After screening, 8,901 records were excluded because they did not meet the inclusion criteria (wrong topic, wrong study design, not in the desired language, etc.).

• 5. Eligible studies

• ° n = 14,617
• ° This is the number of studies that passed the screening stage and were considered eligible for inclusion.

• 6. Included in analysis

• ° n = 14,617
• ° All eligible studies were included in the final analysis. There were no further exclusions after the eligibility stage (which is relatively uncommon; many reviews exclude additional studies after retrieving and reading the full texts).

3.3 Analytical procedures

The researcher used VOSviewer (version 1.6.20) and the bibliometrix package in R (version 4.2.1; Aria and Cuccurullo, 2017) to conduct a bibliometric analysis. To illustrate how the use of different methods changed over time, the researcher listed the number of publications and citations each year. The study then identified thematic groups in the methods literature by examining co-occurring keywords, considering only those that appeared together at least 15 times. These keyword pairings were adjusted for how often they'd be expected to occur by chance (using a measure called “association strength”) and then shown as diagrams of points that repel each other, with forces directing their arrangement. Thematic labels for the algorithmically generated clusters were not assigned unilaterally by the author; instead, a structured three-stage procedure involving two independent reviewers was used to validate the interpretive labels before they were applied (see Section 4.3 for the full procedure). The researcher carefully read 420 complete articles (selected randomly from all articles, ensuring a balance of years, method types, and journals with strong, medium, low, or very low influence) to evaluate the strength of the research methods, the extent to which ethical issues were described, and how methods were combined.

The minimum co-occurrence threshold of 15 was selected in line with established bibliometric practice for medium-to-large corpora. Van Eck and Waltman (2014) recommend calibrating this threshold so that the retained keyword network represents approximately the top 10%−20% of unique keyword pairs by frequency, thereby excluding idiosyncratic or noisy pairings whilst retaining all substantively recurrent co-occurrences. Applied to the 14,617-document corpus used here, a threshold of 15 retained 312 unique keyword pairs and produced five stable, interpretable clusters, consistent with the structural resolution guidelines provided in the VOSviewer technical manual (Van Eck and Waltman, 2023). Thresholds of 10 and 20 were also evaluated as sensitivity checks. A threshold of 10 increased the number of retained pairs to 487 and produced six clusters, with the additional cluster representing a methodologically heterogeneous grouping of lower-frequency terms that did not map cleanly onto a distinct research tradition; this indicated over-segmentation driven by noise. A threshold of 20 reduced the number of retained pairs to 198 and merged the digital ethnography and NLP clusters into a single computational-methods cluster, obscuring a distinction that content analysis confirmed was substantively meaningful. The threshold of 15, therefore, offered the best balance between network completeness and cluster interpretability, and the five-cluster solution was robust across the ±5 range examined.

3.4 Situating the study within bibliometric reporting standards

The use of the term “bibliometric analysis” in the title and throughout this manuscript invokes a methodological tradition with established, though evolving, reporting norms. Two principal frameworks are relevant: PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews; Tricco et al., 2018) and ROSES (Reporting standards for Systematic Evidence Syntheses; ?). This study is best characterized as a bibliometric mapping study combined with a systematic content analysis subsample, rather than as a scoping review or a full evidence synthesis in the ROSES sense. It is important to be transparent about where it aligns with and where it departs from each framework. In terms of alignment, the study follows core PRISMA-ScR principles in its treatment of the search and selection process: a pre-specified search strategy was applied across three major databases (Web of Science, Scopus, and PsycINFO); Boolean search strings and eligibility criteria were defined in advance; a PRISMA flow diagram is provided (Section 3.2) documenting identification, deduplication, screening, and final inclusion at each stage; and the full-text content analysis subsample (n = 420) was drawn using stratified random sampling with documented strata. ROSES' emphasis on explicit inclusion and exclusion criteria, database search transparency, and inter-rater reliability reporting is also met: eligibility criteria are stated in Section 3.2, search strings and filters are documented, and inter-rater reliability for the content coding instrument is reported (Cohen's κ = 0.81).

Regarding departures, this study does not conform to PRISMA-ScR in three respects that are methodologically justified. First, PRISMA-ScR expects dual independent screening of titles and abstracts, whereas this study used single-reviewer screening with a 10% verification sample; this reflects the scale of the corpus (31,482 initial records) and is consistent with accepted practice in large-scale bibliometric mapping studies (Zupic and Cater, 2015). Second, neither PRISMA-ScR nor ROSES was designed for the bibliometric-analytical components of this study, namely the co-keyword network analysis, publication trend modeling, and citation distribution analysis, which follow the reporting conventions of the bibliometric methods literature (Aria and Cuccurullo, 2017; Donthu et al., 2021) rather than evidence synthesis guidelines. Third, this study does not synthesize effect estimates or produce pooled quantitative outcomes; it maps a methodological landscape and constructs a conceptual framework, which is outside the scope of both PRISMA-ScR and ROSES as originally conceived. In summary, this study adopts the PRISMA-ScR framework for transparency in selection and screening, supplements it with bibliometric reporting conventions for its network and trend analyses, and explicitly acknowledges the points at which its design exceeds or falls outside the scope of both standards.

To support careful reading, the researcher compiled a list of items to look for, based on the four areas of the broader literature review, and refined it by reviewing 40 articles. Coding categories included methodological transparency, ethical disclosure, quality of mixed-methods integration, epistemological alignment, reproducibility indicators, and digital governance practices. Training involved iterative coding of 40 pilot articles before formal analysis commenced. Stratified random sampling grouped studies by publication year, methodological orientation, journal ranking, and database source, followed by random selection within each stratum to ensure representativeness across methodological traditions. The coding framework comprised six analytical domains: methodological transparency, reproducibility, epistemological alignment, quality of mixed-methods integration, ethical governance, and methodological implementation fidelity. The manuscript also underwent linguistic standardization to ensure consistent spelling, terminology, referencing style, and academic tone throughout the document.

4 Results and discussion

4.1 Publication trends and citation patterns

Table 1 shows the number of papers published each year and their citation frequency over the study period. In total, the number of publications rose from 1,842 in 2020 to 3,312 in 2024, a rise of 79.8%. Papers that relied heavily on numbers (and used advanced statistics or computing, rather than just OLS regression and ANOVA) accounted for 64.8% of the total in 2020 and 64.9% in 2024. Thus, the field remains largely quantitative. The number of papers using qualitative methods increased from 648 in 2020 to 1,164 in 2024, a 79.6% increase, indicating that new qualitative approaches are emerging at a rate comparable to quantitative approaches. The total number of citations each year rose from 38,241 (in 2020) to 79,816 (in 2024), a 108.7% increase. This is much larger than the increase in the number of publications, suggesting that research using new methods is attracting much more attention from scientists. Table 1 presents the annual publication and citation trends associated with methodological innovations in organizational psychology between 2020 and 2025.

Table 1. Annual publication and citation counts: methodological innovations in organizational psychology (Web of Science, Scopus, and PsycINFO combined), 2020–2025.

^*2025 figures cover January–June only. Publication counts include journal articles, systematic reviews, and book chapters. Citations are cumulative annual totals as indexed in the respective databases. Because 2025 data cover only January–June, the 2025 row in this table is presented descriptively and is not used in isolation to calculate year-on-year growth rates; those rates are derived from the full-year 2020–2024 columns. Citation totals for 2025 similarly reflect only the first six months of indexing and will undercount eventual annual citations because of ongoing database accumulation and the 18–24-month citation lag characteristic of this literature.

Figure 1 shows the number of papers published each year, clearly indicating a steady increase. The growth rate accelerated sharply from 2021 to 2022. This speed-up occurred as research priorities shifted after the pandemic to focus on topics such as working from home, monitoring people's use of digital tools, and the algorithmic management of employees. This turning point aligns with what Kellogg et al. said in 2020: COVID-19 was a huge, unplanned change in how organizations work. It created new research questions that older methods simply couldn't handle, which quickly led to more digital ethnography, natural language processing, and experience sampling method (ESM)-based studies. Rudimentary text-based figures were conceptually replaced by publication trend graphs and bibliometric network visualizations generated using statistical software.

4.2 Methodological adoption rates and growth trajectories

Table 2 shows how often certain new ways of conducting research were used and how much their use increased during three periods: 2020 to 2021, 2022 to 2023, and 2024 to 2025. The largest increase was in the use of Natural Language Processing (NLP) for information about organizations (296%), followed by Qualitative Comparative Analysis (239%), Mixed-Methods Social Network Analysis (248%), and Digital Ethnography (229%). These large increases in how often they're used really mean something: the combination of analyzing text with computers and organizational theory is now one of the fastest-growing areas for new research methods. This supports what Grimmer et al. said in 2022: that text is an underused source of information for organization studies, but it also raises issues for understanding the results, as Leavitt et al. described in 2021. Dynamic Structural Equation Modeling (186%) and following a qualitative study over time (182%) both grew significantly, but perhaps not as much, because these methods are more difficult to get into.

Table 2. Frequency counts and growth rates of specific methodological innovations in organizational psychology literature, 2020–2025.

^*2024–2025 figures combine full-year 2024 observed data with a linear annualization of January–June 2025 observed data (January–June 2025 frequency × 2 = projected full-year 2025 equivalent). The 2-year biennial total was then used to compute growth rates relative to the 2020–2021 baseline. Linear doubling was selected for transparency and reproducibility; it assumes a uniform within-year distribution of publications. See Section 4.2 for a full description of this procedure and a discussion of three sources of potential distortion (seasonal publication patterns, accelerating adoption curves, and database indexing lag), all of which suggest that reported growth rates are conservative estimates. ERGM, Exponential Random Graph Model; NLP, Natural Language Processing; SNA, Social Network Analysis.

These growth rates require methodological qualification because the 2024–2025 column in Table 2 combines full-year 2024 data with only January–June 2025 data; the database search was conducted in January 2026, and full-year 2025 indexing was not yet available for all publications. To produce a comparable 2-year period estimate, the observed January–June 2025 frequencies were doubled to generate an annualized projection for the full calendar year 2025. This projected figure was then summed with the complete 2024 count to represent the 2024–2025 biennial total, and growth rates were calculated as the percentage change from the 2020–2021 baseline biennial total to this projected 2024–2025 equivalent. Linear doubling was chosen as the annualization method because it is transparent, reproducible, and requires no distributional assumptions about within-year publication timing. However, three potential sources of distortion should be acknowledged.

First, the method assumes that publications are distributed uniformly across calendar months, whereas academic publishing exhibits systematic seasonal variation: submission and acceptance rates are typically elevated in the second half of the year following major conference seasons, grant reporting cycles, and end-of-year journal volume completions (Lariviére et al., 2013).

If this pattern holds for the methods literature, doubling a January–June count will systematically underestimate full-year 2025 output, so the reported growth rates are likely conservative rather than inflated. Second, fast-adoption methods such as NLP and digital ethnography may follow accelerating rather than linear growth curves; if adoption accelerated over the course of 2025, the linear projection would understate second-half output and therefore underestimate the true growth trajectory. Third, database indexing operates with a lag of approximately 1–3 months, so some publications submitted or accepted in May and June 2025 may not yet have appeared in Web of Science, Scopus, or PsycINFO at the time of the January 2026 search, further compressing the observed January–June baseline from which the projection is derived. For these reasons, the reported growth percentages should be interpreted as conservative lower-bound estimates of actual adoption trajectories rather than as precise point estimates, and comparisons across methods should focus on the relative ordering of growth rates rather than on the absolute percentage values. Table 2 summarizes the adoption frequencies and growth trajectories of major methodological innovations identified in the reviewed literature.

Here is a key point about the link between using many different methods and the quality of those methods. Something being used quickly does not mean it's being used properly. When the researcher carefully reviewed the full texts of 420 articles, the study found that over 43% of those that used machine learning or NLP did not explain how their results were obtained, and 61.4% did not include sufficient technical detail for others to reproduce their work. This aligns with Saraf et al. (2013) and their broader criticisms of the use of computer-based methods in management research. Similarly, 38.7% of articles that said they were doing Braun and Clarke's reflexive thematic analysis were doing something closer to simply summarizing themes, not the deeper, theory-building interpretation (Braun and Clarke 2021) expect. This all indicates that just following the latest methodological trends (using the newest names for methods) without really embracing the principles and how to analyze things they are linked to is as harmful to the advancement of our field as being too stuck in old ways.

Looking at how often these articles are cited adds weight to this concern, but from a different angle. In this collection of articles, the average number of citations for mixed-methods papers was 47.3. Quantitative-only papers averaged 31.6, and qualitative-only papers averaged 19.4. This difference could be due to a genuine blending of methods being more valuable. However, it could also be due to “citation inflation,” in which articles are cited more because they are visible to researchers using both quantitative and qualitative approaches. As a result, they receive more citations, regardless of how good they are. Bakker and De Vries (2021) have said something similar about cross-disciplinary research in occupational health psychology. They point out that in areas that mix different approaches, citation counts tend to exaggerate how widely the combined method is adopted, compared with the number of people citing the original, separate theoretical ideas behind each part of the method.

These three mean citation values are reported as descriptive summaries derived from the full corpus of 14,617 papers and should be interpreted accordingly. No inferential statistical tests (such as a Kruskal–Wallis test or one-way ANOVA with post-hoc comparisons) were conducted to assess whether the differences between groups are statistically significant, and no effect size measures (such as eta-squared or epsilon-squared) were calculated. This is a deliberate methodological boundary: the study is a bibliometric mapping exercise rather than a hypothesis-testing study, and the citation-count distributions for papers of this kind are typically heavily right-skewed and zero-inflated, complicating standard parametric inference. The means are therefore presented solely to characterize the central tendency of each methodological group within this corpus, not to support a causal or inferential claim that mixed-methods research produces higher scholarly impact. The observed ordering (mixed-methods > quantitative-only > qualitative-only) is consistent with patterns reported in analogous bibliometric studies of interdisciplinary work (Lariviére and Gingras, 2010; Plano Clark and Ivankova, 2016), but readers should treat the absolute values and inter-group gaps as descriptive summaries rather than as statistically validated effect estimates.

4.3 Co-keyword network analysis: thematic clusters and emerging Frontiers

Analyzing how often keywords co-occur across the 14,617 papers revealed five main thematic groups, each characterized by a distinct combination of approaches and ideas. Thematic labels for each cluster were assigned through a structured three-stage procedure to ensure that interpretations were grounded in the data rather than in the author's prior theoretical preferences. In Stage 1, the author independently examined the ten most frequent keywords in each algorithmically generated VOSviewer cluster and drafted a candidate label to capture the shared methodological or substantive theme represented by those terms. In Stage 2, both independent reviewers who had participated in the content analysis coding (see Section 3.3) were provided with the ten highest-frequency keyword lists for each cluster, without being told the candidate labels, and were asked to independently propose a descriptive label for each cluster. Reviewers were thus blind to the author's labels at the point of their own labeling. In Stage 3, the author's candidate labels and both reviewers' proposed labels were compared: where all three labels conveyed the same core theme, the author's original label was retained; where discrepancies arose, the three parties discussed the keywords jointly until a consensual label was agreed.

Agreement across all five clusters was reached without substantive disagreement in four cases; Cluster 3 (computational text analysis and organizational culture) required one round of discussion to distinguish the computational-methods emphasis from the broader organizational-culture framing suggested by one reviewer, with the final label reflecting the cluster's methodological rather than topical character. The five cluster labels used below should therefore be understood as inter-subjectively validated interpretive summaries of algorithmically derived keyword groupings, not as fixed or uniquely correct characterizations of the underlying literature. The first and largest group (3,847 papers, 26.3% of all the material) centers on the experience sampling method, within-person variation, dynamic structural equation modeling, how you feel now, and daily diaries. This underscores the importance of taking multiple measurements over time, as is now done in organizational psychology. The second group (2,914 papers, or 19.9%) is built around social network analysis, exponential random graph modeling, team networks, knowledge sharing, and cross-organizational reach. The third (2,631 papers, 18%) involves machine learning, computer-based language processing, information extraction from text, inferring how people feel from their words, and organizational culture. This is the cutting edge of using computers to understand and interpret. The fourth group (2,188 papers, 15%) covers digital ethnography, working remotely, virtual teams, managing with algorithms, and the structure of platform organizations. The fifth (1,893 papers, 12.9%) focuses on qualitative comparative analysis and configurational theory, using both numbers and words, complex ideas about cause and effect, and case-based thinking. The network visualization below illustrates the thematic relationships between dominant methodological approaches identified through co-keyword analysis (see Figure 2).

The distance between these groups in the keyword network is concerning. Those using intensive repeated measurements and statistical analysis (groups 1 and 2) and those using computer approaches and digital qualitative methods (groups 3 and 4) interact little. In other words, those who rely heavily on time-series data rarely use digital ethnography or natural language processing, and vice versa. This separation is exactly the issue that mixed-methods research seeks to address by integrating different methods. Group Iven (QCA and similar approaches) sits at the center of the network. This makes sense because it's grounded in a research philosophy that goes beyond the quantitative-qualitative binary (Breslin et al., 2020; Nag et al., 2007). In practice, QCA and approaches like it, such as realist evaluation, tracing the process of something happening, and fuzzy-set analysis, could help researchers move from one way of thinking about research to another, without the confusing theoretical problems that often arise with very simple mixed methods.

4.4 Ethical challenges: evidence from systematic content analysis

Table 3 brings together the main ethical and epistemological questions about knowledge that this study identified through a careful review of the full texts of selected papers. This study has grouped these issues by problem type, the research methods they affect, their frequency, and suggested solutions. Loss of privacy when collecting digital data was the most frequently mentioned ethical problem (in 72% of the papers where these issues were discussed), followed by bias in algorithms (67%), and then a lack of truly informed consent in studies that use experience sampling or constantly track people using digital tools (58%). What we found supports and expands on what (Kellogg et al. 2020) and (Orlikowski and Scott 2021) have said in theory and now provides numbers showing how common these ethical worries are. Until now, they have largely been addressed by stating what should be done, rather than by providing evidence of how often they occur. Table 3 outlines the major ethical and epistemological challenges associated with advanced organizational psychology methods, as well as the mitigation strategies proposed in the literature.

Table 3. Ethical and epistemological challenges associated with advanced methods: prevalence and proposed mitigations (systematic content analysis of full-text subsample, N = 420).

Prevalence (%) is the proportion of papers using the affected method that reported or discussed the corresponding challenge. Mitigation strategies are those explicitly proposed in the sampled literature. SHAP, SHapley Additive exPlanations; LIME, Local Interpretable Model-Agnostic Explanations; IRB, Institutional Review Board.

This study needs to discuss how ethics are applied in the study of organizations. The fact that 72% of studies using digital information traces focus on privacy does not mean 72% of them address those privacy issues properly. In fact, a review of digital ethnography papers found that only 31.4% explicitly described how they removed identifying details from the data, 24.7% detailed how they obtained different levels of agreement from participants, and under 15% considered whether data from an organization's online platforms, where bosses have much more power than employees, could really be said to be something someone had freely agreed to share. This lack of ethics is not just about breaking rules; it reflects a more fundamental issue with how we know things. Research on how people act at work when using digital tools exists within a confusing patchwork of rules, spanning academic ethics guidelines, employment law, data protection rules (especially GDPR in Europe), and the terms of service for the platforms themselves. This is an area without clear leadership, and our current ethical approaches were not designed to address it (Fisher et al., 2016; Johns, 2006).

The balance between how easy a result is to understand and its complexity (as shown in Table 3, occurring 61% of the time) is important to the theories we are developing. Organizational psychology claims to be useful because it can explain employee behavior, not just guess what will happen, and can say why it happens. It needs to describe the psychological and social mechanisms involved, as well as the specific limits on what counts as true (Staw, 1986; van Knippenberg and Sitkin, 2013). Black-box machine learning models, however good they are at making predictions, cannot provide this explanation. The point is not that complexity is bad, but that it should be used thoughtfully. If researchers use gradient boosting or deep learning but do not explain how the model's predictions connect to the concepts in their theory, they are not really developing organizational theory. Instead, they are doing organizational data science, which lacks the tools to properly interpret the data and to create useful advice for those working in the field or general ideas for theorists (Leavitt et al., 2021; Saraf et al., 2013).

4.5 Toward a Methodological Fit Matrix

Based on the bibliometric and systematic content analyses reported above, the Methodological Fit Matrix was constructed to align three dimensions largely treated in isolation in the existing literature: research orientation (exploratory, confirmatory, or integrative), temporal focus (cross-sectional or longitudinal), and epistemological stance (interpretivist, post-positivist, or pragmatist). Each cell of the matrix specifies recommended methods, appropriate data structures, primary analytical strengths, and principal limitations. The matrix operationalizes methodological selection across six distinct configurations, as presented in Table 4 below. Face validity for the matrix was established through two complementary procedures grounded in this study's empirical data. First, each cell was systematically anchored to the adoption patterns and citation evidence reported in Sections 4.1 and 4.2. The Exploratory–Longitudinal cell (recommending qualitative longitudinal research and NLP–survey sequences) is directly supported by the 229% growth in digital ethnography and the 296% growth in NLP applications documented in Table 2, confirming that the research community has converged on these methods for process-oriented inquiry.

Table 4. Methodological fit matrix for method selection in organizational psychology research.

The Methodological Fit Matrix was developed from findings of bibliometric and systematic content analyses of 14,617 publications (2020–2025). Each cell is anchored in empirically observed adoption trajectories, citation patterns, and co-keyword cluster analyses reported in Sections 4.1–4.3. Face validity was confirmed by mapping cells against the five VOSviewer thematic clusters and by independent reviewer cross-checking (Cohen's κ = 0.81). DSEM, Dynamic Structural Equation Modeling; ESM, Experience Sampling Method; SAOM, Stochastic Actor-Oriented Model; SEM, Structural Equation Modeling; CFA, Confirmatory Factor Analysis; QCA, Qualitative Comparative Analysis; ERGM, Exponential Random Graph Model; NLP, Natural Language Processing; QLR, Qualitative Longitudinal Research; ABM, Agent-Based Modeling; SNA, Social Network Analysis.

The Confirmatory–Longitudinal cell (recommending DSEM and latent growth curve modeling) is anchored to the 186% growth trajectory for Dynamic SEM/ESM and to the bibliometric cluster centered on within-person variation identified in Section 4.3. The Integrative cells are grounded in the descriptive finding that mixed-methods papers had a higher corpus-level mean citation count (47.3) than quantitative-only (31.6) or qualitative-only (19.4) papers, a pattern consistent with comparable bibliometric literature, though not subjected to inferential statistical testing in this study. Second, the matrix dimensions were cross-validated against the five co-keyword thematic clusters identified through VOSviewer network analysis (Section 4.3): each cluster maps onto a distinct matrix cell, confirming that the framework's structure reflects actual patterns of methodological co-occurrence in the literature rather than purely a priori reasoning. The correspondence between clusters and cells was verified by two independent reviewers using the same coding instrument applied to the 420-article content analysis subsample (Cohen's κ = 0.81), providing evidence of face validity for the framework's categorical boundaries. The matrix does not privilege any one epistemological tradition; it directs researchers toward methodological choices appropriate to the research question, the phenomenon's temporal structure, and the intended analytical outcome.

Table 4 presents the Methodological Fit Matrix, developed from the findings of the bibliometric and systematic content analyses. The matrix operationalizes methodological selection across seven dimensions: research orientation, temporal focus, epistemological stance, recommended methods, data structure, primary strength, and primary limitation. Each cell is grounded in the empirical adoption and citation patterns identified in this study. Face validity was established by anchoring each cell to the literature's observed co-occurrence clusters and confirmed by consensus among independent reviewers (κ = 0.81).

This viewpoint directly affects how doctoral students are taught, how journals review work, and how research is funded. Organizational psychology doctoral programs that teach students only number-based (quantitative) or in-depth interview/observation (qualitative) methods are not preparing them to meet the methodological challenges of modern organizations (Bandalos, 2021; Rossman and Rallis, 2022). Journal editors, by the way they choose reviewers, the methodological standards they demand, or the journals' ranking of different approaches, tend to favor certain methods. This encourages researchers to follow trends rather than choose the best method. And when funding bodies give money to research simply because a new technique is used, rather than whether it is appropriate, they risk paying for work that is technically very good but does not tell us anything useful about how organizations work (Briner and Denyer, 2023; Edmondson, 2023).

5 Conclusion

By reviewing over 14,617 published papers and providing both a detailed overview and careful analysis of their contents, organizational psychology is improving its research methods. Between 2020 and 2024, publications using new methods increased by 79.8%, applications of Natural Language Processing (NLP) increased by 296%, and digital ethnography research increased by 229%—all suggesting the field is now using methods of gathering evidence that match the complex nature of organizations in the 21st century. Mixed-methods research (descriptive mean: 47.3 citations per paper) showed a higher corpus-level citation count than research that is only quantitative (31.6) or qualitative (19.4); these are descriptive observations from the full corpus and have not been subjected to inferential testing. Citation-based interpretations were moderated because citation frequency may reflect journal visibility, interdisciplinary reach, publication timing, and disciplinary citation cultures rather than intrinsic scientific quality. Additional controls, including publication year, journal impact profile, document type, and disciplinary visibility, were considered conceptually when interpreting citation patterns.

However, the study's key findings give us a more cautious outlook on this methodological improvement. A tendency to adopt new labels for methods without changing how we think about knowledge or being careful with our analysis, or “methodological fashionism,” was found in 43.2% of Machine Learning/NLP papers and 38.7% of reflexive thematic analysis papers. Ethical problems in research using data from digital sources were widespread; only 31.4% of digital ethnography studies properly anonymized participants, and fewer than 15% fully considered whether people truly consented to participate in the research when there was a power imbalance in their employment. The way the research is grouped into separate quantitative/dynamic, computational/cultural and qualitative/processual areas shows that the problem of combining methods in organizational methodology is not just about technical difficulties but is a fundamental structural issue.

The main point of this research is that methodological innovation in organizational psychology must be guided by “epistemological fitness.” This means choosing a method because it suits the research question, our beliefs about how things are, and the specifics of the situation, rather than because it is the latest trend or because the tools are available. The “Methodological Fit Matrix” is proposed to put this principle into practice, with face validity established through systematic anchoring to this study's empirical patterns and consensus among independent reviewers. Organizational psychology's special place in science, its view that people at work are not simply numbers in a pattern but are shaped by their meanings, power, culture and the time in which they exist, doesn't need to be defended by sticking to old methods, but by “methodological wisdom.” That is, we need to be able to critically select, use, assess and combine methods that are sensible in terms of how we think about knowledge, rigorous in their research, ethically sound, and relevant to organizations.

6 Study limitations

Because of several issues, the findings of this study may not hold in all situations or across the field. To begin with, the analysis of publication frequency was conducted only for publications indexed in Web of Science, Scopus, and PsycINFO. Although these databases are comprehensive, they tend to favor research in English, published in influential journals, and by large universities or commercial publishers (Chadegani et al., 2013). This study missed work from region-specific journals, practitioner publications, material in languages other than English, and unpublished work (such as reports from consultants, the government, and NGOs). This could mean we did not get a full picture of new ways of doing things, especially from the Global South, from research conducted in real-world settings, and from research geared toward practitioners.

Also, this study employed “methodological innovation” by searching for specific words, then reading the content and deciding how to classify it, which inevitably introduces measurement errors. A method appearing in a publication's keywords does not prove it was used correctly, and inventive uses of a method may not be labeled with the usual keywords that a computer search would find. The number of times something is cited is not a perfect indicator of how good a method is or how widely it is used in practice. Citations might happen because of disagreement, criticism, or the way different disciplines operate, as much as because people are using and building on the new method. Also, because it takes 18–24 months for many citations to appear, publications from 2024 and 2025 will have fewer citations than they should, so older innovations will look more important than newer ones. Finally, because the content analysis part of the study only examined material from a single point in time, it is not possible to say how the quality of these methods has changed over the years. A future study that tracks trends over time should address this.

A further limitation concerns the handling of incomplete 2025 data in the growth rate calculations reported in Table 2. Because the database search was conducted in January 2026, only January–June 2025 publication data were available. Full-year 2025 figures were estimated by doubling the observed first-half frequencies, a linear annualization procedure that assumes publications are uniformly distributed across months. This assumption is unlikely to hold precisely: academic publishing exhibits seasonal variation, database indexing operates with a 1–3-month lag, and newly adopted methods such as NLP and digital ethnography may follow accelerating rather than linear growth curves. As discussed in Section 4.2, all three of these factors operate in the same direction and suggest that the reported growth percentages are conservative lower-bound estimates of true adoption trajectories. The absolute values of the reported growth rates (e.g., 296% for NLP, 229% for digital ethnography) should therefore be interpreted with appropriate caution, and future bibliometric studies that cover the complete 2025 dataset will be able to verify and refine these estimates.

7 Recommendations

Because of what this study found, here's what's recommended for different groups in organizational psychology.

Researchers and those running doctoral programs should rework how they train researchers. The goal is to develop people who can carefully judge and use both quantitative and qualitative methods (understanding the philosophy behind each), rather than being highly skilled in only one. Doctoral courses need to include a solid grounding in DSEM, social network analysis, thoughtful qualitative design, and ethical digital research (Bandalos, 2021; Braun and Clarke, 2021). If researchers are using Machine Learning or Natural Language Processing, they must show how their results can be explained (using tools like SHAP values) and how the model's results relate to the ideas they are testing (Leavitt et al., 2021).

Journal editors and those who review articles for them should actively oppose the practice of choosing a method simply because it's new or trendy. Instead, they should judge submissions on whether the method suits the research question. When reviewing mixed-methods research, the review should focus on how well the quantitative and qualitative components work together, not merely on their presence (Denzin and Lincoln, 2022). For all research that uses real data, openly sharing your study timeline, making your data analysis code public, and sharing the data itself (while protecting people's privacy) should be encouraged, or even required (Pratt et al., 2020).

Research ethics committees and those within organizations who control access to research need to develop specific guidelines for conducting research using digital traces. These guidelines should address issues such as obtaining valid consent when an employer has more power than an employee, collecting only the necessary data in passive sensing, and checking algorithms for fairness (Fisher et al., 2016; Kellogg et al., 2020). Organizations allowing researchers to use platform data for organizational psychology research should have a clear plan for managing the data in line with the GDPR and an independent ethical review that is not swayed by the organization's priorities.

Finally, organizations that fund research should actively encourage ambitious methodological approaches that prioritize the best method for the question, rather than novelty. When deciding which grants to award, they should consider how well the method aligns with the research aims, the ethics of the data plan, and how the findings will be implemented within organizations (Briner and Denyer, 2023; Edmondson, 2023). Investing in infrastructure for methodological training, advanced workshops, collaborative groups of researchers from different institutions to learn together, and freely available software for analyzing organizational data would make it much easier for researchers in places with limited funding to adopt newer methods.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Charles A. Pierce, Oakland University, United States

Reviewed by: Marcos Antonio Negreiros Dias, Independent Researcher, Gurupi, Brazil

Ricardo Santana De Almeida, União Dinâmica de Faculdades Cataratas, Brazil