Testing the predictive validity of configurations of variables: criterion profile analysis in organizational psychology

1 Introduction

Individual differences in behavior, cognition, and affect have traditionally been captured by examining how people differ from one another in how they behave, think, or feel on average. In personnel selection, for instance, candidates are typically ranked on the basis of their overall score on a predictor, for example, how dominant they are on average, how conscientious, how skilled. This approach has proven valuable, and the predictive validity of such mean-level scores is well-established across a wide range of organizational outcomes (e.g., Barrick and Mount, 1991; Barrick et al., 2001; Judge et al., 2013). However, a pure focus on inter-individual differences in average tendencies neglects the possibility that the same person may behave, feel, or think differently across situations and that these intraindividual fluctuations may themselves carry meaningful predictive information.

The importance of considering such within-person variation has a long theoretical history. In a seminal contribution, Cronbach and Gleser (1953) proposed decomposing an individual's profile of scores into three distinct elements: elevation, scatter, and shape. Elevation refers to the overall height of the profile, that is, the mean score across data points, capturing a person's general standing. Scatter refers to the degree of variability across the profile, that is, how much a person's scores fluctuate around their own mean. Shape refers to the unique configuration of relative highs and lows across the profile, that is, the specific pattern of scores that characterizes an individual, independent of their overall level or degree of variability. Together, these three elements provide a more complete description of a person's profile than any single component alone.

Despite the conceptual richness of Cronbach and Gleser's (1953) framework, research and practice in organizational psychology have not treated all three elements equally. Historically, the field has devoted most of its attention to elevation or average scores. For instance, assessment centers, despite obtaining information on a number of assessment center simulations, evaluate candidates primarily on the basis of their aggregated dimension scores across those simulations (e.g., Kleinmann and Ingold, 2019; Sackett et al., 2017). Similarly, situational judgment tests derive their predictive validity largely from mean-level performance across items (e.g., Christian et al., 2010; McDaniel et al., 2007).

More recently, however, scatter has begun to attract empirical attention. For example, Lievens et al. (2018) demonstrated that intraindividual variability in candidates' responses across situational judgment test items—that is, the degree to which their scores fluctuate across situations—carries predictive validity above and beyond their mean-level scores when predicting real-life behavior. Similarly, Vossen and Hofmans (2021) showed that within-person variability in personality states across occasions predicts counterproductive work behavior (but not organizational citizenship behavior) over and above trait-level scores. These findings suggest that knowing how variable someone is across situations or occasions adds meaningful information beyond knowing their average tendency.

Shape, however, has received far less attention. Whereas, scatter captures the amount of variability in a profile, shape captures its configuration, that is, the specific pattern that characterizes a person across a set of variables or situations. For example, two individuals can have identical elevation and the same amount of scatter yet display entirely different shapes: As Mischel and Shoda (1995) emphasized in their theory of behavioral signatures, this patterning of behavior across situations constitutes a meaningful and stable signature of the individual. Yet, shape has remained almost unexplored as a predictor of organizational outcomes (for an exception, see Wiernik, 2016; Leyens et al., 2026; or Wilmot et al., 2026).

At the same time, there has been a broader and growing recognition in organizational psychology that patterns and configurations matter. Reviews of person-centered research methods (Morin et al., 2018; Hofmans et al., 2020; Hofmans and Morin, 2024; Woo et al., 2024) document an increasing interest in approaches that capture how variables combine within individuals, rather than examining variables in isolation. The dominant used statistical methods for these patterns are cluster analysis or latent profile analysis (LPA). These approaches identify groups of individuals who share similar profiles, reducing the many different patterns present in the data to a smaller number of prototypical configurations. This is genuinely useful for describing the heterogeneity in a dataset and for identifying naturally occurring profile types. Yet, cluster analysis and LPA are designed to answer a different kind of research question than CPA. Their goal is to identify naturally occurring profile types who share similar configurations of scores, and they do so without reference to any outcome variable. This makes them well-suited to questions about what configurations exist in the data. When the research question concerns prediction, however, this becomes a limitation: because the outcome plays no role in the clustering process itself, the clusters or latent profiles that emerge are optimized for capturing within-profile homogeneity and between-profile heterogeneity in the predictor space, not for differentiating individuals on the outcome. At the same time, it is worth noting that extensions such as mixture regression analysis (MRA) and hybrid models that combine mixture modeling with LPA do incorporate the outcome directly into the modeling process. These approaches identify latent subgroups not only based on their profiles, but also on differences in how those profiles relate to an outcome, thereby offering a criterion-oriented alternative within a person-centered framework.

Other approaches have also tried to get at the idea that configurations matter. Polynomial regression, for instance, has been used to test whether the fit or congruence between two profiles—for instance, the match between a person's skills and their job demands—predicts an outcome (e.g., Edwards, 1994). Profile similarity indices (e.g., Cronbach and Gleser, 1953) capture the degree to which an individual's profile resembles a target profile or another person's profile. SEM-based approaches, meanwhile, allow predictors to be modeled at the latent variable level, but do not inherently decompose predictive variance into level and pattern components. As such, none of these approaches directly tests whether the profile of scores, in and of itself, carries predictive validity for a criterion. Instead, polynomial regression is fundamentally about the fit between two separate profiles rather than the predictive validity of a single profile; profile similarity indices quantify resemblance but do not decompose what drives prediction, and standard SEM approaches do not separate the contribution of overall profile elevation from the contribution of the specific configuration of scores. CPA fills this gap by asking: does the specific configuration of scores in a profile predict an outcome, above and beyond the overall level of those scores?

Testing the predictive validity of a pattern of scores requires an approach in which the outcome is an integral part of the analysis from the very start. Criterion-Related Profile Analysis (CPA; Davison and Davenport, 2002; Davison et al., 2023) is a regression-based technique that does this by decomposing the predictive variance in a set of predictor scores into two components: a level effect, capturing how much prediction is due to the overall elevation of a person's profile, and a pattern effect, capturing how much prediction is due to the specific configuration of relative highs and lows. By doing so, CPA enables direct hypothesis testing of whether it is the overall level of a set of predictors or their configuration (or both) that drives prediction of a given outcome. This decomposition maps directly onto Cronbach and Gleser's (1953) seminal work: the level component corresponds to elevation, and the pattern component captures shape. The third element of their framework—scatter, reflecting the degree of variability across a person's profile—is not decomposed as an independent component in standard CPA, but can be incorporated into the regression model as a control variable (see R code and Mplus code in Supplemental materials). Doing so allows researchers to ask whether the specific configuration of scores predicts the criterion above and beyond both the overall level and the degree of variability in the profile, providing a more complete test of all three profile components (see Leyens et al., 2026).

Despite this unique and useful positioning, CPA remains largely unknown in organizational psychology. The present paper aims to address this gap by providing an accessible introduction to CPA and its extensions, illustrated with a working example using publicly available data, and situating the method within organizational research contexts. For example, in multi-situation assessment methods such as assessment centers (ACs), situational judgment tests (SJTs), and multiple, speeded assessments (MSAs; Herde and Lievens, 2020, 2023) candidates are evaluated across a range of standardized situations, yielding a profile of scores that can be decomposed into level and pattern components. CPA is ideally suited to test whether it is a candidate's overall performance level or their specific pattern of performance across situations (or both) that predicts relevant criteria. Beyond assessment, CPA can be applied wherever a meaningful set of related variables yields a profile: personality profiles, interest profiles, and emotion profiles in relation to workplace outcomes come to mind, to name a few. The approach is most informative in settings where both level and pattern are expected to carry meaningful and potentially distinct predictive information—that is, where the pattern of the profile, and not merely its overall elevation, is theoretically expected to matter.

Importantly, for CPA to yield a meaningful level effect, the predictors that make up the profile should come from a conceptually coherent domain—that is, they should be sufficiently related that averaging across them is theoretically meaningful. This is either the case when the predictors are distinct subdimensions of a single broader construct (e.g., the three components of organizational commitment as used in this paper) or when the same construct is assessed across a standardized set of situations or exercises (e.g., in assessment center exercises). In both cases, the profile level captures something theoretically meaningful—overall commitment level, or overall performance level—and the pattern captures how an individual's scores are configured around that average. That said, CPA has also been applied to broader predictor set. For instance, CPA has been applied across the five factors of personality (Wiernik, 2016; Wilmot et al., 2025), where level effects reflect a kind of general dispositional elevation and pattern effects capture the specific configuration of Big five traits.

In what follows, we first describe the ideas behind CPA and its implementation, before illustrating the method with a step-by-step example. We then discuss data considerations. Next, we review extensions of the original CPA framework, including meta-analytic CPA (Wiernik et al., 2021; Wilmot et al., 2025), moderated CPA (Davison et al., 2023), latent variable CPA (Davison et al., 2014a), non-linear CPA (Davison et al., 2014b; Booth et al., 2016), and an extension of CPA to canonical regression (Aragon et al., 2014). We conclude by discussing the contributions of CPA to organizational psychology and directions for future research.

2 Criterion profile analysis

Criterion Profile Analysis (CPA) is designed to test the predictive validity of a profile of scores. Specifically, it tests whether the specific pattern of scores across a set of predictor variables matters for predicting an outcome above and beyond the average score across those predictor variables. To answer this question, CPA builds on an important insight from Davison and Davenport (2002): that any standard multiple regression equation can be mathematically rewritten—or reparametrized—in terms of two variables: one capturing the overall level of a person's predictor profile (i.e., the profile level, see Figure 1C) and one capturing how closely their pattern of scores (i.e., their profile pattern, see Figure 1B) resembles the pattern that is optimally predictive of the outcome (i.e., the criterion pattern, see Figure 1A). This reparameterization does not change the model's predictive power (i.e., it does not change the overall R² of the model), but it reframes the regression in a way that allows level and pattern contributions to be directly compared and tested.

Consider a simple example, which is shown in Figure 2. Imagine two employees, Nina and Tina, who both score an average of four out of seven on the three dimensions of organizational commitment (OC; Meyer and Allen, 1991): affective commitment (AC; emotional attachment to the organization), continuance commitment (CC; perceived cost of leaving), and normative commitment (NC; sense of obligation to stay). Nina scores two on affective, three on continuance, and six on normative commitment. Tina scores five on affective, two on continuance, and four on normative commitment. Their profile levels—the average of their three scores—are identical. Their profile patterns, however, look very different. A variable-centered approach that relies only on the mean commitment score would treat Nina and Tina as equivalent in terms of their (overall) commitment. CPA, by contrast, asks whether the specific pattern of commitment scores is predictive of an interest criterion.

2.1 A working example: organizational commitment predicting OCB

In this paper, we illustrate CPA using a publicly available dataset from Phuong (2023), who collected data on organizational commitment and organizational citizenship behavior in the water supply industry among Vietnamese state-owned enterprises. The dataset includes measures of affective commitment (AC), continuance commitment (CC), and normative commitment (NC) as predictors (constituting to a predictor set of three variables), and organizational citizenship behavior (OCB) as the criterion variable¹. This example is well-suited to illustrate CPA for several reasons. First, the three commitment types create a theoretically meaningful profile: they share a common focus on the employee-organization relationship but capture distinct dimensions of that relationship (Meyer and Allen, 1991). Second, while single commitment dimensions have been shown to be related to OCB before (Bakhshi et al., 2011; Grego-Planer, 2019), the overall commitment level in relation to OCB and potential pattern effects remain unexplored. Third, the data are freely available, ensuring full reproducibility. The R code used to conduct all analyses reported in this paper is available in the Supplemental materials.

One caveat worth noting is that both the predictors and the criterion in this example are measured via self-report, meaning that part of the observed associations may partly reflect shared method variance rather than true substantive relationships. This limitation is not specific to CPA; but applies to any regression-based analysis using single-source data, and readers should bear it in mind when interpreting the results, alongside a standalone Mplus implementation for researchers who prefer that software.

2.1.1 Reparameterizing the regression: level and pattern

CPA builds on the insight that any standard multiple regression equation can be reparametrized using two components, as shown in Equation 1: a level component and a pattern component. Consider a regression model with V predictor variables. In the working example, the standard regression equation predicting OCB for person p is:

OCB^p= bACXp,AC + bCCXp, CC + bNCXp, NC + a(1)

Where b_*AC*, b_*CC*, and b_*NC* are the unstandardized regression weights for affective, continuance, and normative commitment, respectively, and a is the intercept. Davison and Davenport (2002) show that this equation can be rewritten as:

OCB^p=∑v(bv-b̄)(Xpv-X̄p.)︸pattern component+vb̄X̄p.︸level component + a(2)

Where X¯p.= 13 (Xp,AC + Xp,CC + Xp,NC) is person p's average commitment score across the three dimensions, and b¯ = 13 (bAC+ bCC+ bNC) is the average regression weight.

The term on the right-hand side of Equation 2, vb¯ X¯p, is the level effect. The level effect depends only on X¯p., the person's overall commitment level. This component captures how strongly a person's average commitment (across AC, CC, and NC) predicts OCB. The term on the left-hand side of Equation 2, ∑v(bv b¯) (Xpv X¯p.), is the pattern component. This component reflects how the pattern of a person's commitment profile (being the relative highs and lows of AC, CC, and NC) aligns with the criterion-related pattern. The criterion-related pattern is represented by the vector:

bv* = bv -b.(3)

This centering step removes the influence of the overall magnitude of the regression weights, leaving only their relative differences, that is, the degree to which each predictor's weight deviates from the average weight across all predictors.

These b^*values are the regression weights expressed as deviations from the average regression weight. Because they are deviations, they always sum to zero—which means that the criterion pattern captures relative differences among predictors rather than their absolute importance. Interpretation focuses on the b^*values whose confidence intervals do not include zero, as these indicate which profile features meaningfully predict OCB. A positive b^*means that higher-than-average scores on that predictor (relative to the person's own mean) are associated with higher OCB. A negative b^*means that lower-than-average scores on that predictor are associated with higher OCB. For example: A positive bNC* means that relatively higher normative commitment predicts higher OCB, wheareas a. negative bAC* means that relatively lower affective commitment predicts higher OCB.

With three predictors, the pattern component has V − 1 = 2 degrees of freedom, corresponding to the two independent ways a three-variable profile can vary in configuration.

Importantly, the reparametrized model explains exactly the same amount of variance as the original regression model. CPA simply decomposes the prediction into level and pattern components; it does not increase predictive power.

2.1.2 Step-by-step implementation of CPA

We illustrate the CPA implementation step by step, using the Phuong (2023) dataset, which includes data from 336 employes (238 men and 98 women between 18 and 55 years old) who work in Vietnamese state-owned enterprises in the water supply industry. The R code for all steps is provided in the Supplemental materials, as is a standalone Mplus implementation covering the same analyses.

*Criterion pattern estimation*. The first step in CPA is to estimate the criterion pattern, which is defined as the “predictor-score arrangement associated with high scores on the criterion” (Davison and Davenport, 2002, p. 468). To identify this pattern, the criterion is regressed on the full set of predictor variables. This regression shows which parts of the predictor profile are most strongly associated with the outcome. The resulting regression coefficients are then converted into a criterion-related pattern by centering them—that is, by subtracting the average regression coefficient from each individual regression coefficient (see Equation 3). This produces a set of b^* values that express each predictor's contribution relative to the average contribution across all predictors. In some contexts, it is advisable to standardize the predictor variables before estimating the criterion pattern. In the present working example, standardization is not necessary because AC, CC, and NC are all measured on the same seven-point response scale within the same questionnaire, making their raw scores directly comparable.

In the working example, the criterion pattern (see Figure 3) is estimated by regressing OCB on AC, CC, and NC. The resulting regression coefficients b_*AC* = 0.036, b_*CC* = 0.226, and b_*NC* = 0.389 are then centered by subtracting the average regression coefficient b¯=0.217, yielding bAC* = 0.181, [95% CI (−0.284, −0.078)], bCC* = 0.009 [95% CI (−0.085, 0.103)], and bNC* = 0.172 [95% CI (0.059, 0.286)]. These b^* values constitute the criterion pattern: they describe the specific configuration of commitment scores most strongly associated with OCB, independent of overall commitment level. As can be seen in Figure 3, the resulting criterion pattern in the example rises from left to right, with affective commitment showing a negative b^*, continuance commitment close to zero, and normative commitment showing the largest positive value. Notably, the confidence interval for bCC* includes zero, indicating that continuance commitment does not contribute meaningfully to the OCB pattern prediction, whereas affective and normative commitment drive the pattern effect. The criterion pattern suggests that employees who score relatively higher on normative commitment and relatively lower on affective commitment, compared to their own average commitment score, tend to show higher OCB.

The criterion pattern regression also provides a benchmark R²: because CPA reparametrizes rather than extends the regression, the R² of the full CPA model will be identical to the R² obtained here [R²_Full = 0.425, adjusted R² = 0.420, F_{(3, 332)} = 81.92, and p < 0.001].

*Testing the profile level*. Profile level scores are calculated as the mean of a person's predictor scores across all V predictors. Profile level scores reflect the overall elevation of one's profile independent of how the scores are distributed across the specific predictor variables (see Figure 1B). Including this variable in the regression model allows testing how much of the criterion's variance is accounted for by overall average tendencies, independent of profile pattern.

In the working example, each person's profile level is computed as the mean of their AC, CC, and NC scores. The level-only model regressing OCB on this mean commitment score yielded RLevel2 = 0.403 (adjusted RLevel2 = 0.401), F_{(1, 334)} = 225.2, and p < 0.001, showing that overall commitment level is a strong predictor of OCB.

*Testing the profile pattern*. To test the predictive validity of the pattern effect, a set of pivoted predictors is created for each person. These pivoted scores represent how much each predictor deviates from a chosen reference predictor. In other words, they show whether a person scores relatively higher or lower on a given predictor compared to that reference. Taken together, these deviations capture the pattern of the person's profile while removing the influence of their overall level (see Figure 1C). The predictive validity of the pattern is then tested by entering all pivoted predictors simultaneously in a pattern-only regression model. The resulting regression coefficients indicate how the configuration of these predictor deviations (i.e., the profile pattern) relates to the criterion.

In the working example, AC serves as the reference predictor, yielding two pivoted predictors for each person²: CC–AC and NC–AC. With V = 3 predictors, this yields V – 1 = 2 pivoted predictors, consistent with the two degrees of freedom associated with the pattern component. The pattern-only model (which regressed OCB on CC–AC and NC–AC simultaneously) yielded R²_Pattern = 0.016 (adjusted R²_Pattern = 0.010), F_{(2, 333)} = 2.68, and p = 0.070, indicating that commitment profile pattern carries little predictive information.

*Predictive validity of level and pattern effects. In the final step, the criterion is regressed on both the profile level score and the pivoted predictors simultaneously. This allows us to determine how much the level and pattern component contribute to predicting the outcome. If the pattern effect remains a significant predictor after controlling for the level score, this indicates that it provides predicative information beyond simply how high or low someone scores on average. The unique contribution of each component is assessed by comparing the full model to each component-only model: ΔR²_Pattern is obtained by comparing the full model to the level-only model, and ΔR*²_Level is obtained by comparing the full model to the pattern-only model. These comparisons quantify how much additional variance each component explains beyond the other.

In the working example, the full CPA model yielded R²_Full = 0.425—identical to the baseline regression used to estimate the criterion pattern. This confirms that CPA decomposes the predictive variance rather than increasing it. Incremental F-tests showed that the pattern effect explained unique variance above and beyond level [ΔR²_Pattern = 0.023, F_{(2, 332)} = 6.55, and p = 0.002], and the level effect explained unique variance above and beyond pattern [ΔR²_Level = 0.410, F_{(1, 332)} = 236.61, and p < 0.001]. Results are summarized in Table 1. All of this suggests that both components contribute independently to the prediction of OCB, though the level effect dominates: overall commitment level accounts for the large majority of the predictive validity in OCB, whereas the pattern of commitment types adds a small but statistically significant increment. This means that OCB is predicted not only by how committed an employee is on average, but also—to a lesser degree—by the specific way in which their commitment is distributed across affective, continuance, and normative commitment. Employees whose commitment profile resembles the criterion pattern—relatively higher normative and lower affective commitment, independent of their overall commitment level—tend to show higher OCB than those with a different commitment pattern configuration.

Table 1. CPA results for predicting organizational citizenship behavior.

The dominance of the level effect over the pattern effect in this example is not surprising and should not be interpreted as evidence that pattern effects are unimportant. A useful analogy is the relationship between main effects and interaction effects in regression: just as main effects typically absorb the majority of explained variance, leaving interaction effects to account for a smaller but potentially meaningful increment, the level effect in CPA tends to dominance because it captures the single strongest source of individual differences (overall commitment in this case). The pattern effect, by contrast, captures something qualitatively different: the specific configuration of scores, independent of their overall elevation, and even a modest increment in explained variance can carry meaningful theoretical and practical implications. Pattern effects are most likely to emerge when the predictors that make up the profile themselves meaningfully related to the criterion. Whether individual predictors relate to the criterion in turn depends on their relevance for that particular outcome. Researchers should, therefore, consider the theoretical relevance of each predictor for the criterion of interest.

2.1.3 Extending CPA by incorporating scatter

As noted above, scatter—the degree of variability across a person's profile—can be incorporated into the CPA model as a control variable, allowing researchers to test whether the profile pattern predicts the criterion above and beyond both overall level and scatter. In practice, this is done by entering scatter as an additional predictor alongside the profile level score and the pivoted predictors (i.e., the pattern). Importantly, scatter is treated here as a control variable rather than a decomposed component: the goal is to isolate the unique contribution of pattern above and beyond both level and scatter, not to split variance into three independent components.

In the working example, each person's scatter score was computed as the standard deviation of their AC, CC, and NC scores. On its own, scatter was weakly but significantly associated with OCB [R²_Scatter = 0.013, F_{(1, 334)} = 4.28, and p = 0.039], suggesting that employees with more variable commitment profiles tend to show slightly lower OCB. However, scatter explained no unique variance above and beyond level [ΔR²_Scatter = 0.000, F_{(1, 333)} = 0.114, and p = 0.736], nor above and beyond level and pattern combined [ΔR²_Scatter = 0.001, F_{(1, 331)} = 0.194, and p = 0.660]. Critically, the pattern effect remained significant above and beyond both level and scatter [ΔR²_Pattern = 0.023, F_{(2, 331)} = 6.575, and p = 0.002], confirming that it is the specific pattern of commitment scores—and not merely how variable a person's commitment is across dimensions—that carries unique predictive information for OCB.

2.2 Statistical and conceptual considerations

When using CPA, researchers should pay attention to several practical issues, including sample size, the number of predictors in the model, missing data, multicollinearity, and the robustness of results. Attending to these issues will help ensure that the conclusions about level and pattern effects are robust.

*Power considerations*. Because CPA is regression-based, it shares the same power concerns as multiple regression: the sample must be large enough relative to the number of predictors. This is especially important for the pattern effect, which uses V−1 degrees of freedom (where V is the number of predictor variables in the profile). As the number of predictor variables increases, the pattern effect consumes more degrees of freedom, and larger samples are required for stable and well-powered estimates. Larger samples also improve the stability of the criterion pattern itself. Reporting adjusted R² alongside unadjusted R² helps readers judge whether results may be influenced by the number of predictors and sample size.

*Missing data*. CPA requires complete predictor profiles for each individual; it does not include built-in missing-data handling. Listwise deletion is the simplest approach but can substantially reduce sample size. This is particularly problematic when the profile contains many variables. Multiple imputation is a useful alternative when missing values are present, as it preserves sample size and yields unbiased estimates under standard assumptions. When the number of predictors is large relative to the sample, missing data handling becomes particularly important for obtaining a stable pattern estimate. If a predictor has substantial missingness and cannot be imputed in a theoretically defensible way, it may need to be excluded from the profile. Any such decisions should be reported transparently. Alternatively, when CPA is implemented within an SEM framework—as in the latent variable extension described below—full information maximum likelihood (FIML) estimation becomes available. FIML handles missing data without requiring imputation, and within the SEM formulation it can in principle accommodate missingness on both the predictors and the outcome. Researchers comfortable with SEM software such as Mplus may find this a practical and principled alternative to imputation. Indeed, Mplus makes it possible to estimate multiple regression models with a maximum likelihood-based estimator, so that FIML can be used for regression-based CPA. We provide sample input code to support these analyses in the Supplemental materials.

In the working example, no missing values were present, so neither deletion nor imputation was required.

*Multicollinearity*. As with any regression-based method, CPA is sensitive to multicollinearity among predictors. A potential misconception is that the decomposition of profiles into level (profile mean) and pattern (deviations from that mean) automatically produces orthogonal components, thereby eliminating multicollinearity. Although the deviation scores that define the pattern component sum to zero within individuals, this does not guarantee orthogonality at the sample level. Orthogonality concerns covariation across individuals: if individuals with higher profile levels also tend to exhibit particular configurations of scores, the level and pattern components will be correlated. As a result, the level and pattern effects in CPA are not constructed to be orthogonal and may show non-negligible correlations in practice (Davison and Davenport, 2002).

High intercorrelations among predictors inflate the standard errors of the regression coefficients and, in turn, the standard errors of the criterion pattern coefficients. This makes individual pattern elements unstable and harder to interpret. Researchers should inspect variance inflation factors (VIFs) before interpreting results. As a rule of thumb, VIFs below five are generally considered acceptable, but researchers should be aware that even moderate multicollinearity can affect the precision of individual pattern coefficients. Standard errors or confidence intervals should be reported alongside pattern coefficients to aid interpretation and allow readers to assess the precision of individual pattern estimates.

In the working example, VIFs ranged from 1.94 to 2.37—all well-below the recommended threshold, indicating that multicollinearity was not problematic.

*Cross-validation*. Cross-validation is recommended to assess the stability of the pattern effect. Pattern effects are more prone to overfitting than level effects, especially when the number of predictors is large relative to sample size. Beyond overfitting concerns, it is also important to recognize that the criterion pattern is empirically derived: it is estimated from the regression coefficients in a given sample, and like any sample-based estimate, it is subject to sampling variability. The specific configuration of b^*values that emerges in one sample may not perfectly replicate in another. Cross-validation provides a way to evaluate whether the pattern is a stable feature of the data or an artifact of the specific sample at hand. Two approaches are available. The most straightforward one is to use the pcv() function of the profileR package (Desjardins and Bulut, 2020), which implements the split-half procedure described by Davison and Davenport (2002): it randomly splits the sample, estimates the CPA model in each half, and reports R² values for the full model, pattern effect, and level effect for each half. Alternatively, researchers can manually split the sample and re-run the full CPA procedure in each half. While more effortful, this approach offers greater flexibility by providing access to the full set of estimated b^* values in each subsample. These can be compared directly across halves—both numerically and visually by plotting the criterion patterns side by side—to assess whether the same predictors emerge as relatively high or low in both subsamples. This convergence of the b^* values across subsamples is particularly informative: it provides a more direct assessment of whether the criterion pattern itself replicates, rather than only the variance it accounts for. Still, cross-validation only concerns stability within a single dataset. Because the criterion pattern is estimated rather than specified in advance, showing that it generalizes requires evidence from independent samples, contexts, and populations.

In the working example, we applied the split-half cross-validation procedure using the pcv() function of the profileR package. The full model R² replicated across both subsamples (R² = 0.420 and 0.439). The level effect was strong and consistent in both halves (R² = 0.406 and 0.410, both p < 0.001). The pattern effect was small in both subsamples (R² = 0.005 and 0.020), consistent with the modest pattern effect observed in the full sample. The incremental R² of pattern beyond level was significant in both subsamples (p = 0.042 and p = 0.004), showing that the ΔR²_Pattern replicates across independent subsamples.

*Selecting predictors for the profile*. CPA is not appropriate for every set of multiple predictors. For criterion patterns to be meaningful, the predictors must meet certain conceptual requirements.

First, the predictors must be administered in a standardized way across individuals so that patterns are comparable³. This means that every individual must respond to the same set of predictors—as is the case in multi-situation assessments or self-reports on standardized questionnaires, for example. In the working example, all employees completed the same three commitment scales (AC, CC, and NC) and self-reported on the same criterion variable, ensuring that each person's profile reflects responses to an identical set of items.

Second, not all multi-item instruments are suitable for CPA. The key distinction is between instruments designed to be aggregated for reliability and instruments designed to sample distinct situations or contexts. Personality questionnaires, for example, typically consist of many individual items, each of which carries some degree of measurement error. The aggregation of items into scale scores is precisely what makes personality measurement reliable. Using individual items as the predictors in a CPA profile would, therefore, be challenging for at least two reasons: individual item-level scores tend to be less reliable than scale composites, which may make pattern effects harder to detect, and the sheer number of predictors (i.e., the number of items in the test) would require sample sizes that are difficult to achieve in practice. In contrast, multi-situation assessments are well-suited for CPA because each predictor (i.e., AC or MSA simulations, or SJT item simulations) represents a meaningful, distinct context.

2.3 Extensions of CPA

The original CPA method developed by Davison and Davenport (2002) was designed for primary data with continuous predictors and one continuous criterion. Since its introduction, a number of extensions have been developed to accommodate more complex data structures and research questions that are common in organizational psychology. Given that the present paper's primary aim is to introduce the original CPA framework to an organizational psychology audience—a method that itself has received little attention in the field—a detailed treatment of each extension goes beyond its scope. We, therefore, briefly describe five extensions that have been well-developed and applied in prior research, with the goal of creating awareness and pointing interested readers to the relevant literature, and summarize their data requirements and primary areas of application on organizational psychology in Table 2. Note that this list is not exhaustive, and that we return at the end of this section to some promising directions for future methodological development.

Table 2. Overview of CPA variants and their application potential in organizational psychology.

2.3.1 Meta-analytic CPA

An important limitation of the original CPA method is that it requires individual-level data to compute profile level and pattern scores. This restricts its application to primary studies and neglects the large body of meta-analytic research that organizational psychology has to offer. Wiernik et al. (2021) addressed this limitation by developing meta-analytic CPA (MACPA), demonstrating that the key CPA statistics can be estimated from a correlation matrix alone, without access to individual observations.

A central advantage of MACPA is that it can be integrated with psychometric meta-analysis, to correct for the statistical bias that typically distort findings in primary studies. These measurement errors have a number of effects on CPA results: they make the peaks and valleys of criterion-patterns less pronounced and harder to detect; they weaken the estimated predictive validity of both profile level and pattern effects, and they cause relatively more explanatory power to be attributed to the level effect rather than the pattern effect (Wiernik et al., 2021). Correcting for measurement error at the meta-analytic stage, therefore, yields more accurate estimates of criterion patterns and their predicted validity than any single primary study could provide. Additionally, Wiernik et al. (2021) provide methods to address overfitting using statistical shrunken-R² estimators in place of the split-half cross-validation procedure that standard CPA suggests.

MACPA is most valuable when the goal is to examine whether a criterion pattern is consistent across an accumulated body of literature, rather than within a single sample. It is also well-suited to reanalyze existing meta-analytic correlation matrices that were originally compiled for other purposes, as it can reveal pattern effects that conventional variable-centered meta-analyses would not have detected.

2.3.2 Moderated CPA

Standard CPA identifies the general criterion-related pattern. In many organizational research contexts, however, researchers are interested in a subsequent question: is the criterion-related pattern the same across different groups or conditions? For example, does the configuration of commitment types that predicts OCB differ between men and women, or between employees at different organizational levels? If the criterion-related pattern varies across groups, this has important implications for the validity and fairness of profile-based selection or assessment approaches. In other words, a pattern that works well in one group may not generalize to another group.

Davison et al. (2023) developed moderated CPA to address exactly this issue. Three moderated models are proposed, each representing a different hypothesis about how groups may differ. In the fully moderated model, both the level effect and the pattern vector are allowed to vary across groups. In the level-moderated model, only the level effect differs across groups while the pattern vector is constrained to be equal (meaning that groups share the same criterion-related pattern but differ in how strongly overall profile level predicts the criterion). In the pattern-moderated model, only the pattern vector differs across groups while the level effect is constrained to be equal (meaning that groups differ in their criterion pattern, but not in the overall importance of profile level). These models are hierarchically nested, enabling researchers to determine not only whether the criterion-related pattern differs across groups, but also whether any observed difference is attributable to the pattern, the level, or both.

An additional contribution of Davison et al. (2023) worth noting is that their reformulation also corrects an underestimation problem in the original Davison and Davenport (2002) approach. In the original method, the pattern effect is estimated conditional on level, which can lead to the variance accounted for by pattern alone being somewhat deflated when there are more than two predictors. The reformulation by Davison et al. (2023) corrects this by estimating the pattern-only model directly, yielding a more accurate estimate of the variance accounted for by profile pattern.

2.3.3 Latent variable CPA

Standard CPA operates on observed scores (i.e., self-report scale composites, assessor ratings, and test scores), which means that measurement error can affect the results. This is particularly relevant when comparing level and pattern effects, because the two components are not equally affected by measurement error: profile level—being an average of positively correlated variables—tends to be more reliable than the pattern component, which is based on deviation scores that reflect relative differences among predictors. Because differencing positively correlated variables reduces reliability, pattern effects are particularly susceptible to underestimation (Davison et al., 2014a). Observed differences between level and pattern effects in standard CPA should, therefore, be interpreted with caution, as they may partly reflect this reliability imbalance rather than true differences in predictive validity.

Davison et al. (2014a) extended CPA to a structural equation modeling (SEM) framework to address this issue. Their idea is straightforward: rather than computing level and pattern scores from observed composites, the model estimates them at the latent variable level—that is, from the underlying true scores of the predictors, corrected for measurement error⁴. The interpretation remains the same as in standard CPA: the criterion pattern still describes the configuration of predictor scores most strongly associated with the criterion, and level and pattern effects are tested in the same way.

In practice, latent variable CPA requires multiple items per predictor (because a measurement model is needed for each construct) and larger samples than standard CPA. It is, therefore, most useful when each predictor is measured by multiple items, when the relative importance of level vs. pattern is a central research question, and when there is reason to believe that the reliability difference between level and pattern may be distorting the comparison—for instance, when the pattern effect appears small relative to level but theory predicts it should matter.

For researchers wishing to apply this extension, we provide sample Mplus input code for latent variable CPA in the Supplemental materials, covering the estimation of level and pattern effects as well as an illustration of how scatter can be incorporated.

2.3.4 Non-linear CPA for categorical outcomes

The original CPA assumes a continuous criterion variable. When the criterion is categorical—for example, whether a candidate is hired or not—the standard linear regression framework is inappropriate. Davison et al. (2014b) and Booth et al. (2016) addressed this by embedding the CPA level and pattern decomposition within a logistic or multinomial logistic regression framework. The criterion pattern is derived in the same way as in standard CPA (with the logistic regression coefficients centered on their mean) and its interpretation remains unchanged: it describes the configuration of predictor scores that is maximally predictive of the outcome. What changes is how the variance accounted for by level and pattern is expressed: rather than R², model fit is evaluated using pseudo-R² statistics which are appropriate for maximum likelihood estimation.

This extension broadens the applicability of CPA to organizational research contexts in which outcomes are categorical—such as hiring decisions, promotion, or group membership.

2.3.5 Canonical regression analysis

In many applied contexts, the criterion of interest is itself multidimensional—comprising several conceptually related outcomes that are better understood jointly than in isolation. Aragon et al. (2014) extended CPA to accommodate this structure by embedding the level and pattern decomposition within a canonical correlation analysis, in which both the predictor and criterion sides of the model are represented by sets of variables rather than a single outcome.

The interpretation of the criterion pattern, again, remains consistent with standard CPA: the canonical variates describe configurations of predictor scores that are maximally associated with configurations of criterion scores. As such, this extension also allows researchers to detect multiple distinct criterion-related patterns within the same sample—each capturing a different dimension of the relationship between the predictor and criterion sets.

This extension is most appropriate when the criterion is inherently multivariate and when collapsing it into a single composite would lead to a meaningful loss of information. Additionally, it is best suited to exploratory research contexts where the goal is to characterize the full structure of predictor–criterion relationships rather than to test a specific pattern hypothesis.

2.3.6 Further developments

The extensions described above provide an established methodological foundation for organizational researchers to build on. At the same time, CPA as a method remains relatively underutilized in organizational psychology, and there is considerable room for further development.

One promising avenue concerns longitudinal extensions. The random intercept cross-lagged panel model (RI-CLPM; Hamaker et al., 2015), for instance, separates stable between-person differences from time-varying within-person fluctuations, a distinction that resembles CPA's decomposition into level and pattern components. However, this correspondence is only partial: whereas CPA decomposes the cross-sectional configuration of variables within a profile (i.e., differences across indicators at a given moment), the RI-CLPM decomposes temporal dynamics within the same variable across occasions. A fruitful direction for future research would be to integrate these perspectives by modeling both time-invariant differences in profile level and time-varying profile patterns across repeated measurements. Such an approach would allow researchers to examine how both stable levels and dynamic patterns jointly predict outcomes.

3 Discussion

The present paper introduced criterion profile analysis (CPA) to an organizational psychology audience. The main methodological contribution is that CPA enables direct hypothesis testing of whether a profile of predictor scores—not just its overall elevation—predicts a criterion of interest. By decomposing predictive variance into a level effect and a pattern effect, CPA makes it possible to test whether the average scores across a set of predictors (i.e., level), how their scores are configured (i.e., pattern), or both, drive prediction of a given outcome. This is a qualitatively different question from what mean-level comparisons can answer, and one that has been largely neglected by organizational researchers.

The working example illustrates this well. In the dataset of Phuong (2023), both level and pattern on organizational commitment dimensions contributed independently to the prediction of OCB. While overall commitment level dominated the prediction of OCB, the specific configuration of commitment types added a small, replicable increment. This means that organizational citizenship behavior is predicted not only by how committed an employee is on average, but also—to a lesser degree—how their commitment scores configuration across affective, continuance, and normative commitment looks like.

3.1 Future directions

At its core, CPA provides a way to assess the predictive value of individual differences in person-situation interactions—that is, how different people respond to the same standardized situations. This framing connects personality psychology and organizational behavior. Advancing our understanding of which situational features elicit the expression of personal characteristics, and which configurations of person (and situation) features predict important outcomes, is a central challenge for the field and CPA provides a methodological tool for making progress on exactly this question.

A related and promising direction concerns the potential of CPA to test theoretical propositions more explicitly, rather than simply describing criterion-related patterns. Although the criterion pattern is derived empirically—estimated from the data rather than specified in advance—this does not mean CPA is inherently exploratory. At the broadest level, theories that predict that behavioral configuration matter beyond overall tendencies generate directly testable predictions about whether a significant pattern effect should emerge. The CAPS model (Mischel and Shoda, 1995), for instance, holds that stable, person-specific patterns of behavior across situations constitute so-called behavioral signatures of the individual, which carry predictive information above and beyond average behavioral tendencies. At a more specific level, theory may also generate expectations about the configuration of the criterion pattern itself: When predictors represent different facets of a broader construct or different situational contexts, theory may suggest that certain predictors are more strongly associated with a given outcome than others, researchers may have directional hypotheses about which predictors are most strongly associated with a given outcome. In both cases, CPA provides the tools to put those expectations to an empirical test.

A particularly promising avenue for future research concerns the link between CPA and the recent development toward a more dynamic approach to personality. If profiles reliably predict outcomes above and beyond overall levels, personality might be understood as a dynamic system of situation-contingent responses than as a set of stable mean tendencies. CPA allows us to disentangle pure “person effects (i.e., level effects) from “person x situation” effects (i.e., pattern effects). In line with earlier calls to move beyond the mean-level focus that has dominated the field for the past decades, future research should capitalize on assessment methods that reveal patterns of behavioral variability and go beyond what mean-level scores can capture. More broadly, the field needs more empirical work on behavioral variability across situations, examining when and why people respond differently to the same situational demands, and what those differences tell us about individual functioning. A particularly important open question in this regard is what makes certain patterns more adaptive than others: understanding the conditions under which a specific configuration of situational responses translates into effective performance, wellbeing, or other valued outcomes remains a central challenge for future research.

CPA also provides new avenues beyond pure prediction. Knowledge about the optimally predictive pattern may contribute to our understanding and our operationalization of adaptivity. Specifically, adaptive behavior is not the same as behavioral variability—an individual may show high cross-situational variability yet low adaptivity if their behavioral fluctuations do not align with situational demands. CPA provides a way to formally test this distinction by allowing researchers to assess the degree to which an individual's behavioral pattern matches the criterion pattern, that is, the score configuration that is most effective for a given outcome. Since the criterion pattern is not defined a priori but inferred from the data, adaptivity in this framework is criterion-specific: the same behavioral configuration may be adaptive for one outcome and maladaptive for another.

In an applied setting, the identification of criterion patterns opens the door to targeted interventions where feedback and development efforts can be directed not at overall tendencies but at specific situational contexts in which an individual's behavioral pattern diverges from what is most effective. Specifically, in recruitment contexts, individual behavioral profiles could be matched against the criterion pattern identified for a given role. In training contexts, knowledge of the criterion pattern allows feedback to be targeted at the specific situations in which an individual's behavior diverges most from the optimal configuration. At the organizational level, criterion patterns themselves become an object of inquiry: do they remain stable over time, or do they shift as organizational demands change? And do the patterns that empirically predict performance align with what the organization would theoretically have expected, or do they reveal something surprising about what actually drives performance in a given context?

Realizing these practical applications requires building a sufficiently large body of evidence in support of the pattern hypothesis first. But for the field to move from prediction to intervention we need cumulative evidence across contexts, criteria, and populations establishing that patterns matter consistently and reliably. Building that evidence base should be a worthwhile investment organizational psychology can make in the coming years.

Taken together, CPA and its extensions represent an excellent toolkit for testing whether patterns matter and carry predictive validity beyond level scores. We encourage organizational researchers to look beyond average scores and consider applying CPA wherever a theoretically motivated profile exists.

3.2 Reproducibility and open materials

In the interest of transparency and reproducibility, all analyses reported in this paper were conducted in R. A fully annotated script implementing each step of the CPA procedure described in this paper—including criterion pattern estimation, level and pattern testing, as well as cross-validation—is available in the Supplemental materials. For researchers who prefer Mplus, we additionally provide a standalone Mplus implementation covering the same analyses, using the ML estimator which relies on FIML and can, therefore, accommodate missing data without requiring imputation. We hope these Supplemental materials lower the practical barrier to applying CPA and encourage researchers to examine profile-based predictions in their own data.

Generative AI statement

The author(s) declared that Generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/forgp.2026.1861747/full#supplementary-material

Supplementary material Supplementary material at the source

Supplementary material Supplementary material at the source

Supplementary material Supplementary material at the source

Footnotes

Edited by: Alexandre J. S. Morin, Concordia University, Canada

Reviewed by: Julien Morizot, Montreal University, Canada

Herb Marsh, Australian Catholic University, Australia

Footnotes

For demonstration purposes, we only use three predictor variables and one outcome variable from the full dataset. For a complete overview of the used items and dataset one can consult Phuong (2023). A more detailed description of the research can be found in Thao et al. (2024).

The pivoted predictors (e.g., CC–AC, NC–AC) are algebraically equivalent to the deviation scores (Xpv X¯p.) used to derive the criterion pattern: both have the same V−1 degrees of freedom of profile pattern and yield identical model fit, R², and F-tests. For manual regression-based implementations, pivoted predictors are the standard approach in the CPA literature.

Note that, while CPA empirically tests the predictive validity of pattern effects, it does not offer explanations or interpretations of what causes the criterion pattern shape. Interpreting situational features that might cause certain behaviors to be more predictive than others is a qualitative step prior to data collection and/or after obtaining the criterion-pattern.

Readers familiar with bifactor modeling may wonder how it differs from (latent variable) CPA, given that bifactor models also extract a general component and specific components from a set of indicators. Importantly, the two approaches serve different purposes and operate at different levels of analysis. Bifactor models decompose measurement variance into an orthogonal general factor and orthogonal specific factors, thereby ensuring statistical independence among latent predictors. In contrast, (latent variable) CPA decomposes predictive variance by reparameterizing the regression of a criterion on a set of predictors into level and pattern components. These components are not constrained to be orthogonal and may correlate depending on the empirical structure of the data. Moreover, whereas bifactor models represent specific variance through multiple domain-specific factors, CPA captures shape as a single, criterion-relevant pattern across the full set of predictors. As such, bifactor models and (latent variable) CPA should be viewed as complementary: the former clarifies the structure of the predictors, whereas the latter clarifies how configurations of predictors relate to outcomes.