Availability bias and Campbell's Law: why Quebec's rising graduation rate masks stagnant PISA scores

Policymakers often rely on simple quantitative indicators to navigate complex policy environments, yet these indicators frequently become targets that distort the very processes they are meant to monitor. Drawing on Campbell's Law (1979)) and two cognitive biases—availability heuristic and attribute substitution—this interpretive case study of Quebec's education policy examines the consequences of elevating the high-school graduation rate to the central performance target. Since the 2017 Politique de la réussite éducative, education spending has risen by 55% (2018–2019 to 2024–2025), the graduation rate has increased from 71.9% to 82.4% (2023–2024), and pass rates on ministerial examinations have improved. However, Quebec's scores on the Programme for International Student Assessment (PISA) in mathematics, science, and reading have stagnated or declined over the same period. The analysis identifies rational institutional adaptations—such as the rapid expansion of students designated with handicaps or learning difficulties, widespread use of assistive technologies in assessments, and teaching to the test—as mechanisms that boost the visible indicator without corresponding gains in deeper cognitive competencies. This case illustrates how cognitive biases and reductionist heuristics in governance can produce an illusion of progress, underscoring the need for decision-making frameworks that incorporate independent external benchmarks to better align incentives with genuine student competency development.

Policymakers rely on quantitative indicators to navigate environments characterized by high complexity. In such contexts, they must balance conflicting objectives—such as performance and equity—while contending with incomplete information and ambiguous signals. Cognitive science shows that decision-makers, even when well-intentioned, are prone to reductionist approaches: they tend to focus on what is most easily measured, recalled, and communicated. A wide array of cognitive biases could be mobilized to explain this kind of policy failure—anchoring effects, status quo bias, confirmation bias, and optimism bias, among others. Two biases are particularly salient here. The availability bias leads actors to judge performance based on information that comes most readily to mind—typically headline indicators widely reported in the media. The attribute substitution effect leads them to replace a difficult question (“Are students’ fundamental cognitive competencies genuinely improving?”) with a simpler, more measurable proxy (“Is the graduation rate increasing?”). The result is often an illusion of validity: the indicator comes to stand in for the complex reality it is meant to represent.

Education policy is a paradigmatic domain in which these dynamics play out with particular intensity. Real outcomes—students’ cognitive skills—are difficult to measure directly and to attribute causally, while political, media, and electoral pressures strongly favor indicators that are simple, visible, and communicable. When a quantitative metric becomes the central focus of accountability, Campbell's Law (1979) predicts that it will be subject to corruption pressures: actors rationally adapt their behavior to improve the indicator, sometimes at the expense of the underlying processes the indicator was intended to monitor.

The Quebec case offers a compelling illustration of these mechanisms. Since the Politique de la réussite éducative of 2017, the government has made the graduation rate its central Key Performance Indicator (KPI) and has substantially increased education spending. The rate has risen significantly since the late 2000s. Yet over the same period, Quebec's scores on PISA—an independent, externally administered assessment of cognitive competencies in mathematics, reading, and science—have stagnated or declined. This divergence between an improving official indicator and deteriorating results on an external benchmark constitutes a contradictory signal typical of complex policy environments.

This article offers an interpretive case study (Flyvbjerg, 2006; Odell, 2001; Yin, 2018) rather than a causal demonstration. It does not claim to establish a direct causal link between specific policies and the observed decline in PISA scores. Instead, it proposes a plausible interpretive hypothesis: The patterns observed are consistent with the combined effect of institutional incentives created by the elevation of the graduation rate as the dominant KPI and cognitive biases that make simple, visible indicators disproportionately salient in policy discourse. While this interpretation is plausible, alternative or complementary explanations — such as deliberate political prioritisation of an easily communicable success narrative or administrative accountability arrangements — cannot be ruled out on the basis of the available evidence. By applying Campbell's Law to the Quebec context and examining concrete practices—such as the expansion of EHDAA designations, the widespread use of assistive technologies, and teaching to the test—the article seeks to illuminate the mechanisms through which these dynamics operate. This framing is consistent with a broader research program on governance in complex environments, where the gap between measured indicators and emergent system outcomes is a structural feature rather than an anomaly (Béchard, 2023).

This article is submitted as a Perspective contribution. Perspective articles present a viewpoint on a specific area of investigation and are particularly suited to the analysis of complex policy phenomena where interpretive synthesis, theoretical framing, and forward-looking implications are more appropriate than large-scale causal inference. Accordingly, this paper does not aim to establish statistical causality between specific policies and PISA trends. Instead, it offers a theoretically grounded interpretive perspective on the divergence between a rising administrative indicator (high-school graduation rate) and stagnant or declining results on an independent external benchmark (PISA).

Quebec was selected as the focal case for three reasons: (1) it exhibits a clear, sustained, and publicly documented divergence between sharply improving graduation rates and non-improving PISA scores over the period 2009–2022; (2) the graduation rate was explicitly elevated to the status of central Key Performance Indicator following the 2017 Politique de la réussite éducative; and (3) rich publicly available data exist on both administrative indicators and PISA results, allowing for transparent comparison.

The analysis draws on the following sources: Official cohort-based graduation and qualification rates published by the Ministère de l’Éducation du Québec (MEQ) and Statistique Québec (2009–2024);

Pass rates on mandatory ministerial examinations in mathematics and science (regular French-language tracks);

PISA results for Quebec in mathematics, science and reading (OECD, 2010, 2014, 2016, 2019, 2023);

Policy documents (MEES, 2017; successive strategic plans), budget speeches and official statistical dashboards;

Peer-reviewed and grey literature on EHDAA designations, assessment accommodations and teaching-to-the-test practices.

Graduation rate

Launched in 2017, the Politique de la réussite éducative set a target of 90% graduation or qualification by 2030, while targets of around 80% already existed in the strategic plans of the 2010s (MEES, 2017; Radio-Canada, 2017). The Policy also included several complementary objectives, such as reducing achievement gaps between certain groups, increasing to 80% the proportion of children free from developmental vulnerabilities at school entry, and improving literacy skills (MEES, 2017). Both the government and the media have focused their attention on the graduation rate to such an extent that it can be regarded as the Key Performance Indicator (KPI).

If the graduation rate is the primary KPI of Quebec's education policy, financial investment is its preferred lever of action. Generally speaking, expenditures have been rising since 2008–2009, but it is above all from 2018 to 2019 onward that the increase accelerated. Between 2018 and 2019 and 2024–2025, education spending rose by 55%, placing Quebec at the top of the Canadian provinces for per-student expenditure in constant dollars (Fraser Institute, 2024; Gouvernement du Québec, 2025).

Given such substantial investments, one must ask whether they have produced results. In fact, a significant improvement has occurred since 2008–2009, when the rate stood at 71.9%. It reached a peak of 84.1% in 2021–2022 and subsequently declined to 82.4% in 2023–2024 (Statistique Québec/Ministère de l’Éducation du Québec, 2025).

Following this rise in the graduation rate, it comes as no surprise that results on mandatory ministerial examinations in mathematics and science have generally trended upward in recent years (Tableau de bord de l’éducation, Gouvernement du Québec, 2026).

PISA: an external signal contrasting with the official indicator

Despite a recent decline attributable in part to the consequences of the pandemic, the graduation rate presents an overall positive picture. However, other indicators call for caution in interpreting this rate. The Programme for International Student Assessment (PISA), administered by the OECD to 15-year-old students every three years in more than 80 countries, provides standardized measures of cognitive competencies in mathematics, reading, and science. Its independent sampling and absence of integration into local accountability frameworks make it a an interesting external benchmark.

While PISA offers a robust and internationally standardized external benchmark for applied cognitive competencies, scholars have long debated the extent to which large-scale assessments can fully capture the broader spectrum of complex cognitive skills. Recent evidence suggests, for example, that PISA scores primarily reflect “cognitive school skills” shaped by curricular knowledge and schooling context rather than context-independent cognitive ability (Boman, 2023). This limitation, however, does not undermine PISA's value as a comparative trend indicator over time.

It should be noted, also, that PISA is a low-stakes assessment for participating students. Economists have demonstrated that performance on low-stakes tests can be influenced by students’ motivation and the effort they choose to exert during the assessment itself (Gneezy et al., 2019). Stable or declining PISA scores must therefore be interpreted with appropriate caution. Nevertheless, the standardized administration, representative sampling, and consistent framework across cycles continue to make PISA a reliable available tools for identifying broad trends in cognitive competencies at the system level.

Also, while PISA does not assess exactly the same competencies as the Quebec school curriculum and targets 15-year-old students—whereas certification assessments leading to the Quebec diploma generally take place at ages 16–17—PISA nevertheless allows for an appraisal of fundamental cognitive competencies and their comparison with the trend observed through the graduation rate over recent decades.

That being said, Quebec's PISA scores have diverged sharply from its rising graduation rate. In mathematics, they fell overall from 543 points in 2009 to 514 in 2022 (see Figure 1). Science showed a statistically significant decline, dropping from 524 in 2009 to 512 in 2022 (see Figure 2). Reading followed a similar downward trend, from 522 in 2009 to 501 in 2022, despite a brief peak at 532 in 2015 (OECD, 2023, 2025).

Since Quebec remains well above the OECD average in all three assessed domains, this relatively favorable position can create the impression that the education system is performing well. Indeed, media outlets and governments generally boast of Quebec's strong results (CMEC, 2023; Vailles, 2023). However, as noted in the preceding section, when Quebec is compared over time, results are not improving—on the contrary, they are regressing. This evolution constitutes a contradictory indicator that tempers the optimistic image conveyed by the rising graduation rate and pass rates on ministerial examinations.

Mechanisms consistent with Campbell’s Law

The divergence between the rising graduation rate and the stagnation or decline in PISA scores is consistent with the mechanisms predicted by Campbell's Law (1979). Campbell's Law states that the more any quantitative social indicator is used for social decision-making, the more subject it will be to pressures and the more apt it will be to distort the social processes it is intended to monitor. The Quebec case illustrates patterns that align with this prediction: institutional actors appear to have responded to a powerful incentive structure in which the graduation rate functions as the dominant and highly visible KPI. However, the available aggregate data do not permit direct observation of intentional manipulation, goal displacement, or strategic gaming at the school or individual level. The analysis therefore distinguishes between documented trends in the indicators, institutional adaptations whose timing and direction are consistent with the incentives created by the KPI, and the interpretive hypothesis that these adaptations contribute to the observed divergence.

This interpretation draws on, and seeks to extend, a substantial literature on the unintended consequences of governance by indicators. Campbell's Law is closely related to Goodhart's Law and has been elaborated through concepts such as goal displacement (Merton, 1940), measure fixation (Smith, 1995), gaming and strategic behaviour in accountability regimes (Bevan and Hood, 2006; Hood, 2006), tunnel vision (Smith, 1995), and performance paradoxes (van Thiel and Leeuw, 2002). In the education sector, these dynamics have been documented across multiple jurisdictions when high-stakes quantitative targets are prioritised without adequate complementary measures (e.g., Nichols and Berliner, 2007; Lingard et al., 2017). The Quebec case adds to this literature by highlighting the interaction between institutional incentive structures and cognitive biases — specifically the availability heuristic and attribute substitution (Kahneman, 2011; Tversky and Kahneman, 1973) — that make simple, visible, and communicable indicators disproportionately influential in policy discourse and decision-making.

Analysis of the 2017 Politique de la réussite éducative, subsequent ministerial strategic plans, budget speeches and media coverage reveals a consistent and dominant focus on the graduation rate as the headline metric of educational success, with comparatively limited public discussion of PISA trends or of the distinction between certification outcomes and deeper cognitive competencies. This pattern of attention is consistent with availability bias, whereby decision-makers and the public rely on information that is most readily recalled and communicated. It is also consistent with attribute substitution, whereby the difficult question of whether fundamental cognitive competencies are improving is replaced by the simpler, more measurable proxy of whether the graduation rate is rising. Alternative or complementary explanations — including deliberate political prioritisation of an easily communicable equity-and-success narrative, administrative accountability arrangements, and data availability — cannot be ruled out and may interact with cognitive mechanisms.

The political and media focus on the graduation rate, elevated to a central KPI, functions as an implicit heuristic among decision-makers. Cognitive science readily explains why such a strategy of cognitive reductionism—though intuitive and politically advantageous—leads to errors of judgment. Research on decision-making in complex systems similarly documents this tendency: when faced with multi-dimensional, dynamic environments, decision-makers systematically under-weight delayed or distal signals in favor of immediate, legible indicators (Béchard, 2023; Béchard et al., 2025). In situations of high uncertainty, time pressure, and multiple conflicting objectives, decision-makers naturally activate several well-documented cognitive biases. The availability bias (Tversky and Kahneman, 1973) leads them to assess the education system's performance based on the information most easily recalled and communicated—here, the evolution of the graduation rate, which receives extensive media coverage—rather than on data that are more difficult to interpret, such as PISA scores.

At the same time, the “attribute substitution” (Kahneman, 2011) leads them to replace the complex question to be evaluated (“Are students’ fundamental cognitive competencies genuinely improving?”) with a simpler, more measurable one (“Is the graduation rate increasing?”). The KPI then becomes a substitute for the reality it is meant to represent, creating a powerful illusion of validity. Campbell, 1979 had already anticipated this phenomenon with remarkable clarity.

The Quebec case suggests that policymakers, even well-intentioned ones, remain vulnerable to cognitive biases and reductionist approaches. To date, the persistent divergence between the rising graduation rate and falling PISA scores has still not triggered a formal governmental review, in part because Quebec continues to rank among the top performers in the OECD. Yet this situation highlights the urgent need for more robust decision-making protocols and tools—integrating external cognitive measures—in order to align institutional incentives with genuine competency development rather than its statistical appearance.

The role of assistive technologies

One indicator that has evolved in parallel with the rise in the graduation rate is the sustained increase in the population of Quebec students formally designated as having handicaps or learning difficulties (EHDAA). From one in ten students at the turn of the 2000s, this proportion has now reached one in five province-wide and one in three at the secondary level (Commission des droits de la personne et des droits de la jeunesse, 2026). Each EHDAA student must have an individualized intervention plan that may include adaptations to assessment conditions. This increase in designations may reflect several concurrent factors, including greater awareness and identification of learning difficulties expanded diagnostic practices, broader eligibility criteria, and policy orientations favoring inclusion. While the exact contribution of each factor is difficult to isolate with currently available data, the substantial rise has coincided with a parallel expansion in the use of individualized intervention plans and assessment accommodations.

Among the most common accommodation measures are additional time for assessments and assistive technologies (spell and grammar-checkers and word predictors). The longitudinal study by Rousseau et al., 2018 shows that these technological tools significantly reduce spelling and grammar errors without improving overall writing competencies (coherence, vocabulary, narrative quality). These competencies progress at the same rate with or without the tools, thanks to systematic classroom instruction. A quasi-experimental Quebec study of 314 adolescents confirms that gains are limited to lexical and grammatical spelling, with no effect on other dimensions of writing competency (Ouellette et al., 2022).

Boyer, 2025 has offered a critical interpretation of recent developments in Quebec orthopedagogy, describing a gradual transition from what he characterizes as remedial orthopedagogy to relief orthopedagogy. While remedial intervention explicitly aims to close the performance gap between struggling students and their peers by enabling them to reach the same learning objectives as the reference group, the relief approach, according to this perspective, merely alleviates difficulties through accommodations (assistive technologies, reduced requirements, modified assessment conditions) without genuinely seeking to close cognitive gaps (Boyer, 2025; Boyer and Bissonnette, 2025). Boyer argues that this second form of intervention does not produce lasting gains, as it functions more as a palliative measure than a genuine remediation lever. He warns against the long-term consequences of such an approach: by masking deficiencies rather than correcting them, it may perpetuate inequalities among students. It is important to note that this analysis represents Boyer's interpretive critique of evolving policy orientations and professional practices rather than an empirical finding based on large-scale longitudinal data tracking individual student trajectories and outcomes before and after specific changes in service models.

It is in this context that the use of assistive technologies as accommodation measures raises a pertinent question. By reducing spelling errors during assessments—particularly on mandatory examinations required for diploma attainment—these tools can improve success indicators without the underlying cognitive competencies advancing proportionally. They thus function more as compensation than as a catalyst for deep learning (Rousseau et al., 2018), optimizing the surface indicator (i.e., passing the examination) while potentially leaving intact the very gap it is meant to measure.

Alternative explanations for the observed divergence

Several factors unrelated to indicator-driven adaptation could contribute to the divergence between graduation rates and PISA scores. First, the COVID-19 pandemic caused major learning disruptions between 2020 and 2022; PISA 2022 captured a cohort directly affected by school closures, whereas graduation rates reflect cohorts that experienced varying degrees of disruption and, in some cases, adjusted certification requirements. Second, demographic and socioeconomic changes, including shifts in student composition due to migration and changes in family background distributions, may affect the two indicators differently. Third, variations in PISA sampling frames, exclusion rates, or student effort on a low-stakes assessment could influence results (Gneezy et al., 2019). Fourth, curricular reforms, the expansion of vocational and alternative qualification pathways, and modifications to the rules governing access to the Diplôme d’études secondaires may have facilitated higher certification rates without producing equivalent gains in the applied cognitive skills assessed by PISA. Finally, the broader implementation of inclusive education policies has increased support for diverse learners; while these policies pursue important equity objectives, their net effects on average performance metrics are complex and not yet fully understood. A comprehensive assessment of the relative contribution of these factors would require longitudinal microdata linking individual student trajectories, EHDAA status, accommodations received, and cognitive outcomes — data that are not currently publicly available in Quebec.

In addition to the use of assistive technologies, students and teachers invest considerable effort in maximizing results on mandatory ministerial certification examinations, which condition diploma attainment. These efforts often take the form of intensive preparation focused on specific strategies such as memorization of standard formulas, learning of standardized writing structures, and repeated exam simulations (Baribeau, 2015; Copp, 2018). This phenomenon of “teaching to the test” has been frequently discussed and documented in the Canadian and Quebec educational literature, although the precise prevalence, mechanisms, and long-term effects remain subjects of ongoing empirical debate and are sometimes inferred from indirect indicators rather than large-scale direct observation.

From the perspective of policymakers, teaching to the test and the increase in the number of EHDAA students and the measures attributed to them appear as inclusive solutions that simultaneously address equity and achievement objectives.

These developments — the increase in EHDAA designations and the expanded use of assessment accommodations — are consistent with rational responses by schools and teachers to the strong institutional incentives created by the central graduation-rate KPI. They may contribute, at the system level, to a loosening of the relationship between certification outcomes and the cognitive competencies measured by PISA. However, the present data do not allow us to establish the relative weight of this mechanism compared with other factors, nor to conclude that the increase in designations or accommodations was primarily intended to inflate statistics.

Is the graduation rate a good KPI?

Beyond the cognitive biases that lead decision-makers to privilege easily measurable indicators such as the graduation rate, a second and complementary problem arises. Even if the graduation rate were a perfectly valid measure of system performance, the literature in the economics of education suggests it would still constitute a relatively weak proxy for the cognitive competencies that ultimately matter for individuals and society. This disconnect between what is measured and what is valued reinforces the dynamics predicted by Campbell's Law.

Even if part of the rise in the graduation rate appears to reflect surface adaptations rather than a deep improvement in competencies, one might argue that this indicator remains a good KPI insofar as obtaining a diploma is itself a legitimate and important objective. Yet the literature in the economics of education once again urges caution. Indeed, the two strongly improving indicators –the graduation rate and per-student financial resources—do not guarantee an education of quality that generates economic benefits for the population. Numerous studies conclude that there is no strong and consistent relationship between school resources and student performance, once other factors are controlled for (Hanushek, 1986; Hanushek, 1997; Hanushek and Woessmann, 2008, 2012). These resources are often absorbed by expenditures with no demonstrated effect on learning rather than being concentrated on what truly matters: teaching quality.

Further work by Hanushek and Woessmann confirms that “there is strong evidence that the cognitive skills of the population—rather than mere school attainment—are powerfully related to individual earnings, to the distribution of income, and to economic growth” (2008, p. 607). In short, a simple graduation rate and increased education budgets do not constitute good indicators of cognitive development.

While the quality of teaching is a critical determinant of student learning, large-scale syntheses indicate that explicit, systematic instructional approaches produce substantial effect sizes on fundamental outcomes, particularly for students in difficulty (Bissonnette et al., 2010). The governance implication of the present analysis is nevertheless that heavy reliance on a single outcome indicator such as the graduation rate risks diverting attention from the inputs and processes most consistently associated with improved cognitive competencies. The recent creation of the Institut national d’excellence en éducation (INEE) in 2025, whose mandate includes the promotion of evidence-based practices, constitutes a potentially important complementary mechanism. Effective governance in complex educational systems is more likely to be achieved through balanced indicator frameworks that combine administrative outcome measures with independent external benchmarks and that direct attention toward practices shown to enhance deep learning rather than solely toward easily measurable certification targets.

Limitations

This analysis has several limitations that circumscribe the scope of its conclusions. On the causal level, the central argument rests on a plausible hypothesis rather than a direct empirical demonstration. Establishing a definitive causal link would require at least individual longitudinal data (students’ EHDAA status, type of intervention plans, nature of accommodations granted, and actual cognitive outcomes) that are not publicly available in Quebec. The analysis should therefore be read as a coherent and substantiated interpretive hypothesis rather than as established causal proof.

Another limitation of this study is that the comparison of PISA mathematics and science scores with Quebec's regular ministry examinations in mathematics and science relies on a single representative pathway rather than a fully weighted provincial average. In Quebec, students follow multiple programs, including enriched or regular tracks offered in both French and English. Although pass rates are publicly available for each examination, the exact number of students enrolled in each program is not disclosed, making it impossible to compute a true weighted average across all sequences. The regular French-language mathematics and science examinations were therefore selected (Culture, société et technique), as they constitute the most representative option with the largest student enrollment. In any case, all ministry exams consistently show a general upward trend in performance compared to the initial years of data compilation. This targeted approach does not undermine the pertinence of the overall comparison, which remains valid for highlighting broader educational trends in Quebec.

It is also important to note that this study does not contend that the graduation rate is entirely artificially inflated. The slight decline observed in 2023–2024 (82.4%, a decrease of 1.9 percentage points from the peak of 84.1% in 2021–2022) shows that the indicator remains sensitive to genuine system-wide shocks, including the learning disruptions caused by the COVID-19 pandemic (Tableau de bord de l'éducation, Gouvernement du Québec, 2026). The graduation rate therefore reflects, in part, authentic dynamics within the Quebec education system.

Nevertheless, its promotion as a singular central KPI exposes the system to the pressures predicted by Campbell's Law. The persistent divergence between the rising graduation rate and the stagnation, or even regression, of PISA scores suggests that part of the indicator's progression results in part from rational adaptation by actors to the incentives of the assessment system rather than from a generalized and lasting improvement in actual cognitive competencies. This interpretation, though indirect, invites a cautious reading of official indicators and should prompt decision-makers to examine the contradictory signals sent by different measures of educational performance.

Throughout this analysis, claims drawn from the educational literature have been presented with the recognition that they often rest on interpretive syntheses of policy trends, professional observations, and smaller-scale studies rather than on definitive large-scale causal evidence. The interpretive hypothesis advanced here should therefore be read as a plausible account of observed divergences between indicators, intended to stimulate further empirical investigation rather than as a fully demonstrated causal explanation.