Policy evaluation as a source of policy advice in Italy: a tale of two policy sectors

1 Introduction

Evaluation and policy advice are closely related. Evaluation is commonly defined as an applied inquiry process that collects and synthesises evidence to draw conclusions about the value, merit, or quality of programmes, policies, or plans (Fournier, 2005). Its findings can support elected officials and civil servants in improving policy design and implementation (Varone et al., 2023). Policy advice, in turn, can be understood as the provision of analytical support to government decision-making (Colebatch et al., 2010) by the diverse actors who produce different flows of information, data, and knowledge about the policy and the politics in a specific subject matter (Galanti, 2022). Yet the relationship between the two activities remains insufficiently explored: evaluation and policy advice are not coextensive, and the conditions under which evaluative knowledge is translated into actionable advice have received limited systematic attention.

This article pursues two objectives. First, drawing on the literatures on policy advice and policy evaluation, it examines how the two activities relate and differ in terms of their position in the policy process, uses of knowledge, functions, sources of legitimacy, and required expertise. Second, it investigates the conditions under which the use of evaluators’ recommendations in organisational planning and decision-making is formally documented. This question is explored through a comparison of two organisational evaluation systems in Italy. Italy provides an instructive setting because the weak institutionalisation of evaluation at the national level (Stockmann et al., 2020) coexists with substantial variation across policy domains. The analysis focuses on two systems that acquired their current institutional form in the early 2010s: performance evaluation in public administration and the university quality assurance system known as AVA (Autovalutazione, Valutazione periodica e Accreditamento). The comparison addresses three analytical questions: how the two systems are institutionalised and how they operate; on what basis evaluators establish their legitimacy; and whether their recommendations are documented as informing subsequent decision-making.

The remainder of the article is organised as follows. Section 2 examines the relationship between evaluation and policy advice and reviews comparative research on evaluation institutionalisation. Section 3 introduces the Italian context, analytical framework, and research design. Section 4 presents the two evaluation systems and the documentary findings, while Section 5 discusses the results comparatively. Section 6 concludes with the article’s contributions, limitations, and implications for future research.

2 Literature

2.1 Positioning in the policy cycle

The public policy literature often presents evaluation as a link between the past and the future of public policies. An initial distinction is therefore needed between policy analysis and policy evaluation. Although both concern public policies, they differ in orientation. As Geva-May and Pal (1999) observe, policy analysis tends to be prospective and normative, addressing how a policy problem should be tackled, whereas evaluation has traditionally looked retrospectively at what has been done and why an intervention has succeeded or failed. The two fields have consequently developed partly separate bodies of literature, leaving some of their intersections insufficiently explored. One such intersection—and the focus of this article—is the relationship between policy evaluation and policy advice.

In stage-based representations of the policy process, evaluation is typically positioned after implementation and before policy reformulation (Howlett and Ramesh, 2003). At least in principle, this is the stage at which judgements can be made about whether a policy should be continued, modified, or terminated, while generating feedback for future decisions. Howlett (2023) accordingly defines policy evaluation as the monitoring of policy results by state and societal actors, which may lead to the reconceptualisation of both policy problems and their solutions.

This cyclical representation is an analytical heuristic rather than a description of how policymaking necessarily unfolds. Evaluation scholarship has long questioned its linear assumptions. Weiss (1973/1993) was among the first to show that the relationship between evaluative knowledge and policymaking is rarely straightforward. Evaluation operates within a political environment in which evidence about programme outcomes competes with other considerations. Evaluative findings may consequently exert limited influence when organisational actors prioritise institutional survival over policy effectiveness. This does not necessarily mean that the policy considerations arising from evaluation are abandoned; they may instead be deferred until institutional or political conditions become more favourable.

Moreover, positioning evaluation only downstream of implementation understates its scope. Evaluation can inform different phases of the policy process, from planning through ex ante evaluation and impact assessment, to ongoing monitoring and process evaluation, and finally to ex post judgements about the merit, worth, and value of interventions (Scriven, 1991). Pawson (2002) similarly noted that the alignment between the policy cycle and the evaluation cycle often remains abstract: decisions may be taken before evaluation findings become available or without using them. He therefore advocated systematic reviews of previous evaluation studies as a knowledge base for policymaking in a non-linear environment (Pawson and Tilley, 1997; Pawson, 2002).

Policy advice, by contrast, is more explicitly associated with policy formulation (Craft and Howlett, 2017). It combines the production of knowledge about a policy problem with recommendations concerning possible solutions, thus encompassing the analysis of problems and the proposal of responses (Halligan, 1998). The distinction between evaluation and policy advice, however, cannot rest solely on their position in the policy process. It also concerns whether and how knowledge is translated into recommendations and oriented towards decision-making.

2.2 Use of evaluative knowledge in policymaking and functions

The view of evaluation as an important component of policymaking rests on the assumption that it improves understanding of public problems and possible responses. This view echoes Heclo’s (1974) argument that existing policies can shape subsequent politics and policymaking through processes of learning.

Rooted in the rationalist tradition, the systematic evaluation of public interventions began to consolidate in the 1960s, particularly in connection with the major social programmes promoted by the Kennedy and Johnson administrations in the United States (Madaus et al., 1983; Stufflebeam and Webster, 1980). The need to determine whether, why, and to what extent substantial public investments were achieving their intended outcomes stimulated the development of different approaches to producing evaluative knowledge.

The relationship between evaluation and public decision-making was subsequently reinforced by the evidence-based policymaking agenda. Promoted most prominently under Tony Blair’s Labour government in the United Kingdom, this approach to public service reform was later adopted across countries and political traditions (Newman, 2017; Hadorn et al., 2022; Howlett and Newman, 2010). Evidence-based policymaking also became central to the better regulation agenda, influencing how governments design, assess, and revise regulatory frameworks (Radaelli, 2004). European impact assessment is a prominent institutional expression of this development: by requiring the likely effects of Commission proposals to be assessed, it institutionalises evaluative knowledge at the outset of the policy process (Dunlop and Radaelli, 2015; Melloni, 2013).

The rise of evidence-based policymaking has brought evaluation and policy advice into closer relation. When advisory processes are expected to draw on robust evidence alongside political judgement and administrative experience, evaluation—as a systematic and methodologically transparent form of knowledge production—can provide an important input into policy advice (Capano et al., 2023) and remains an important source of knowledge for decision-makers (Davies, 2012). This convergence does not imply an uncritical acceptance of a rationalist model of policymaking: its assumptions have been widely criticised, and evidence is increasingly understood to encompass multiple forms of knowledge (Head, 2008, 2016; Sanderson, 2002).

Evaluation should therefore be understood as one among several sources of knowledge that may inform policy advice, alongside scientific research, administrative experience, stakeholder knowledge, and political judgement. The argument advanced here is not that evaluation and policy advice coincide, or that evaluators should become general-purpose policy advisers, but that evaluative findings and judgements can acquire an advisory function when they are translated into recommendations oriented towards future decisions.

Whether and how evaluation should directly contribute to policymaking has generated considerable scholarly debate. A foundational contribution is Weiss’s (1977) distinction between the instrumental use of knowledge in specific decisions and its broader function of enlightening policy actors about problems and possible solutions. Subsequent research has expanded this distinction to capture the multiple ways in which evaluation may influence individuals, organisations, and policy processes (Christie, 2007; Henry and Mark, 2003; King and Alkin, 2019; Kirkhart, 2000). To systematise the diversity of theoretical approaches and methodological orientations within the field, Alkin (2004) grouped the main evaluation traditions into three branches: methods, concerned with methodological development and validity, represented by scholars such as Campbell, Cook, and Rossi; valuing, centred on evaluative judgement (Scriven and others); and use, focused on evaluation’s contribution to decision-making (Patton and others). The use branch comes closest to assigning evaluation a role analogous to policy advice. Scholars in this tradition (Stufflebeam, 2000; Wholey, 1972; King and Alkin, 2019) argue that evaluation should assist programme stakeholders in making decisions and should be designed to support programme management and organisational change.

The most influential expression of this approach is Patton’s utilisation-focused evaluation (2008; first ed. 1978). It requires evaluators to identify intended users from the outset, involve them in the design and interpretation of the evaluation, and support their engagement with its findings. Patton also emphasises process use: participation in evaluation can itself generate learning, independently of the subsequent use of findings. More broadly, use-oriented approaches stress the need to cultivate relationships with stakeholders and establish “social learning infrastructures” (Shaw et al., 2006, p. 194). This points to the importance of institutional arrangements that sustain dialogue between evaluators and policymakers. Use-oriented evaluations therefore come closest to performing a policy-advisory role. They do not abandon the methodological rigour on which evaluative legitimacy rests, but deliberately orient evaluative processes and findings towards decision-making and create the conditions for their uptake.

The extent to which this advisory potential is realised, however, depends on the function that evaluation is expected to perform. The evaluation literature identifies several functions of evaluation, not all of which are directly concerned with policymaking. Chelimsky and Shadish (1997) distinguish three main functions: knowledge development, which generates new understanding of a programme or policy area; organisational improvement, which strengthens the analytical and problem-solving capacities of public organisations; and accountability, which produces evidence about the effectiveness or efficiency of policies, programmes, or organisations. The first two functions provide clear areas of overlap with policy advice, as both can support learning and improve the capacity to address policy problems. Accountability marks a clearer boundary. Accountability-oriented evaluation responds primarily to an institutional logic of verification and control and may carry reputational consequences for the organisations and programmes under scrutiny. Its defining purpose is therefore not to formulate recommendations for future action, although its findings may still inform subsequent decisions. It should be noted, however, that accountability is not entirely absent from the policy advice literature, albeit with a different connotation: here, the concern is not with verifying the results of programmes, but with the accountability of advisors themselves—whose insulation from democratic scrutiny has been identified as a potential source of legitimacy deficits (Jarman et al., 2025).

The structure of demand therefore helps clarify when evaluation approaches policy advice: the overlap is strongest when evaluation is sought for learning or organisational improvement and weaker when its primary purpose is oversight and control.

2.3 Legitimation

A further distinction between policy evaluation and policy advice concerns the bases on which their respective actors are recognised as credible and entitled to contribute to decision-making. Legitimacy refers to the recognised rightfulness of an actor’s authority, whereas legitimation encompasses the processes through which that authority is established, sustained, or challenged. These processes determine whether an actor is considered entitled to participate in policymaking (Lippi, 2024; Lippi and Terlizzi, 2024; Esposito et al., 2026).

Evaluation derives much of its legitimacy from evaluators’ recognised methodological expertise, and the authority of its judgements rests on the soundness of the research design and its suitability for the object being evaluated. Rossi’s claim that “getting it right is more important than getting along” (as cited in Alkin, 2004, pp. 127–128) captures this priority. Use-oriented scholars, however, warn that methodological purity may reduce the usability of evaluative knowledge. Evaluation therefore faces a persistent tension between responsiveness to decision-makers and evaluators’ independence and impartiality. Policy advice derives its legitimacy more directly from advisers’ recognised epistemic authority, grounded in academic standing, institutional position, or substantive expertise (Galanti, 2023). Their legitimation also depends on their ability to select, interpret, and synthesise knowledge from different sources, transforming it into recommendations that decision-makers can use (Howlett et al., 2025). Effective advisory relationships thus require alignment between the demand for and supply of advice, supported by expectations of competence, performance, and reputation (Farazmand, 2009). The two activities consequently foreground different knowledge profiles. Evaluation emphasises methodological competences applicable across policy domains, whereas policy advice relies more explicitly on field-specific expertise and contextual judgement. Evaluation-based policy advice is most likely to emerge where these profiles intersect.

Expert authority nevertheless requires continuous sociopolitical legitimation as the bases of credibility change over time (Capano et al., 2023). Significantly, Capano et al.’s mapping of policy advisers includes researchers, think tanks, consultants, and bureaucratic analysts, but not evaluators as a distinct category. This relative invisibility can also be situated within the broader challenge identified by Jacob (2024) concerning the identity, relevance, and specificity of evaluation in a field increasingly populated by auditing, performance measurement, deliverology, and other knowledge-producing practises.

From this perspective, the question is not only whether evaluators can contribute to policy advice, but also how they can demonstrate the distinctive contribution of evaluative judgement while making it relevant and usable for decision-makers. Evaluation scholars offer contrasting answers. Scriven (1996) maintains a clear boundary between evaluation and advice: although evaluators may assist programme staff with evaluability, performance indicators, or quality control, involvement in programme design risks co-optation and loss of objectivity. Use-oriented approaches, by contrast, regard engagement with decision-makers as essential. Patton’s utilisation-focused evaluation requires evaluators to identify intended users and facilitate their engagement with findings, while constructivist approaches treat dialogue and stakeholder participation as central to evaluation’s contribution to learning and democratisation (Guba and Lincoln, 1989). Skolits et al. (2009) reconcile these positions by distinguishing among roles that evaluators may assume at different stages: designer, researcher, judge, reporter, and evaluation-use advocate. The last role, particularly relevant after an evaluation has been completed, involves promoting the meaningful use of findings and most closely resembles policy advice. These roles are neither fixed nor mutually exclusive: evaluators may preserve their independence during data collection while actively supporting use during dissemination.

Evaluators’ ability to perform an advisory role therefore depends not only on their status and competence, but also on the legitimation of the roles they assume. The central question is how they can move closer to policy advice without weakening the credibility on which their authority rests.

2.4 Clients and competencies

Policy advice has a relatively clearly defined client base. It is primarily sought by elected officials and senior executives to strengthen the knowledge base, credibility, or political sustainability of their decisions. Advice is not always provided through transparent and formalised procedures: it may take less visible and more confidential forms, particularly when politically sensitive or complex issues are involved (Di Francesco, 2000). A smaller strand of the literature has examined the relationship between advisory committees and citizens, highlighting the reputational support these bodies may provide in legitimising policy choices (Bundi and Pattyn, 2023, 2024).

The client base for evaluation is more diverse. Demand often originates from administrative and managerial actors responsible for organisations, policies, or programmes. Evaluation may therefore be commissioned for control, accountability, review, or operational learning by actors such as the managing authorities of EU co-funded programmes or managers responsible for specific policy sectors. Parliamentary and representative bodies may also commission evaluations, bringing evaluative knowledge closer to political decision-making and increasing its potential advisory role. More recently, the literature has highlighted evaluation’s democratic function and its capacity to inform citizens’ choices, including through participatory and citizen-led forms of evaluation promoted by civil society organisations (Dahler-Larsen, 2011; Stame, 2016).

Different clients require different forms of evaluation and different ways of framing and communicating findings. Overall, however, evaluation has a weaker and more mediated relationship with political decision-makers than policy advice.

These differences also shape the competencies evaluators need to perform an advisory function. Scriven (1991, p. 369) observed that making evaluation useful requires skills that evaluators often lack, including the ability to manage evaluation anxiety and the “connection power” needed to engage effectively with clients and wider audiences. Patton’s utilisation-focused approach further emphasises evaluators’ ability to manage the relational dimensions of the evaluation process. Clinton and Hattie (2021) distinguish between knowing that (knowledge of evaluation models, standards, methods, and procedures) and knowing how (the capacity to exercise judgement, manage stakeholder relationships, interpret findings in context, and translate evidence into actionable recommendations). Their analysis suggests that evaluator competency frameworks continue to emphasise the former, while the latter remains comparatively underdeveloped. Yet knowing how is particularly important to the advisory function. It enables evaluators to communicate conclusions in ways that are meaningful to decision-makers, understand the institutional and political context of policy choices, and engage with the normative dimensions of policy problems.

Developing these competencies creates a further tension. Relational and communicative skills must be combined with reflexivity and safeguards, including transparent criteria, traceable evidence, and explicit role boundaries, that protect evaluators from pressure to accommodate clients’ expectations. This balance is particularly important for evaluation units embedded within institutions: organisational proximity may enhance the relevance and use of their findings, but it must not compromise their independence or methodological rigour. Table 1 summarises the main differences between policy evaluation and policy advice identified in the literature, while highlighting the areas in which their functions, knowledge bases, and professional roles overlap.

Table 1. Main differences and areas of convergence between policy evaluation and policy advice across key analytical dimensions.

2.5 Institutionalisation of evaluation within the policy process: a comparative perspective

The preceding discussion identified several conditions under which evaluation may contribute to policy advice, including its position in the policy process, sources of legitimacy, and professional competencies. A further condition concerns the formal and informal structures through which evaluative knowledge enters policymaking. Institutional design shapes who commissions evaluation, for what purposes, and how its findings are expected to be used (Galanti, 2023).

The institutionalisation of evaluation varies considerably across countries. Comparative research by Stockmann and colleagues documents these differences across European and non-European contexts (Stockmann et al., 2020, 2023, 2026; Stockmann and Meyer, 2022). Research findings show that institutionalisation does not automatically translate into the use of evaluation as policy advice. Formal anchoring may increase the frequency, scope, and routinisation of evaluation without making its findings decisive inputs into political choices. Even in highly institutionalised systems, evaluative knowledge competes with political bargaining, administrative priorities, and existing policy commitments. This is consistent with the policy advice literature, which shows that evidence reaches political decisions through complex institutional and political filters rather than through a direct process (Weiss, 1977; Cairney, 2016).

According to the country analyses conducted by Widmer (2020) and Astbury and Bayley (2023), Switzerland and Australia illustrate this distinction from opposite directions. Switzerland combines strong legal and institutional foundations with a selective and politically mediated use of evaluation. A constitutional requirement mandates the assessment of the effectiveness of federal measures, while legislative evaluation clauses, parliamentary interventions, and committee deliberations provide several channels through which findings can enter policymaking. Nevertheless, evaluation is shaped by the consensual and negotiated character of Swiss politics. Findings may inform policy design and implementation but may also be used symbolically to legitimise legislative proposals. Their influence on referendum campaigns is mediated by experts and remains limited in media coverage (Widmer, 2020, pp. 214–217).

Australia presents a contrasting configuration. Despite comparatively weak system-wide legal and organisational anchoring, evaluation can achieve substantial influence through specialised institutions. The Productivity Commission, an independent research and advisory body within the Treasury portfolio, conducts public inquiries and produces recommendations that have influenced policy and programme reforms (Astbury and Bayley, 2023, p. 73). The Australian case thus suggests that institutional capacity concentrated in a specialised organisation can support evaluation use even in the absence of a strongly institutionalised national evaluation system.

Taken together, the two cases indicate that the relationship between institutionalisation and advisory use is not linear: strong national institutionalisation does not guarantee that findings will inform decisions, while weak national institutionalisation does not necessarily prevent it. National-level measures may also conceal substantial variation across policy sectors, levels of government, and individual organisations. Administrative culture, political demand, professional networks, and the presence of actors who champion evaluation use (Patton, 2008) may compensate, at least partly, for the absence of overarching frameworks.

3 Research questions and methodology

3.1 Case selection and comparative design

The preceding discussion raises a specific empirical question concerning Italy, where weak institutionalisation at the national level (Melloni, 2020) coexists with more developed arrangements in specific sectors and organisations: evaluation is well established in some policy areas but remains recent or fragmented in others. This makes it possible to examine whether sectoral arrangements can enable evaluation to perform an advisory role even under conditions of weak national coordination.

The study adopts an exploratory, theory-building comparative design. It examines two systems of organisational evaluation that have undergone substantial development in Italy: performance evaluation in public administration and the university quality assurance system known as AVA (Self-assessment, Periodic Evaluation and Accreditation). Both acquired their current institutional form in the early 2010s but differ in scope, governance, and relationship with organisational decision-making.

The comparison is not intended to contrast a more institutionalised case with a less institutionalised one. Rather, it examines how different combinations of institutional rules, assigned functions, evaluator roles, and oversight arrangements shape the capacity of evaluation to contribute to policy advice.

3.2 Analytical questions

The factors identified in the literature review and summarised in Table 1 are explored through three analytical questions.

• (1) To what extent is each evaluation system institutionalised, and how does its design structure the relationship between evaluation and decision-making?

The analysis considers the sectoral rules governing evaluation, the functions assigned to evaluation bodies, their formal interfaces with decision-makers, and the powers of supervisory bodies. It also distinguishes between arrangements that support the substantive use of findings and those that primarily encourage procedural compliance.

• (2) On what bases are evaluators legitimised, and to what extent is their role recognised within each system?

This question examines the criteria through which evaluators are selected and how Evaluation bodies operate. It also considers the formal recognition of their role and the safeguards protecting their independence.

• (3) What documentary evidence indicates that evaluators’ recommendations are taken into account in organisational planning and decision making?

The analysis looks for documentary traces of uptake, including references to evaluative findings in planning and decision documents, formal responses to recommendations, commitments to corrective action, changes explicitly linked to evaluation, and subsequent monitoring of implementation.

Taken together, the findings from the three analytical questions shed light on how different configurations of institutional design and the formal bases of evaluators’ legitimacy help account for variation in the documented use of their recommendations in organisational planning and decision-making across the two systems. The study deliberately focuses on dimensions observable at the formal, documentary, and system-wide levels, while recognising that the relationship between evaluation and policy advice may also be shaped by the configuration and maturity of individual organisations, evaluators’ competences, and patterns of interaction among the actors involved. Future in-depth qualitative research at the level of individual organisational units and professional actors could examine how these factors interact and assess their relative contribution to the emergence of an advisory role.

3.3 Sources and methods

The empirical analysis is primarily desk-based and draws on two categories of documentary sources: the regulatory frameworks governing the two systems and documents produced by a purposive sample of public administrations and universities. The two evaluation systems constitute the primary units of comparison. The selected organisations are treated as embedded observations used to examine how each system operates and how the use of evaluators’ recommendations is formally documented, rather than as matched cases for organisation-level causal inference.

For the university sector, five cases were selected from the 22 institutions reviewed by ANVUR in 2025. For each institution, the documentation prepared by the university and the final accreditation report issued by the external evaluation panel are publicly available on the agency’s website. The cases were selected to provide variation in geographical location, institutional size, and final accreditation outcome.

For performance evaluation, the sample comprises three regional capitals—Milan, Bologna, and Rome—and two national administrations: the Ministry of Justice and the Ministry of Economy and Finance. This selection provides variation across national and local levels of government and different administrative contexts. The relevant planning and evaluation documents are publicly available online. Claude AI assisted the documentary analysis¹, and all outputs were manually checked against the original documents. Table 2 summarises the cases and documentary sources included in the analysis.

Table 2. Organisations and documentary sources included in the analysis.

Comparison across regulatory documents, organisational records, and professional knowledge² provides partial triangulation of the findings. The analysis nevertheless has important limitations. Its small purposive sample does not support statistical generalisation, while documentary evidence captures recorded responses to evaluation rather than the full process through which findings influence decisions. More important, the study does not systematically reconstruct interactions within individual organisations or assess how evaluators’ personal characteristics affect their advisory role. These questions require more extensive interviews and process-level evidence.

4 Research findings

4.1 The evolution of evaluation in Italy

In Stockmann and colleagues’ comparative analysis, Italy is ranked among the European countries with the lowest degree of evaluation institutionalisation. This assessment rests primarily on the absence of a general evaluation law in the Italian legal system and on the weakness of institutions with an explicit evaluative mandate. The picture is, however, only partially representative: at the regional level, reference to evaluation appears in the statutes of almost all Italian Regions, and evaluation has nonetheless spread across specific sectors, with varying pace depending on the policy domain.

During the 1990s, Italy undertook an extensive process of institutional reform aimed at devolving powers and responsibilities from central to subnational levels, culminating in the constitutional reform of 2001. These reforms also formed part of the broader New Public Management turn, which sought to introduce managerial principles, performance measurement, and results-based accountability into public administration. Against this backdrop, the so-called “Bassanini laws” (1997–1999) introduced a range of tools drawn from the private sector and Anglo-Saxon administrative traditions, including strategic planning, management control, performance evaluation, merit-based systems, and customer satisfaction. As Marra (2018) observes, however, the institutionalisation of evaluation and performance measurement in Italy remained ambiguous: managerial instruments were layered onto established legalistic and bureaucratic traditions and often reinforced procedural compliance rather than organisational learning. Although these reforms failed to become fully embedded in the broader administrative culture, they created opportunities for reform-oriented public managers and organisations to develop their own performance-management systems.

A further impetus came from the European Union. Since the late 1990s, mandatory evaluation requirements for national and regional Cohesion Policy programmes have contributed both to the diffusion of an evaluation culture and to the development of a market for evaluation services. Over the past two decades, the use of evaluation has expanded across policy domains, albeit unevenly. This institutionalisation has advanced through new sectoral regulations, the creation of dedicated institutions and processes in sectors such as health and social services, environment and agriculture, schools and universities and, more recently, the non-profit sector. This expansion has also fostered the development of in-house evaluation units within organisations, complementing the private consultants and research institutes that have traditionally provided evaluative services.

4.2 Performance evaluation in Italian public administration

A major step in the consolidation of this New Public Management-oriented trajectory came with Legislative Decree 150/2009, known as the “Brunetta reform.” The decree introduced a mandatory performance management cycle, requiring central administrations to adopt multi-year plans, define measurable objectives, and report annually on organisational and individual performance. Its general principles also apply to other public bodies, including local authorities.

Each administration must appoint an Independent Evaluation Body (Organismi Indipendenti di Valutazione—OIV, or in some cases, Nuclei di Valutazione-NdV) to oversee the performance measurement system and validate the annual performance report. The candidate evaluators members must be registered on a national list. Registration requires at least 5 years of professional experience in areas such as performance measurement, planning, management control, or financial programming, but no specific expertise in policy or programme evaluation. The Evaluation Body members are selected through public procedures and appointed by the administration’s governing authority.

The reform initially assigned methodological guidance and central oversight to CiVIT. When CiVIT became the National Anti-Corruption Authority (ANAC), responsibility for performance management was transferred to the Department of Public Administration in 2014. A coherent system of central governance has nevertheless never become fully established.

Although evaluation is formally embedded in this system, accountability and compliance have tended to prevail over knowledge production and organisational improvement (Chelimsky and Shadish, 1997). The reform was intended to focus administrations on results for citizens, but its performance and sanction mechanisms also encouraged cautious target setting. Similar effects were identified by Bevan and Hood (2006) in England, where target-based governance generated gaming and other opportunistic responses.

Evaluation bodies functions focus primarily on methodological oversight and compliance. They may formulate recommendations in their annual reports, but administrations are not normally required to respond unless non-compliance is identified. The system therefore lacks a formal feedback loop connecting recommendations to subsequent decisions. Performance evaluation may consequently become a bureaucratic exercise rather than a resource for organisational learning and improvement (Rebora et al., 2017).

The analysis of the Integrated Plans of Activities and Organisation (PIAOs) of five administrations -three major municipalities and two central ministries—confirms this gap between the formal recognition of Evaluation bodies and the use of their findings. Although all the documents acknowledge their role, they provide little evidence of initiatives undertaken in response to their recommendations (Table 3).

Table 3. Recognition and documented uptake of evaluation in the planning documents of the selected public administrations.

^*To facilitate reading, I use the term Evaluation body to refer to both Nuclei di Valutazione (in universities and local administrations) and Organismi Indipendenti di Valutazione (OIVs) (in central government administrations and, in some cases, local administrations).

Author’s analysis of the 2026–2028 Integrated Plans of Activities and Organisation (PIAOs).

This pattern is not universal. The cases examined by Adinolfi and Uricchio (2025) show that performance information can contribute to revising objectives, improving organisational processes, and generating public value when evaluation is supported by adequate capacity, integrated with strategic planning, and actively used by administrative leaders. Such practises remain uneven, suggesting that the effects of a common institutional framework depend substantially on how individual administrations interpret and implement it.

4.3 The quality evaluation system for universities—AVA 3

The university sector has developed a similarly formalised, but more encompassing, evaluation framework through the AVA system, currently in its third version (AVA 3). Established following Law 240/2010 and Legislative Decree 19/2012, AVA is implemented by the Italian National Agency for the Evaluation of Universities and Research Institutes (ANVUR). ANVUR’s membership of the European Association for Quality Assurance in Higher Education (ENQA), reconfirmed in 2025, subjects the agency to periodic review against the European Standards and Guidelines for Quality Assurance (ESG). Universities are also covered by the public administration performance framework and participate in the national Research Quality Assessment (VQR).

AVA assesses five broad areas: institutional strategy and organisation, resource management, internal quality assurance, teaching and student services, and research and third-mission activities. Universities must maintain an internal quality assurance system and produce standardised monitoring and review documents. Approximately every 5 years, external evaluation panels (Commissioni di Esperti della Valutazione, CEVs) assess the institution and a sample of its degree programmes, doctoral programmes, and departments through documentary analysis and remote and on-site visits.

Based on the CEV’s findings, ANVUR publishes an accreditation report and submits a proposal to the Ministry of Universities and Research, which adopts the formal decision. Satisfactory outcomes result in 5-year accreditation, whereas a conditional judgement requires the university to address the identified deficiencies. The internal Evaluation Body monitors the resulting corrective measures. Persistent non-compliance may lead to the withdrawal of institutional or programme accreditation and prevent the admission of new student cohorts. Evaluation therefore has direct regulatory consequences.

Each university has an Evaluation Body (Nucleo di Valutazione, NdV), established under Law 370/1999. It comprises between five and nine members, at least two of whom must have evaluation expertise. Law 240/2010 also requires members to possess high professional qualifications and to be predominantly external to the university. The Evaluation Body operates autonomously, has access to the necessary institutional data, and publishes its findings. Compared with the eligibility requirements for evaluation members in public administrations, these provisions give greater prominence to evaluation expertise while allowing methodological competence to be combined with knowledge of the university sector. The members of the University Evaluation Committees are appointed by the university’s governing bodies.

ANVUR provides strong central coordination by defining AVA requirements, issuing methodological guidance, organising external accreditation, and regulating the annual reports produced by Evaluation bodies. The accreditation process also assesses the role of the Evaluation Body within the university and the extent to which the institution follows up on its recommendations. Organisational responses to evaluation are therefore themselves subject to external review.

The documentary analysis of the five selected universities (Table 4) shows both formal recognition of the Evaluation Body and evidence of its recommendations entering institutional planning. In the four universities receiving satisfactory or fully satisfactory judgements, the Evaluation Body is recognised as an important component of governance and quality assurance. Although the documentary evidence varies in specificity, it indicates that recommendations were formally linked to planning, corrective measures, or quality-improvement processes.

Table 4. Recognition and documented uptake of evaluation in the planning documents of the selected universities.

Author’s analysis of the ANVUR accreditation reports and self-assessment documents prepared by the universities (2025).

Bocconi University provides a notable example. Despite its favourable accreditation outcome, the CEV recommended increasing the external membership of its Evaluation Body. At the same time, the Evaluation Body chair sits on the university’s Planning and Coordination Committee, creating a direct organisational interface between evaluation and planning. By contrast, in IUL’s conditional accreditation, ANVUR expressly identified the Evaluation Body’s limited autonomy and weak influence on decision-making as deficiencies.

Accreditation outcomes reflect multiple dimensions, not only the role and effectiveness of the Evaluation Body. Nevertheless, the evidence shows that its institutional standing and the follow-up to its recommendations are considered relevant within the AVA system. The AVA system thereby creates a formal connection between evaluation, planning, and institutional consequences.

5 Discussion

The comparison addresses the three analytical questions set out in the methodological section. Table 5 summarises the evidence for the two systems.

Table 5. Comparative evidence on the institutionalisation, legitimacy, and uptake of evaluation in the two systems.

The first question concerns how institutional design connects evaluation with policymaking. The comparison between the performance evaluation system in the public administration and the quality evaluation system for the universities confirms that national-level measures provide only a partial account of institutionalisation. Although Italy lacks overarching evaluation legislation and a general evaluation authority, both systems are extensively regulated at the sectoral level. Formal institutionalisation, however, is not sufficient to ensure that evaluative knowledge enters decision-making: rules may require evaluation without requiring decision-makers to consider its substantive implications.

This limitation is particularly evident in the public administration performance evaluation system. Evaluation Bodies oversee the performance cycle and may formulate recommendations, but administrations are not normally required to respond or explain how those recommendations inform subsequent planning. Neither the system’s design nor its central coordination establishes a formal feedback loop. Advisory use and an advisory role of evaluators may nevertheless emerge through organisation-specific arrangements, but assessing these practises requires organisation-level research.

AVA presents a different configuration. It combines internal quality assurance, external accreditation, and strong coordination by ANVUR. Crucially, both the operation of each university’s Evaluation Body and the institutional response to its recommendations are subject to external review. Identified deficiencies require corrective action and may have accreditation consequences. The main difference therefore concerns not the amount of regulation but what organisations are required to do with evaluation. The performance evaluation system primarily regulates the production and validation of evaluative information; AVA also regulates organisational follow-up. Although this may encourage ceremonial compliance, it makes the connection between evaluation and decision-making recurrent, visible, and open to external scrutiny.

The second question concerns evaluator legitimation. Evaluation Bodies are formally recognised in both systems, but their legitimacy rests on different foundations. In public administration, membership emphasises independence and experience in performance measurement, planning, and management control. Specific expertise in policy or programme evaluation and in the administration’s policy domains is not required. The resulting profile is primarily managerial and procedural, consistently with the methodological and oversight functions assigned to Evaluation Bodies.

University Evaluation Bodies combine external membership and professional standing with evaluation expertise. Their composition can therefore support both methodological and epistemic legitimation, allowing evaluators to be recognised for their independence as well as their knowledge of evaluation and the university sector. The evidence does not establish that this profile produces greater uptake, since individual competence, reputation, and relational skills were not systematically assessed. It nevertheless appears more compatible with the role of evaluation-use advocate identified by Skolits et al. (2009).

The third question concerns whether recommendations enter organisational planning and decision-making. In the public administration cases, PIAOs consistently recognise the formal responsibilities of Evaluation Bodies and sometimes involve them in planning but provide little evidence that identifiable recommendations have influenced subsequent decisions. The university documents show a different pattern. In four of the five cases, recommendations made by Evaluation Bodies informed planning, quality-improvement measures, or subsequent monitoring, although with varying degrees of specificity. In the remaining case, the Evaluation Body’s weak contribution was expressly identified as a deficiency, and the institution received a conditional accreditation judgement.

Two qualifications are necessary. First, the documents examined in the two sectors serve different purposes. PIAOs are planning instruments and are not specifically designed to report responses to recommendations made by Evaluation Bodies. ANVUR accreditation reports, by contrast, explicitly examine the university Evaluation Body and institutional follow-up. The comparison therefore captures differences not only in use but also in how use is documented. Moreover, the absence of documentary evidence does not demonstrate the absence of influence, since advice may be informal or confidential (Di Francesco, 2000).

Second, documented follow-up does not by itself demonstrate that evaluation has become policy advice. Responses driven by accreditation requirements may reflect compliance rather than recognition of evaluators as advisers. At the same time, evaluation-based advice may be conveyed through informal channels that leave no documentary trace, as is often noted in research on the difficulties of assessing policy advice (Di Francesco, 2000). Uptake should therefore be treated as an indicator of evaluative influence and advisory potential, not as proof of an advisory relationship.

Even with these qualifications, the comparison identifies a meaningful difference. AVA requires universities to demonstrate that evaluative findings have been considered and subjects their responses to external scrutiny. It thus creates an institutionalised feedback loop between evaluation and planning. By contrast, although the public administration system formally recognises Evaluation Bodies, the use of their recommendations depends largely on the initiative and receptivity of individual administrations and on the professional standing of Evaluation Body members.

6 Conclusion

This article has explored the relationship between policy evaluation and policy advice, clarifying their principal areas of convergence and distinction. Drawing on a comparison of two Italian organisational evaluation systems, it has then examined how configurations of institutional design and the formal bases of evaluators’ legitimacy may help account for differences in the documented use of their recommendations in organisational planning and decision-making.

Taken together, these findings indicate that national-level institutionalisation is neither necessary nor sufficient for evaluation to contribute to policy advice. It is not necessary because advisory use may develop through sector-specific institutions and organisational practises even in the absence of an overarching national framework; it is not sufficient because formal requirements may routinise the production of evaluation without ensuring that its findings enter subsequent decisions. Both Italian systems establish Evaluation Bodies and recurrent evaluation processes but differ in how evaluation is connected to planning. In public administration, recommendations are non-binding and no general feedback loop requires organisations to document their responses. Under AVA, the role of Evaluation Bodies and institutional follow-up are themselves assessed during accreditation, making responses more visible, traceable, and consequential.

The documentary evidence reflects this difference. Links between recommendations and planning are rarely documented in the selected public administrations but are identifiable in four of the five university cases. This finding requires caution because the documents serve different purposes and informal influence may leave no trace. What the comparison shows more clearly, however, is that AVA institutionalises the external review of organisational follow-up, whereas the performance evaluation system does not.

Third, evaluator legitimation involves more than formal independence. The university system more explicitly supports a mixed profile, combining evaluation knowledge and epistemic legitimation, whereas the public administration model emphasises independence and managerial or procedural competence. This does not resolve the tension between independence and engagement but indicates that they must be balanced rather than treated as mutually exclusive. These system-level configurations cannot, however, fully explain whether evaluation exerts advisory influence within individual organisations. Such influence may also depend on evaluators’ professional standing, organisational culture, and the organisation’s internal structures and capacities. Although these factors fall outside the scope of the present article, they should be a central focus of future empirical research at the level of individual organisations.

The article contributes to evaluation scholarship by shifting attention from the presence of evaluation requirements to the institutional interfaces connecting findings with subsequent decisions. It contributes to the policy advice literature by identifying evaluators as a relatively neglected category of actor that can contribute to advisory processes. Their contribution is necessarily bounded: evaluation constitutes one source of knowledge within policy advice and does not replace other forms of expertise, experience, or political judgement. Their position between retrospective judgement and prospective recommendation, and between methodological independence and organisational engagement, is inherently ambiguous. This ambiguity identifies the institutional and professional conditions that must be managed if evaluators are to become credible participants in policy advisory systems.

The findings also have implications for evaluation-system design. Reforms should not focus exclusively on additional requirements, indicators, or reporting obligations. The distinction between knowing that and knowing how (Clinton and Hattie, 2021) suggests that technical knowledge must be complemented by contextual judgement and relational competence. Evaluation systems should provide access to planning processes, establish expectations that decision-makers respond to recommendations, and review subsequent follow-up. Evaluation Bodies should also combine methodological competence with relevant substantive expertise, while safeguards preserve their autonomy, transparency, and accountability.

The analysis remains exploratory. It draws on a small purposive sample and primarily on documents that perform different functions in the two systems. It cannot fully reconstruct informal interactions or determine whether documented changes resulted from evaluation rather than concurrent pressures. The findings should therefore be understood as comparative patterns and theoretically informed propositions, not causal claims. Future research could test them across larger samples, additional policy sectors, and other countries, combining documentary analysis with interviews involving evaluators, administrative leaders, and political decision-makers.

The Italian case ultimately shows that the conditions enabling evaluation to function as policy advice must be actively constructed and sustained within sectors. They include governance arrangements connecting findings with planning, evaluators whose legitimacy is recognised, and incentives for substantive rather than ceremonial use. Where these conditions converge, evaluation can become a recognised and consequential voice within policy advisory systems.

Generative AI statement

The author(s) declared that Generative AI was used in the creation of this manuscript. The author used Generative AI assistance (Claude, Anthropic) during the drafting and revision of this article to refine the prose, check the language, and support the retrieval and initial organisation of relevant passages in the selected documents. Identical prompts were applied to all documents, and every passage identified by the AI was manually verified against the original source. AI was not used to generate data, identify sources, or formulate the research questions, hypotheses, interpretations, or findings. All intellectual content, analytical claims, interpretive choices, and bibliographic references remain the sole responsibility of the author.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Andrea Lippi, University of Florence, Italy

Reviewed by: Anna Malandrino, University of Turin, Italy

Giancarlo Vecchi, Polytechnic of Milan, Italy

Osvaldo Feinstein, Complutense University of Madrid, Spain

Footnotes

The documents selected for analysis were uploaded to Claude AI and processed using identical prompts. The system was asked to identify passages referring to the evaluation process or Evaluation Body, the observations and recommendations made by evaluators, and any decisions or initiatives documented as having been taken in response. All passages identified by the system were manually checked against the original documents. The AI output was used only to support the retrieval and initial organisation of relevant material; the coding, interpretation, and comparative analysis were conducted by the author.

The author has professional experience as a member of evaluation units in both public administrations and universities.