Public sector innovation: what if we "unfolded" evaluation?

View larger

A year ago, we wrote a post (external link)explaining why La 27e Région had renewed its interest in evaluation and how we are now striving to embed an "evaluative mindset (external link)" (see Quadrant Conseil) into our programs and general activities. Since then, we have also been observing developments within our networks; we were struck to see certain local authorities implementing highly innovative practices-often without framing them as evaluation initiatives, but rather as tools for steering, strategy, collaboration, or capacity building. To discuss this, we brought several of these authorities together last January (*). Here, we recap the key points from that discussion, while attempting to bring some order to the ideas shared.


(*) The metropoles of Grenoble, Nantes, and Lille; the cities of Nancy and Malaunay; the Occitanie Region; and the DITP-with the expertise of Quadrant Conseil and the University of Paris-Créteil.


A vocabulary challenge

Evaluation often carries negative connotations, yet several shifts in terminology could help us broaden our understanding of the concept. First, rather than simply speaking of "evaluation," some authors now refer to "evaluation systems (external link)" to move beyond a "one-shot" perspective and highlight the process's continuous, systemic nature. Others use the term "Monitoring, Evaluation, Accountability, and Learning" (SERA) (external link)-an approach that emphasizes functions that, while inherent to the process, are often overshadowed in evaluation practices:

  • Monitoring is the ongoing process of systematically collecting information-based on selected indicators-to provide managers and stakeholders of an active development initiative with data on progress made, objectives achieved, and the use of allocated funds;
  • Public policy evaluation, strictly speaking, "means forming a judgment on its value," to quote the Viveret report (1989);
  • Accountability is the obligation to report clearly and impartially on results and performance in relation to the mandate and/or established objectives;
  • Finally, learning is a process of acquiring knowledge and skills, but also a perpetual process-a continuous drive for improvement that relies on the ability to identify errors.

Viewed in this light, evaluation once again emerges as a comprehensive and supportive practice, capable of helping organizations clarify their intentions, gather data to assess their effectiveness, monitor and adjust their levers of change, and gauge their capacity to respond and adapt in real time.


Conceptualizing evaluation starting from its uses

At La 27e Région, we returned to evaluation not out of compulsion, but by listening to our immediate needs-specifically, the need to move beyond the vagueness surrounding innovation and the risk of "innovation-washing," as well as to examine the difficulties we faced in designing our programs. Evaluation is best conceived based on the intended uses! But how can we gain clarity on the possible uses? In 2022, to move beyond a technical debate on evaluation, the French Development Agency (AFD), Quadrant Conseil, and Stratégie Design Scénario published a map of evaluation uses. (external link) This map is organized around four categories of use: strategic, steering-oriented, continuous improvement, and dialogue-oriented. Methodological proposals are provided for each category.

Use case: evaluation supporting management at Nantes Métropole

A first example allows us to view evaluation through the lens of strategic steering. How can a common framework be established for the territorial strategy (driven by political leadership) and the local authority's own organizational plan (its internal counterpart)? How can priorities be clarified and stakeholders united around a shared vision of public action? In 2021, the Nantes Metropolis adopted six "markers"-essentially brief reference documents detailing the principles the Metropolis champions across six cross-cutting priorities, which are then integrated into the authority's 26 public policy areas: ecological transition, equality, citizen engagement, local proximity, innovation and experimentation, and territorial partnerships. Each marker goes beyond mere statements of intent, having been translated into highly operational terms. Now that these guiding principles are in place, the next step is to evaluate existing initiatives against these markers-ensuring the organization stays on course.

D’autres exemples permettent d’envisager l’évaluation comme outils de pilotage :

  • Les marqueurs de Nantes sont inspirés de la théorie de changement, « une vision explicitement documentée (et donc évaluable) de la façon dont on pense que le changement doit se produire » (Rick Davies). Pour réfléchir à sa théorie de changement, la chercheuse Lindsay Cole propose de compléter cette fleur, individuellement puis en équipe. 
  • La métropole de Grenoble a établi un « arbre des objectifs », né en 2015 d’une volonté de « dé-siloter » son fonctionnement, aujourd’hui intégré dans le dialogue de gestion ; Il sert tout autant à rendre explicite la déclinaison de grands enjeux en réalisations concrètes, de clarifier ce qui fait politique publique, qu’à piloter l’action publique en servant de repère et de support de dialogue dans chaque service. Cette vidéo décrit son fonctionnement.

Autre cas d’usage : l’évaluation au service de l’amélioration continue à Nancy

En partenariat avec une équipe de recherche de l’ERPI, la Ville de Nancy a imaginé un atelier d’auto-évaluation des pratiques professionnelles des agent.e.s pour en déduire la capacité d’innovation d’une organisation, d’une équipe, ou d’un projet. Grâce à une grille de questions qui en appellent à des pratiques quotidiennes, le niveau de maturité est défini par des preuves tangibles, autour desquelles se construit dans un second temps un plan d’action pour évoluer et gravir quelques paliers de maturité. L’objectif de ce plan d’action est d’identifier des leviers actionnables à court terme, qui représentent peu d’efforts, et des besoins qui nécessiteront l’option « pente douce », pour une montée en maturité dans le temps long.

Other examples illustrate how evaluation can serve as a management tool:

  • The "Nantes markers" draw inspiration from the theory of change (external link)-"an explicitly documented (and thus evaluable) vision of how change is expected to occur" (Rick Davies). To reflect on their theory of change, researcher Lindsay Cole suggests filling out a "flower" diagram (external link), first individually and then as a team.
  • The Grenoble metropolitan area created an "objectives tree (external link)" in 2015, driven by a desire to break down operational silos; it is now integrated into the organization's management dialogue. It serves to articulate how major strategic goals translate into concrete achievements and to clarify the nature of public policy, while also guiding public action by acting as a reference point and a basis for discussion within each department. This video explains how it works.

Another use case: evaluation driving continuous improvement in Nancy

In partnership with a research team from ERPI, (external link) the City of Nancy devised a workshop for staff to self-assess their professional practices, thereby gauging the innovation capacity of an organization, team, or project. Using a questionnaire based on daily practices, maturity levels are determined through tangible evidence; this evidence then forms the basis for an action plan designed to advance the organization through successive stages of maturity. The plan aims to identify short-term, low-effort levers for action, as well as needs requiring a "gentle slope" approach-a gradual progression toward greater maturity over the long term.

Other examples illustrate the use of evaluation for continuous improvement purposes:

These examples offer an initial glimpse into the diverse ways evaluation can be used. What about you-how do you use it, or how would you like to use it?