What can AI really do for public action?

Since the emergence of ChatGPT and Midjourney a few months ago, artificial intelligence has taken centre stage in the media and on the political agenda. Caught between the promise of a new industrial revolution and fears of the 'great replacement' of jobs - or even of all human activities - it gives rise to hopes, fears, fantasies and calls for reason in equal measure; all of which are naturally reflected among public sector stakeholders.

So what can and must (or should) AI contribute to public policy? How can we assess its appropriate use, particularly in terms of efficiency and performance, but also in terms of moderation and social justice? What are the risks associated with its use by public sector stakeholders, and how can these be prevented or regulated? How can we move beyond a purely technological perspective in favour of a political and managerial one?

It was in an attempt to answer these questions that, on 5 April, we organised a webinar attended by over 70 participants, the majority of whom were from our member local authorities (and, interestingly, only a minority were IT directors!), centred on a presentation by Régis Gabriel, Director of the Urban Cleanliness Division at the City of Metz - who is leading a pilot scheme designed to fit within the operational approach and the logic of continuous service improvement - and fascinating talks by two researchers and designers, Pauline Gourlet and Estelle Hary, designers and researchers, who both take a critical and practical approach to the aims, methods and impact of working with AI.

To set the scene, we'll begin by outlining the issues raised by AI in local authorities, as we gathered them whilst preparing for the webinar:

• What awareness-raising and regulatory measures should be put in place internally? What policy framework and internal guiding principles should be established? What national strategy or collaborative approach is required?
• What impact will this have on, or how will it transform, professional roles? What skills and level of expertise are required to oversee these applications? (In relation to staff training)
• What are the ethical challenges surrounding decision-making (accountability, transparency, data protection, fairness (bias) and sovereignty)? How can these technologies be used sustainably, within a context of digital restraint?

The needs expressed therefore fall into three categories: clarification (what is AI?), insight into the issues (risks, uses), and the sharing of case studies (ongoing experience, real benefits of AI).We also discuss the unconsidered issues that emerged during our preparatory interviews: the lack of understanding of how AI is actually used in staff members' day-to-day work; the question of costs (financial, energy and human); the question of reliability, the actual performance of these technologies and how this is measured; and the relevance of AI in terms of solving a public problem.

Régis Gabriel begins by setting out the context of the ViPARE project, a data-collection system designed to analyse cleanliness and hygiene in towns and cities, incorporating artificial intelligence that can recognise litter in a video stream; It should be noted that the project was selected in a France 2030 call for proposals to support low-cost AI demonstrators serving local areas, and that it is being shared within the Association of Cities for Urban Cleanliness (AVPU) to enable other local authorities to adopt the tool in the future. As the ways in which public spaces are used evolve, urban cleanliness faces new challenges and must increasingly take into account factors such as seasonality, neighbourhood dynamics and user behaviour. The issue of objectively defining and assessing cleanliness criteria is becoming increasingly important and is primarily addressed through the counting of litter.

The start-up NAIA Science has therefore proposed to Metz City Council a tool for identifying litter in natural environments, developed in collaboration with the NGO Surfrider. A dialogue has been established to refine and adapt this technology to a practical, operational approach (who will take the photographs? How? What can and cannot be expected of street cleaning staff? - for example, no walking down the street with a pole held above one's head to take photos!) and a principle of frugality (a solution capable of running on a cleaner's mobile phone), whilst considering how it would integrate with the organisation and existing tools (objective cleanliness indicators/operational assessment grids). This process helped to clarify the needs the solution had to address: to automate and simplify data collection by staff (currently, one staff member is assigned, for 25 per cent of their working time, to count litter in a small sample of streets); to characterise litter in greater detail and geolocate it more precisely; and to expand the scope (roads, pavements, green spaces). AI should therefore help to reduce the time taken to collect and process data, whilst increasing the areas covered and the representativeness of the data, leading to a more detailed mapping of problem areas. The ultimate aim is to adapt and improve waste collection routes ("cleaning where it is actually dirty, not where there are the most complaints"), but also to better target awareness-raising or enforcement measures to change the behaviours that cause litter. The project also incorporates a citizen science dimension, with the Water and Environment Laboratory at Gustave Eiffel University in Nantes having joined the consortium to work on models of waste dispersion in urban areas, particularly via water networks ('from the gutter to the watercourse'), and to improve the functioning of sewerage networks, by involving students and members of the public.

For Régis, we need to dispel the myths surrounding AI: “VIPARE remains a tool that uses AI to improve and evaluate a business activity. We will always need a staff member to answer the phone, take the images and correct any recognition errors. The technology will not replace our cleaning roles, but will make them more user-friendly, objective and efficient, as part of a process of continuous improvement.” He also explains that the tool is so easily adopted by staff because of its ease of use, its stability (prerequisites he has set) and the fact that it simplifies their work and saves them time.

Régis's presentation is available here.

Pauline Gourlet, a designer and research associate at the Sciences Po Médialab, begins by questioning the uncontroversial use of the term 'artificial intelligence'. Pointing out that there is no scientific consensus today (nor has there been since its early developments in the 1950s) on the definition of AI (particularly regarding the distinction between algorithms and artificial intelligence), Pauline explains how, as part of the international research project Shaping AI, she focused specifically on how the concept has been portrayed in the press over the last 10 years. Spanning a wide variety of sectors and four types of narrative (abstract critiques, broad promises, promising experiments, local controversies), these narratives have played a major role both in shaping public perceptions (whether doomsday scenarios or techno-optimism) and in legitimising the development and integration of new computational systems. More recently, new frames of reference have emerged alongside new problems, such as racial bias in facial recognition, the fact that AI models have been trained on data without the consent of its owners, and the use of these technologies for political interference.

For Pauline, the use of this vague concept of AI primarily serves an 'economy of promise' (Silicon Valley's famous fake it till you make it), driven by the quest for new investment and the recruitment of an ever-growing number of players, without always providing evidence to back up its claims. Furthermore, the model/software, presented as central to AI, is in reality embedded within a very broad socio-technical ecosystem - a chain incorporating entities that are rendered invisible (from electronic waste to water, or underpaid 'click workers'), which are not perceived as stakeholders in the system even though they are indispensable to it. It is in order to better understand the system, identify levers to influence future developments and move beyond the monopolisation of the debate by technical experts alone that Shaping AI has produced a map of the concerns of AI practitioners in France.

Far from the widely held belief that it is necessarily a source of simplicity, efficiency or cost savings, AI in the public sector faces all these challenges and problems, as Pauline demonstrates through the 'Foncier innovant' case study. Launched in 2017 by the Directorate-General for Public Finances (DGFiP), this system detects undeclared swimming pools by identifying them in aerial images, with the aim of eventually automating the recording of all built structures on cadastral maps. By bringing into view all the components of the chain (databases, servers, regulatory texts, stakeholders, digital interfaces, funding, workers, etc.), this participatory inquiry revealed the wide range of 'concerns' held by practitioners engaged with the components of this 'new socio-technical arrangement' and highlighted the political perspectives underpinning its roll-out.

The investigation shows that the initial promises have not been fulfilled: the new system is not simpler - far from it - and, to date, is not even effective. One aspect, already well documented in other contexts and which is evident here, concerns the fact that work does not disappear: it transforms (sometimes moving abroad to 'click farms') and relocates, leading in the process to the devaluation, precariousness and demotivation of workers. Far from being limited to models, this case shows that 'AI' should be understood and assessed in terms of the reconfiguration of social relations that these new arrangements bring about.

Changes in staff working practices are, of course, a key aspect of this study, but Pauline concludes by also highlighting a number of other issues which she considers crucial, and which have not been given sufficient consideration in the evaluation of the computational systems put in place:

• The resources (materials, energy, water) required to produce and operate these systems, and the significant CO₂ emissions
• The push to exploit government data (for its own sake, outside the original purposes and the context in which it was produced), which is primarily used to promote commercial innovation.
• The loss of expertise and/or motivated staff with knowledge critical to the proper functioning of public services.
• Participation (by public sector staff and, more broadly, by citizens) in the development of public policy instruments
• The consequences for the relationship between citizens and public administrations (beyond the issues of exclusion and precariousness linked to digitisation, the question of fairness in dealings with the administration arises).

Pauline's presentation is available here.

Estelle Hary, a designer and co-founder of the studio Design Friction, a PhD student at RMIT University and affiliated with the Centre for Design Research, focuses more specifically on generative AI, which falls within the field of machine learning - a field she defines as "algorithms that extract forms of statistical truth and reapply them to other data". The public sector is currently taking a particular interest in LLMs (Large Language Models), which, as Estelle points out, produce results that are probabilistic rather than absolute truths (to find out more about the probabilistic model and its implications, listen to this episode of Code a changé!)

The implementation of generative AI in public policy first raises the question of access to and use of data (without data, there is no AI!), with four possible scenarios:

• The data is accessible and usable (for example, to train the LLM Albert, DINUM uses the records from service-publics.fr, which are open data and in a format that is easily machine-readable).
• The data is accessible but not directly usable (such as the information available on government websites). It is therefore necessary to carry out preliminary work to retrieve and validate the information in order to be able to use it.
• The data is not accessible but is usable, for example data held by other government bodies. This raises the question of data sharing between government bodies, a process that is intended to be facilitated by the 3DS Act adopted in 2022, the spirit of which is in line with the 'Tell us once' initiative.
• The data does not exist; it must either be created or the AI project abandoned.

In three out of four cases, it is therefore necessary to put new technical and organisational processes in place and draw on specialist expertise, such as data scientists to analyse the data and develop AI models, or lawyers specialising in digital law to ensure that data processing complies with regulations. These are skills that public authorities do not always have at their disposal.

Another significant issue relating to data is that of its scope and maintenance, which has a direct impact on the relevance of the results produced by generative AI. Albert, for example, which is currently being trialled in France Services centres, did not, at the time of the webinar, have any information on schemes specific to each territory (regions, departments, etc.), as no data relating to these was included in the model's training dataset, even though such information could be useful to France Services advisers. There is also the question of maintaining and updating this data over time, which poses a real technical and organisational challenge given how rapidly administrative and legislative matters evolve. Will it be necessary to make LLMs 'unlearn' one set of knowledge in favour of another every time a scheme changes? And how can this be done in practice?

Estelle also discussesthe impact of generative AI on work, using the example of advisers at the Maisons France Services, who assist users with various administrative procedures and are therefore required to consult, compare and summarise different sources of information. Using Albert, the adviser asks the tool a question directly and then passes on the generated response to the user. This change in practice raises several issues:

• learning how to use the LLM tool, which does not work like a search engine and requires users to formulate the right question (the prompt)*, as the way the question is phrased affects the quality and relevance of the response (this learning process must be repeated with every model update);
• the need to maintain trust and critical thinking when faced with responses generated by LLMs, which are not always reliable. Whilst these responses may seem plausible, they may contain omissions or 'hallucinations' (the invention of facts or information)... Thus, as part of the trial, France Service advisers are encouraged to use the tool for tasks or questions they are familiar with so that they can verify the accuracy of the responses, but what would happen if this were not the case? In particular, a mechanism for citing 'sources' (service-public.fr factsheets) is being tested so that advisers can quickly verify the information. Other ideas under consideration include introducing a confidence score for each response or allowing Albert to say 'I don't know'.
• the necessary 'feedback' on the quality of the answers provided by Albert, to help improve it. Indeed, the improvement of LLM models relies largely on feedback from their users (reinforcement learning from human feedback - RLHF), which amounts to generating evaluation data for the responses produced, via the agent's feedback on the perceived quality of the response (in this case, a '+' or '-' click). This is yet another example of the shift in work associated with AI. Sometimes, this data generation even requires the creation of a dedicated service, such as the one set up by the Court of Cassation to verify the pseudonymisation of court rulings by an algorithm. Estelle therefore urges public authorities wishing to develop AI to consider what impact this will have on existing roles, and what new tasks and roles it will require ... For example, the IGN, which has embarked on a major aerial image recognition project, highlights the risk of not having trained enough staff (and not having anticipated this training need) to recognise trees via remote sensing and annotate the images with the correct information.

She concludes with a broader reflection, particularly on the ethical issues surrounding the use of AI by public services: the probabilistic logic of an LLM, which may provide different answers to different people in response to the same question, could undermine the principle of equality before public services; and what about the responsibility of public authorities for what is generated by an LLM: for who is responsible in the case of a chatbot that contributes to an administrative decision or explains to a user how to circumvent the law or avoid paying tax (as a New York City chatbot recently did)? On this last point, are we not seeing, with AI, a reversal of the burden of proof, whereby it falls to the member of the public to prove that the administration is wrong, whereas previously it was up to the administration to justify its request?

*We'd also recommend this interesting article on ways to hack AI systems using these so-called prompts.

The presentations sparked numerous reactions, starting with a call to map out the controversies surrounding AI and/or the areas it covers, so that everyone can feel empowered to engage with the subject and move beyond black-and-white or ideological perspectives. Several participants also raised questions: for whom does AI bring progress? Who benefits from it? What injustices does it create? These are all criteria that are never taken into account when assessing these systems today.

Pauline calls for a return to the original spirit of participatory design – that of genuine participation, which takes as its starting point the needs identified in staff members’ day-to-day work, as was the case in Metz. Conversely, the development of ‘Foncier innovant’ took place without involving the staff concerned, and despite the promise not to replace full-time equivalent (FTE) staff, there has been a shift in the workload: on the one hand, towards annotation (which is often carried out abroad); and on the other, towards users, who are now taking on part of the work previously carried out by the administration.

Beyond its promises, AI therefore appears today to raise political questions and well-known issues relating to change management, though these seem to go largely unheard in a climate where experts in technology have monopolised the public discourse. Similarly, the sustainability of these systems (particularly with regard to water resources), the risks of which we are in principle able to assess, is not being questioned. As Estelle puts it, "we are setting up projects for which we will not have the necessary computational capacity due to a lack of resources" ... On the environmental cost of AI, we recommend this enlightening article or this book, which focuses on mining issues.

To round off, here are a few Kafkaesque ideas we picked up from the webinar chat: should we invent a role for an AI fitness coach? Should we use AI to produce complex funding application dossiers (so complex that there are sometimes already AIs designed to process them!)? Will users have to take screenshots throughout the process to demonstrate their good faith in the event of an error, in a system where probabilistic statistics are seen as all-powerful?

Friends of local authorities, are you interested in the issues raised in this webinar? If you'd like to set up a pilot project on this topic, please get in touch with us at crotrou@la27eregion.fr!

*****************************************************************

As a bonus, here is the video recording of the webinar made by our colleague Martin Préaud, project manager for regional innovation at the Seine-Saint-Denis Department (thank you, Martin!)