A papal address in Paris has pushed an unfamiliar term — ethical discernment — into the middle of the artificial intelligence debate. The claim that judgment must be trained, not just regulated, is worth taking seriously on its own merits.
Key takeaways
- The Guardian reports that Pope Leo, speaking at the Élysée at the start of a three-day visit to France, warned against losing humanity in what he called a “paradise of machines” and said artificial intelligence must remain at the service of people.
- Most existing AI regulation places obligations on the organisations that build and deploy systems, and says comparatively little about how the individuals using those systems should be prepared to exercise judgment.
- Human oversight is written into many AI rules as a safeguard, but oversight only functions if the person doing the overseeing has the training, authority and time to disagree with a machine.
- The strongest objection to the argument is that appeals to conscience are unenforceable and can be used, deliberately or not, as a substitute for binding rules.
The argument: rules bind systems, while the decisive judgments are left to people
The intervention reported by the Guardian is not, on its face, a regulatory proposal. It is a claim about education: that people need to be formed in the capacity to recognise what is morally right, because otherwise the sheer convenience of automated systems will quietly set the terms of daily life. That framing sits awkwardly in technology policy, which is built around measurable obligations — documentation, risk classification, audit trails, transparency notices.
The argument advanced here is that the papal framing identifies a real structural gap rather than merely restating a religious concern in technological language. AI governance as currently constructed regulates artefacts and the organisations that supply them. It does not, and arguably cannot, regulate the thousands of small discretionary moments in which a teacher accepts a generated grade, a clinician accepts a flagged diagnosis, a caseworker accepts a risk score, or a manager accepts a ranked shortlist. Those moments are where an AI system’s output becomes a decision with consequences for a person. Whatever governs them is not a statute; it is the disposition and competence of whoever is sitting there.
That is why moral formation is a governance question and not simply a pastoral one. If the effective decision point has migrated to individual users, then the quality of individual judgment becomes part of the control system, and the institutions that shape judgment — schools, professional bodies, universities, religious communities — become part of the governance landscape whether policymakers treat them that way or not. The claim is arguable, and it is contested. But it is not empty.
Regulation is addressed to providers and deployers, not to moments of decision
The dominant model of AI regulation internationally works by classifying systems according to the risk they pose and then attaching duties to the organisations that place them on the market or put them into use: conformity assessment, documentation, incident reporting, transparency towards affected people. This is a familiar and reasonable design. It mirrors how product safety, financial services and medical devices are regulated, and it has the great advantage of being enforceable against entities with assets, lawyers and compliance departments.
What it does not do is specify how a particular person should weigh a particular recommendation on a particular afternoon. Regulation can require that a human be able to intervene; it cannot supply the professional confidence required to intervene. Regulation can require that users be told they are interacting with a machine; it cannot determine what they conclude from that disclosure. The rules end where the interpretive work begins.
This is not a flaw unique to AI law. Medical ethics, legal ethics and engineering ethics all exist precisely because codified rules run out before hard cases do, which is why those professions pair regulation with lengthy formation in judgment. The novelty with AI is the speed and breadth of the deployment, and the fact that many of the people now making machine-assisted decisions belong to no profession with an ethics tradition of its own. The gap between rule and judgment is therefore wider and more thinly staffed than in the older regulated fields.
Human oversight only works if the overseer is equipped to disagree
Human-in-the-loop requirements are among the most common safeguards in AI policy, and they depend on an assumption that deserves scrutiny: that a person reviewing an automated output will meaningfully evaluate it. Decades of human factors research on automation describe the opposite tendency. When a system is usually right, reviewers begin to defer to it; vigilance decays; the reviewer’s role slides from evaluation towards ratification. The phenomenon is well documented in aviation, industrial control and clinical decision support long before generative AI arrived.
The practical consequence is that oversight is a skill and a set of working conditions, not a checkbox. It requires the reviewer to understand the system’s failure modes well enough to suspect an error, to have the standing within the organisation to override it without being second-guessed, and to have enough time in the workflow to look properly. Where any of those is absent, a formally compliant oversight mechanism can produce automated outcomes with a human signature attached.
This is the point at which the education argument becomes concrete rather than devotional. “Ethical discernment” in this setting means something fairly specific: the trained capacity to notice when a fluent, confident output is nonetheless wrong or unjust, and the disposition to act on that noticing. That capacity is built through curriculum, professional norms and institutional culture. None of those appear in a risk classification table.
Institutions that form judgment are already embedded in education and care
There is a second, more prosaic reason the intervention matters: the institutions raising this argument are not only commentators. The Catholic Church operates one of the world’s largest non-state networks of schools, universities, hospitals and social services, alongside other religious bodies with comparable reach. These are precisely the settings where AI systems are now being introduced into consequential decisions about learning, health and welfare, and they are also settings with existing mechanisms for teaching ethical reasoning.
That gives statements of this kind a plausible route to implementation that a general appeal to public conscience lacks. Curricula can be revised. Professional formation can include the specific competencies of working alongside statistical systems. Procurement decisions within large institutional networks can be conditioned on explainability or on preserving human authority in particular judgments. Whether any of that will happen is not known, and no such programme is described in the material reported from the Paris speech.
The broader pattern is that religious and philosophical institutions have been positioning themselves as participants in AI governance debates rather than observers of them, and the Vatican in particular has issued doctrinal reflections on artificial intelligence and human dignity in recent years. The details of those documents are beyond the scope of what can be verified here, but the trajectory is consistent: a claim to standing in a debate that has been dominated by engineers, firms and regulators.
The strongest objection: conscience is unenforceable and can crowd out rules
The most serious case against this argument is that it asks an unenforceable mechanism to carry weight that only enforceable mechanisms can bear. Moral formation has no audit trail. It cannot be inspected, cannot be sanctioned, and produces no remedy for a person harmed by an automated decision. A worker with impeccable ethical training and no job security will still approve the shortlist. Framing the problem as one of individual virtue locates responsibility in the least powerful actor in the chain — the user — while the design choices that created the risk sit with the developer and the purchasing organisation.
There is also a documented pattern in technology policy whereby ethics discourse functions, whatever its authors intend, as a delaying tactic. Voluntary principles, ethics boards and educational commitments have repeatedly been offered by industry at moments when binding rules were under discussion, and they have sometimes succeeded in postponing those rules. A high-profile moral appeal can be absorbed into that dynamic regardless of the sincerity behind it.
A further objection concerns legitimacy. Deciding what counts as morally right in contested areas — reproductive health, end-of-life care, sexuality, family structure — is exactly where religious ethical frameworks and secular pluralist ones diverge most sharply. A call for education in ethical discernment leaves open whose account of discernment is taught, and in pluralistic democracies that question is not incidental. Critics can reasonably hold that the state’s business is enforceable rights and remedies, not moral formation, and that conflating the two weakens both.
What evidence would change this conclusion
The argument would be substantially weakened if AI systems in high-stakes settings proved reliably governable through technical and legal means alone — if, for example, auditing regimes and liability rules demonstrably reduced harmful automated decisions without any change in the training of the people involved. Evidence that oversight requirements work as intended even where reviewers lack specialised preparation would undercut the central claim.
It would also be weakened by evidence that ethics education does not transfer to behaviour under organisational pressure. If studies of professionals trained in AI ethics showed no difference in their willingness to override machine outputs when deadlines and management expectations pushed the other way, then formation would be revealed as a poor substitute for structural safeguards such as staffing levels, liability exposure and the right to refuse.
Conversely, the argument would be strengthened by the appearance of concrete programmes rather than declarations: institutional networks changing curricula, professional bodies defining oversight competencies, or procurement standards that require preserved human authority. Whether the Paris intervention is followed by anything of that kind is not yet known.
Sources and further reading
- The Guardian — reported the pope’s remarks at the Élysée and the framing of the three-day visit to France; the only source for the quoted language used here.
- Holy See doctrinal and academic publications — the Vatican’s own documents on artificial intelligence and human dignity, useful for the underlying reasoning rather than the news event.
- European Union AI legislation and accompanying guidance — the primary reference for how risk-tiered obligations on providers and deployers are actually structured.
- Human factors and automation research literature — established work on automation bias and complacency in aviation, industrial control and clinical decision support.
Surfaced from the rss:guardian_tech signal “religious leader on AI ethics”. AI-assisted draft, editorially reviewed.
