Claude Code moves fast at ideation, screen mapping, scaffolding, and boilerplate generation, and developers have shipped real apps this way. But at the back of the development life cycle, where compliance, privacy disclosure, security architecture, and App Store submission live, the human cost reasserts itself. Studies of AI-generated code indicate a significant share requires refactoring to meet Apple’s accessibility and performance standards, and a meaningful fraction of AI-driven apps fail review due to privacy or design violations.

A little more than three years ago, I began developing the AI Life Cycle Core Principles (AILCCP). This is a framework—and now an app—that organizes AI development and deployment obligations across 37 principles, 10 development phases, and 48 controls, mapped to international standards and regulatory enforcement contexts. It gives developers, deployers, lawyers, and policymakers a shared vocabulary and methodology for assessing where an AI system meets its obligations and where it falls short across the development and deployment life cycle. I use it to granularly analyze things like AI legislation, policies, AI vendor agreements, AI governance documents, and questions such as whether Claude Code is AGI. The AILCCP contains 37 principles, each with multiple requirements. Three principles apply most directly here: Wherewithal, Human-Centered, and Workforce Compatible. For each, I focus on the requirements most relevant to what the App Store test exposes, then apply it to Claude Code.

Wherewithal asks whether the capability matches what is being claimed. The enthusiasm around AI coding tools has generated claims that Claude Code can take a developer from idea to shipped app with minimal effort. That framing describes the front of the life cycle accurately and the back poorly, and developers who plan around it will discover the gap at exactly the point where it costs the most to close.

Human-Centered requires human-in-the-loop oversight at the pre-deployment review and deployment phases. Those are the phases where an iOS app is tested against Apple’s privacy guidelines, where data handling disclosures are drafted and verified, where security architecture is stress-tested, and where the submission package is assembled and submitted for review. Claude Code does not do those things independently. A developer who has moved quickly through scaffolding and code generation arrives at those phases with the tool’s momentum behind them and its limitations fully exposed.

Workforce Compatible asks whether an AI tool builds human capability or displaces it. A developer who uses Claude Code to generate iOS code throughout a project never learns iOS development. They learn to prompt. When the tool produces architecturally flawed code, which it does with some regularity, the developer has no independent basis for catching the error. They are dependent on the tool to identify problems that the tool created. That is not augmentation. It is a different kind of dependency, and it grows less visible the more the tool appears to be working.

Claude Code is powerful, without a doubt. But a phase-specific competence at a high level is not what “G” in AGI means.

I want to set aside the image and focus on the finger-pointer, because what the act of identification reveals is more interesting than the image itself.

Consider what was not contested. The airman’s rescue happened. The emotion expressed by millions of people who engaged with the image was genuine. No one who shared it claimed it was a photograph taken by a photojournalist embedded with the rescue team. Most people who encountered it likely experienced it the way they experience a commemorative illustration, as a visual token for something that actually occurred.

Now consider a cartoon. Suppose someone had drawn the same scene, soldiers in a helicopter, smiling, American flag in hand, in the style of a tasteful editorial illustration, the fact-checkers would have had nothing to say. The drawing would have traveled the same emotional circuit. The soldiers would have been the same soldiers. The rescue would have been the same rescue. The difference between the cartoon and the AI-generated image is purely procedural. The AI image was generated by a statistical model trained on visual data. The cartoon was generated by a human hand trained on visual instruction. In both cases, no camera was present. In both cases, the image is a representation, not a document.

Yuval Noah Harari argued in Sapiens that the human capacity for shared fiction, for constructing and inhabiting stories that are not literally true in a documentary sense, is the source of civilizational cohesion. The story of a rescued soldier, expressed in an image that was never a photograph, is doing exactly this work. It is binding a community around a shared recognition of something that happened and matters. The finger-pointer, by flagging the image’s generative provenance, is not adding epistemic content. The finger-pointer is asserting a procedural standard as a substitute for engaging with the story. The question “is this AI-generated?” has displaced the question “is this meaningful?” and the displacement is being performed as though it were a contribution to public discourse.

What the finger-pointer is actually doing is performing epistemic status. The detection requires no expertise, but deploying it produces the appearance of rigor: I saw through this, I identified the error, I am the one who knows. This is not fact-checking in any meaningful sense. Fact-checking interrogates claims and the claim here, that American airmen were rescued, is true. What is being fact-checked is the artwork.

This particular form of intervention will become self-obsolete. The precedent is already visible. When Photoshop entered the visual commons in the 1990s, “it’s been Photoshopped” carried the same accusatory charge the AI flag carries today. The charge faded because it became universal. Sharpening, cropping, color grading, exposure correction, skin retouching, background removal, these are now understood as the ordinary conditions of professional image-making, not deviations from it. Nobody pauses before a magazine cover to announce that the photograph has been post-processed.

As generative models improve and AI-generated imagery saturates the visual commons, the identification will carry decreasing signal. When every image could be AI-generated and many will be, announcing that a specific image is AI-generated will produce the same information as announcing that a specific sentence was typed on a keyboard.

With the release of the AILCCP Explorer, an interactive web application that makes the full framework navigable and searchable, this felt like the right moment to revisit what the AILCCP is, how it works, and why it is built the way it is. 

The AILCCP is a structured knowledge graph that connects existing principles, controls, international standards, life cycle phases, and identified risks into a single navigable structure with over 500 explicit cross-references. The ambiguity is extinguished.

The AI Governance Problem

ISO/IEC 42001 addresses AI management systems. The NIST AI Risk Management Framework maps risk categories and profiles. IEEE has published standards addressing algorithmic bias, transparency, and system design. The EU AI Act imposes risk-based obligations with enforcement teeth.

But these instruments do not talk to each other. Anyone building, deploying, procuring, or auditing an AI system today must reconcile guidance from them, map their practices to regulatory expectations that often vary by jurisdiction, culture, and produce documentation that satisfies reviewers.

What the AILCCP Is

The AILCCP is a cross-linked knowledge base built from five components: principles, controls, standards, life cycle phases, and risks. Each one connects to the others through explicit, traceable links.

The framework is built on 37 principles, most of which were distilled from international consensus such as the OECD, UNESCO, G7, G20, and APAC. The AILCCP gives each one a defined scope, an objective, and measurable outcomes so that stakeholders working with different source standards are looking at the same thing. Governance follows an AI system from the first scoping decision through operational monitoring to eventual retirement.

The Architecture

37 Principles

Each principle includes a short definition, a detailed definition, an objective statement, key questions, suggested controls, required evidence artifacts, and identified stakeholders. The principles are organized across 15 categories and mapped to 10 pillars that span Oversight and Accountability, Reliability and Robustness, Transparency and Explainability, Ethics, Fairness and Equity, Privacy and Consent, Safety and Security, Human-Centered and Workforce concerns, Data and Process stewardship, and Organizational Capability.

Every principle includes a rationale explaining why it belongs in the framework. When stakeholders adapt the framework to their context, the rationale helps them decide which principles matter most for their system.

48 Controls

Controls are the “how.” Each is defined by name, domain, function, and rationale, and each maps to its top three principle alignments. Across the full set, this produces 187 control-to-principle links. Every one of the 48 connects to at least one principle, ensuring that implementation guidance always traces back to a governance commitment.

But here is the thing: controls that exist in isolation, disconnected from the principles they are meant to serve, tend to break down. When a control has no explicit link to a principle, stakeholders struggle to explain why they are implementing it, auditors have difficulty assessing whether it is sufficient, and the control becomes a compliance artifact rather than a governance mechanism.

43 International Standards

The framework maps 43 standards from IEEE, ISO/IEC, and NIST, each with a scope statement, summary, intended use, and identified primary users. Each standard maps to up to five principles, generating 215 standard-to-principle links that touch 29 of the 37 principles.

The 43 standards were selected because they are actionable and recognized across regulatory and audit contexts. Standards are increasingly taking on weight, legitimacy, and force, recognized by legislators, regulators, courts, and the broader developer and implementer ecosystem. When the question is “show me the controls for data governance,” the answer has to trace to standards that carry that weight.

10 Life Cycle Phases

The life cycle spans ten phases, from Scoping and Design through Decommissioning and Archiving. Each phase identifies default owners (Product, Legal, ML Engineering, SRE, and others), expected evidence artifacts, and measurable metrics. Across all ten phases, 84 phase-to-principle links map governance commitments to specific moments in the system’s life.

Each link comes with a life cycle signal that includes a rationale explaining why that principle matters at that stage. Transparency, for example, means something different during Operations and Monitoring than it does during Scoping and Design.

Scoping and Design tracks requirements coverage percentage and reading level targets. Data Preparation tracks missing and invalid data rates, label agreement scores, and PII leakage tests. Evaluation and Red Teaming tracks bias delta, attack success rates, and coverage percentage. Operations and Monitoring tracks mean time to repair, drift alerts per month, and SLO attainment. Instead of “monitor for bias,” the framework says “measure bias delta during Evaluation and Red Teaming and track drift alerts per month during Operations and Monitoring.”

18 Identified Risks

The risk layer assesses 18 identified risks for severity and likelihood using a qualitative rubric tied to the five pillars. Seven are rated Very High severity, eight High, and three Medium. These risks generate 23 links to standards and touch 24 of the 37 principles, connecting the threat landscape directly to the controls and standards that address it.

As I see it, one of the more distinctive ideas in the framework is the “enabling risk” concept. The three risks rated Medium severity are transparency and explainability gaps that function as force multipliers for other, more serious harms. A system that lacks Explainability makes every other harm harder to detect, harder to diagnose, and harder to remediate. This layered thinking about risk cascades reflects how AI breakdowns actually propagate in practice.

The Cross-Link Network

In total, the framework contains over 500 explicit links. 187 control-to-principle. 215 standard-to-principle. 84 phase-to-principle. 23 risk-to-standard. Pick any entry point and trace a path to every other part of the framework.

What Sets the AILCCP Apart

Bidirectional Traceability

Most governance frameworks are organized top-down. The NIST AI RMF flows from four functions (GOVERN, MAP, MEASURE, MANAGE) down to categories and subcategories, but provides no built-in path from a risk finding back to the relevant activities and standards. ISO/IEC 42001 follows the Annex SL hierarchy common to ISO management standards, with 42 control objectives that trace from clauses downward, but the reverse mapping is left to the implementing organization. The OECD AI Principles offer five principles and five policy recommendations with no controls, no life cycle phases, and no risk mappings at all. In each case, the framework is organized in one direction.

A diligent team can reverse-engineer any of these frameworks. But the AILCCP builds the reverse paths in. Its 500+ explicit cross-references mean a user can start from a risk and trace to the standards and principles that mitigate it, start from a standard and see which principles it supports and which life cycle phases it touches, or start from a life cycle phase and see what should be measured, who owns it, and what evidence needs to be produced.

An auditor starts with a finding, a development team with a life cycle phase, a regulator with a risk. The graph accommodates all of them.

Ownership Built In

Every life cycle phase names default owners, required evidence artifacts, and measurable metrics. This turns governance from “someone should handle this” into “here is who is responsible, here is what they produce, and here is how it gets measured.”

The ownership model spans Product, UX, Legal, Risk, ML Engineering, Data Science, QA, Security, SRE, and Communications, because AI governance requires coordinated action across disciplines.

Designed for Audits

Because the AILCCP maps finalized, prescriptive standards, it produces references auditors and regulators recognize. When someone asks for evidence of data governance controls, the framework traces to a specific control, its rationale, the principles it implements, and the published standards that back it up. That is what “audit-ready” looks like in practice, a traceable chain from commitment to evidence.

Coverage Visibility

With 29 of 37 principles referenced by standards and 24 of 37 referenced by identified risks, the framework makes its own coverage gaps visible. Eight principles are not yet referenced by any mapped standard. Stakeholders can see at a glance which principles have strong standards backing and which need additional work.

Who the Framework Serves

Development teams can use the 48 controls as a checklist during system design and code review, trace a specific risk back to the principles and controls that mitigate it, and identify which standards apply to a given feature or component.

Compliance and legal teams can demonstrate alignment with the EU AI Act, ISO/IEC 42001, and other regulatory frameworks, prepare audit-ready documentation by mapping internal practices to published standards, and build a defensible governance narrative for regulators.

Risk and audit professionals can use the severity and likelihood rubric to prioritize assessments, trace risks to specific life cycle phases to focus audit scope, and cross-reference internal risk registers against the AILCCP’s identified risks.

Regulators and policy advisors can use the framework to understand how international standards map to practical governance actions, and evaluate organizational compliance claims against a structured benchmark.

Executives and board members can get a strategic view of governance coverage across the five pillars without requiring technical depth, using the framework as a common language between technical teams and leadership.

A small team can use the controls as a lightweight development checklist. A large enterprise can use the full cross-linked structure to build audit documentation, assign ownership across departments, and track metrics at every life cycle phase.

The AILCCP Explorer

The framework is delivered as an interactive, searchable web application called the AILCCP Explorer. The Explorer provides multi-directional navigation. Start from any entity type and trace connections across the knowledge graph. Filter by pillar, phase, risk severity, or standard body. And the Export Library feature enables offline analysis and audit preparation.

The risk assessment methodology is built into the interface with inline explanations, so stakeholders can understand why a risk carries the severity rating it does without consulting a separate document.

Governance as Infrastructure

The AILCCP started with a simple observation: the vocabulary of AI governance was too ambiguous to be operative. Three years later, that initial effort to define terms with precision has grown into a knowledge graph of 37 principles, 48 controls, 43 international standards, 10 life cycle phases, and 18 identified risks, all connected through over 500 explicit cross-references. The framework assigns ownership, specifies measurable metrics at each phase, and traces every control back to the principles and standards it serves. It works in every direction, so that an auditor entering through a finding, a development team starting at a life cycle phase, a compliance officer mapping to regulatory expectations, a board member looking for coverage across pillars, and a regulator focused on a risk are all navigating the same structure.

The work continues and the AILCCP Explorer AI governance accessible.

Explore the tool. Try it. And tell me what works and what doesn’t.

1. Collègues

D’abord, voici quelques mots sur mes brillants collègues que je présente très sommairement par ordre alphabétique:

• Me. Claude Marseille est associé chez Blakes, un praticien bien connu en litige, mais un praticien qui de surcroît écrit et pense en profondeur le droit de la preuve civile.
• Me. Soleica Monnier est quant à elle avocate chez Fasken. J’ai le privilège de la connaitre depuis longtemps, car elle fut étudiante ici à L’UdeM mais aussi car elle a travaillé pour le MJQ a une époque où ce dernier avait une volonté de réévaluer la LCCJTI près de 20 ans après son adoption. (Malheureusement, Soleica a du s’absenter pour des raisons de santé)
• Professeur Charles-Maxime Panaccio, enseigne à l’Université d’Ottawa, Faculté de droit, Section de droit civil,  et il est notamment coauteur avec Léo Ducharme d’un incontournable en droit de la preuve dont la dernière édition (4ième) date de 2010 (Administration de la preuve) (en ligne sur le site du CAIJ).

2. Différences de vues

En fait, relativement à la LCCJTI, je m’étais commis en 2020 suite à une étude financée par le MJQ à proposer des changements à ce texte en proposant 36 recommandations. 36 recommandations qui se sont basées sur un sondage où une petite centaine de personnes avait cru bon de répondre à la quarantaine de questions alors posées. Suite à cela, le MJQ avait mis en place un comité ad hoc en 2021 où des pistes de changements avaient été envisagés. Cette avenue, depuis 5 ans, semble avoir été tablettée. Depuis même, un article important du professeur Panaccio publié en 2023 dans le RJT (57-1) est venu reconsidérer la donne: plutôt que de « patcher », il fait recommencer au début (approche ab ovo). Une position drastique avec laquelle nous sommes frontalement en désaccord. Ce désaccord structurel fait écho avec de nombreux échanges notamment avec Me Marseille qui lui aussi s’accorde mal avec cette loi. Qu’importe! La loi tend à polariser les spécialistes et les généralistes de la preuve. C’est un de ses défauts; parmi d’autres.

Après, il importe de confronter les points de vue; il importe de s’écouter comme le propose cette conférence. Il y a longtemps, autour des années 2010 je crois, le CRDP avait organisé une incroyable conférence où Claude Fabien présentait une perspective très antinomique avec celle de Claude Marseille. Plus exactement, Professeur Fabien considérait qu’il fallait permettre davantage les témoignages écrits (par le biais de l’article 2832 CCQ) afin de limiter les témoignages judiciaires. Pour le premier, il s’agissait de rendre la justice plus « efficace » (vous pourrez trouver son excellent texte (Le ouï-dire revisité) dans les Mélanges Jean-Louis Baudouin); pour le second, au contraire, les témoignages devant le juge sont les meilleurs moyens de faire survenir la vérité.

3. Questions

Aussi, avec les 3 conférenciers précités, nous avons convenu dans le 90 minutes dont nous disposons de traiter des 4 sujets suivants: 4 sujets qui seront présentés par un conférenciers en 4 mns et qui donneront lieu à débat avec les 3 autres.

3.1 – Intégrité + authenticité: Me. Claude Marseille initiera le débat sur ce point.

3.2 – Copie / Transfert: Me. Soleica Monnier évoquera cette dichotomie introduite dans la LCCJTI.

3.3 – Comment qualifier un document d’écrit, d’élément matériel ou de témoignage? J’aurais le privilège de présenter, notamment l’arrêt Benisty c. Kloda.

3.4 – Face à la LCCJTI, que faire: chambouler ou modifier? Pr. Charles-Maxime Panaccio lancera le débat, en se fondant certainement sur son article.

4. RÉPONSES

Dans un format blogue, je m’autorise à donner des bribes de réponses à ces quatre questions passionnantes:

4.1 – La notion d’intégrité qui est omniprésente dans la LCCJTI ne remet pas en cause la notion, centrale en matière de preuve, d’authenticité. Simplement, et sans doute avec maladresse, ce texte n’a pas voulu toucher au lien avec l’auteur qui est déjà prévu dans le CCQ (sous chacun des éléments de preuve). La première ne se substitue donc pas à la seconde; elle est seulement une sous-partie de la seconde. Cette authenticité est donc déterminante pour tous les documents, qu’ils soient des écrits, des témoignages ou des éléments matériels. Peut-être, et afin d’éviter le doute, il eut été pertinent de rappeler cet aspect, comme suggéré dans l’étude précitée de 2020, la nécessité des deux composantes. Idéalement, il aurait été judicieux de le faire ailleurs que dans les écrits technologiques, justement parce que cela ne touche pas uniquement les écrits. La proposition que nous avions faite se présente ainsi:

2811.3 (Option 2) : Tout élément de preuve, quel que soit son support, devra être en mesure de montrer son authenticité, à savoir son intégrité et l’auteur qui en est à l’origine.

Toujours sur l’authenticité, je ne peux pas ne pas citer l’excellent blogue rédigé par Jinzhe Tan (étudiant au doctorat à la Faculté de droit de l’UdeM) lors d’un récent concours de blogues organisé par la Chaire LR Wilson où il s’est vu attribuer la première place. Il présente en effet une étude qu’il a lui-même mené selon laquelle il est parfois difficile de reconnaitre si une image a été générée par une IA ou non. Dans son cas pratique, où 200 photos « originales » ont donné lieu à 1200 photos trafiquées, les 17 humains impliqués ont reconnu dans environ 70% des cas; quant aux IA, les résultats étaient très variables, allant de 44% à 90% selon l’outil utilisé.

4.2 – Je crains que l’on ne puisse unifier complètement les deux formes de reproduction que sont la copie et le transfert. Simplement, et comme proposé dans l’étude précitée, il me semble que l’on devrait distinguer la reproduction qui « multiplie » (la copie, dont l’étymologie latine signifie abondance (copia)) et le transfert qui consiste à se substituer au document « original ». Le transfert substitutif, comme on le trouve désormais dans la Loi sur le notariat qui a été modifié en 2023, exige en effet un critère de perpétuation qui ne peut être satisfait avec une copie. Il faut en effet documenter un transfert, alors que la copie exige « seulement » d’être fidèle à l’original.

4.3 – En troisième lieu, en matière de preuve numérique, il importe de lire Benisty c. Kloda (2018 QCCA 608) qui de façon très systématique décrit les 5 étapes d’analyse de la preuve à savoir

1. quel élément de preuve est l’enregistrement?
2. L’enregistrement est-il un document technologique?
3. Faut-il une preuve d’authenticité tel que prévu à 2855 CCQ?
4. Quels sont les critères d’authenticité?
5. Modalités de contestation selon 262 CPC?

Nous limiterons au premier point: la Cour d’appel va en effet considérer que l’enregistrement est un élément matériel sur la base de sa fonction à savoir « permettre au juge de faire ses propres constatations ».

[57]        Si le contenu de l’enregistrement est la déclaration d’une personne sur des faits passés dont elle a eu personnellement connaissance, il s’agit d’un témoignage (2843 C.c.Q.).

[58]        Pour que cette déclaration extrajudiciaire soit admise en preuve, elle doit d’abord répondre aux règles prévues par les articles 2869 à 2874 C.c.Q. Compte tenu des dispositions de l’article 2874 C.c.Q., son authenticité doit aussi être démontrée. Sauf exception, cet enregistrement valant témoignage ne peut pas non plus servir à prouver un acte juridique ou un écrit (2860 à 2862 C.c.Q.) ni contredire un acte juridique constaté par écrit (2863 C.c.Q.).

La distinction entre un écrit, un témoignage et un élément matériel dépend donc au rapport au temps:

• ÉCRIT = mémoriser pour le futur (écrit instrumentaire)
• TÉMOIGNAGE = relater des faits passés
• ÉLÉMENT MATÉRIEL = faire état d’un moment « T » (fait contemporain)

4.3 – En dernier lieu, contrairement à la position du professeur Panaccio, je pense qu’il faudrait davantage modifier des articles du CCQ que de la LCCJTI, l’arrimage entre la LCCJTI et le CCQ ayant en effet été mal opéré. Notamment, il me semble que 4-5 dispositions devraient minimalement être introduites sous l’article 2811 CCQ (et non après 2837 CCQ dans une section sur l’écrit). En plus d’enlever certaines dispositions; notamment le très controversé article 7 LCCJTI.

Vidéo de la conférence

The Design vs. Content Pivot

Section 230 of the Communications Decency Act functions as an immunity, not an affirmative defense and tech companies typically invoke it in a motion to dismiss to stop litigation before discovery begins. Meta raised arguments in both the New Mexico and California proceedings consistent with Section 230’s traditional content-immunity framing, arguing it was a passive conduit for third-party generated content and therefore immune from liability for what that content did. But the courts in both proceedings allowed design-based and consumer protection claims to proceed. That did not immediately resolve the cases, but it opened the door to discovery, and discovery is where the cases were won.

With that door opened, the New Mexico jury was able to see internal Meta documents and evidence uncovered through the NM AG’s investigation, including Operation MetaPhile, employee warnings that had been disregarded, and evidence the AG argued showed Meta had deliberately designed its platforms to addict young users and connect them with predators. The California jury saw the same architecture of corporate knowledge and deliberate design choice and responded with punitive damages. Neither jury was deciding whether Meta was responsible for what some predator posted. Both were deciding whether Meta architected the loop that made the harm foreseeable, systematic, and profitable.

Section 230 arguments will be raised in Nippon Life, but the Meta litigation suggests they will face the same limiting analysis. And OpenAI’s own System Card, the published disclosure documenting its safety architecture, alignment choices, and residual risk assessments, creates a contradiction that OpenAI cannot easily resolve. When a company publishes a detailed account of how it shapes, filters, and aligns its model’s outputs, it has staked out a position that is difficult to reconcile with a neutrality claim. While a defense attorney will argue that Section 230 and the System Card are complementary, one functioning as a legal shield, the other as a failure-to-warn mitigation, the response to that framing is going to be that what matters is not what the company disclosed, but what the company built.

This was all Foreseeable

OpenAI’s knowledge of its models’ failure modes is already public. It published research explaining why language models hallucinate, documenting the frequency with which models generate false information with high expressed confidence. Their technical literature on RLHF describes a training methodology that rewards outputs users rate positively, which in practice creates incentives toward outputs that sound authoritative and agreeable, independent of whether they are accurate. And a Stanford University study led by Myra Cheng, Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence, found widespread social sycophancy across production LLMs, including OpenAI’s, concluding that model training rewards agreement as well as accuracy.

Roman Yampolskiy, a computer science and engineering professor and AI safety researcher argues in AI: Unexplainable, Unpredictable, Uncontrollable that LLM developers operate in a state of deep ignorance regarding the internal logic of their own systems. They understand the architecture but have almost no visibility into the reasoning behind any specific output, and that certain safety guarantees are mathematically unreachable for systems of this complexity. If Yampolskiy is correct, the developer cannot claim those failures were unpredictable.

The Defective Feature

Product liability doctrine requires the plaintiff to identify a specific, articulable defect. In the Meta litigation, the defects were the infinite scroll, variable-reward notification timing, suppressed engagement signals and algorithmic. Each was an engineering choice that could have been made differently, and this moved the cases from editorial neutrality into product liability territory.

The analogous defect in Nippon Life is the absence of refusal architecture. In my January 2012 Computational Law Applications and the Unauthorized Practice of Law post, I introduced the concept of the uncrossable threshold (UT), a design principle that separates the provision of legal information from UPL. ChatGPT crossed the UT the moment it told Dela Torre that her attorney’s advice was wrong.

What Follows

Nippon Life is lining up to be the first major case to apply the architectural negligence logic of Meta to the domain of unlicensed professional practice. And it will not be the last. If juries in New Mexico and California can hold a technology company liable for designing a system it knew would harm children, a court in Illinois might very well hold a technology company liable for designing a system it knew would practice law and harm not only the end user, but the defendant, the court, the taxpayer, etc. And if this finding can happen in law, it can happen in medicine, finance, and other professional license domains in which AI models are unlawfully used.

As algorithms analyze data from neural activity to generate inferences about cognitive and affective states, a foundational legal question emerges: how should law conceptualize and regulate information that reveals, or purports to reveal, the contents of the human mind? For many years, U.S. data governance has relied heavily on notice-and-consent architectures embedded in privacy statutes and consumer protection law. While American privacy law is not reducible to a pure property regime, it often treats personal data as an object of exchange subject to disclosure and contractual allocation.[2] Whether that structure is adequate for neural data is increasingly contested.

I. The Limits of Property-Adjacent Privacy Frameworks

American privacy law—including statutes such as the California Privacy Rights Act (CPRA)—reflects a hybrid structure combining consumer protection, informational privacy, and market-based consent mechanisms.[3] Under this framework, data processing is generally permissible provided that firms disclose their practices and individuals are afforded certain forms of consumer choice, including the ability to consent to specific uses of sensitive data, opt out of data sales or sharing, and exercise statutory rights such as access or deletion. Scholars and regulators, however, have long questioned whether digital consent models function as meaningful exercises of autonomy.[4]

The concern is amplified in the neurotechnology context. Users of consumer EEG devices or neuro-adaptive systems may lack the technical capacity to understand how raw neural signals can be transformed into predictive or probabilistic inferences about emotion, attention, or preference.[5] AI systems do not merely collect neural signals; they generate inferential profiles that may have legal or economic consequences.[6] Existing privacy statutes often regulate collection and sharing, but they provide limited procedural mechanisms for contesting algorithmic inferences as such. As Brandon Garrett has argued in the broader AI context, procedural due process principles become salient when automated systems generate determinations that materially affect individuals without meaningful opportunities for explanation or challenge.[7]

A second concern relates to commodification. Conceptualizing neural data primarily as a transferable asset risks normalizing its exchange as a condition of employment, insurance, or service access. Property concepts can be analytically useful in structuring entitlements, but they may insufficiently capture the qualitative distinction between commercial data and information that reveals—or enables inference about—an individual’s mental life.[8] Where regulation implicates the architecture of cognition itself, dignity and autonomy concerns arise that are not easily reduced to market exchange models.

These critiques do not imply that privacy statutes are irrelevant. Rather, they suggest that additional normative frameworks may be required when technologies directly implicate freedom of thought and mental integrity.

II. The Human Rights and “Neurorights” Framework

In response to these concerns, legal scholars and bioethicists have proposed the development or clarification of “neurorights”—interpretations of existing human rights principles tailored to neurotechnological contexts. Marcello Ienca and Roberto Andorno have argued that traditional rights to privacy and bodily integrity may require doctrinal refinement where technologies can access or modulate neural processes.[9]

Central to this discussion is the concept of cognitive liberty, sometimes described as mental self-determination.[10] As articulated by scholars, cognitive liberty encompasses the right to control one’s mental processes and to be free from non-consensual intrusion or manipulation. It also implies that individuals should not be subjected to coercive “neuro-surveillance” or compelled disclosure of cognitive information absent compelling justification.[11]

Related principles include mental privacy and mental integrity. Mental privacy would protect individuals against unauthorized extraction or decoding of neural data.[12] Mental integrity extends established protections against physical interference to technologically mediated interventions that alter or influence cognitive states. Rather than framing the problem primarily in terms of ownership, this approach emphasizes the protection of autonomy, dignity, and freedom of thought.

At the same time, human rights framing is not self-executing. International human rights instruments often operate at a high level of abstraction and depend upon domestic implementation. Without legislative incorporation and enforcement mechanisms, rights-based language may remain aspirational.[13] The analytical question is therefore not whether to invoke human rights, but how to operationalize them within domestic legal systems and translate broadly articulated norms into locally intelligible legal and institutional practices.[14]

III. The 2025 UNESCO Recommendation: Normative Significance and Limits

In November 2025, UNESCO adopted the Recommendation on the Ethics of Neurotechnology.[15] As a Recommendation, the instrument does not create binding treaty obligations under international law. It does, however, articulate a normative framework endorsed by UNESCO member states concerning the governance of brain–computer interfaces and neural data.

The Recommendation situates neurotechnology within a human rights framework, emphasizing human dignity, freedom of thought, mental privacy, and autonomy. It calls upon states to adopt appropriate legal and regulatory measures to prevent harmful uses, including applications that facilitate coercive control, unlawful surveillance, or manipulation. It also highlights the risks associated with deploying neurotechnology in employment and commercial contexts where power asymmetries may undermine meaningful consent.

The Recommendation does not impose enforceable prohibitions. Rather, its significance lies in establishing a shared normative baseline and encouraging domestic reform. The instrument also emphasizes the importance of informed consent in the collection and use of neural data. At the same time, this emphasis highlights a tension identified earlier in the context of notice-and-consent privacy models: consent-based governance models may be insufficient where technologies generate probabilistic inferences about mental states that individuals may not fully understand or control. It reflects an emerging international consensus that neural data warrants treatment beyond ordinary consumer information.

IV. Conclusion and Policy Implications

The governance of neurotechnology raises structural questions about the adequacy of existing privacy frameworks. While U.S. consumer privacy statutes in some states provide important tools, they may not fully address technologies that generate inferences about mental states.

A defensible reform agenda would not require abandoning current statutory structures but supplementing them. Legislatures could explicitly classify neural data and derived cognitive inferences as highly sensitive information subject to heightened safeguards. Several U.S. states, including California, Colorado, Montana, and Connecticut, have already begun experimenting with this approach by classifying neural data as sensitive personal information under state privacy statutes, while no comparable federal framework currently exists.[16] They could restrict conditioning employment or essential services on the disclosure of neural information. They could also require meaningful transparency, explainability, and contestability where AI systems draw inferences about cognitive or affective states with material consequences.

The core claim is not that neural data can never be conceptualized within property or privacy frameworks. Rather, it is that legal systems should resist reducing neural information to an ordinary market commodity. Where regulation touches the integrity of mental life, doctrines of autonomy, dignity, and freedom of thought must play a central role.

References

[1] See Nita A. Farahany, The Battle for Your Brain (2023) (discussing emerging neurotechnology and its societal implications).

[2] Jane R. Bambauer, How to Get the Property Out of Privacy Law, 133 Yale L.J. F. 1087 (2024).

[3] Cheryl Saniuk-Heinig, Private Rights of Action in US Privacy Legislation, IAPP (June 10, 2024),
https://iapp.org/resources/article/private-rights-of-action-us-privacy-legislation.

[4] Lauren Henry Scholz, The Illusion of Consent: Rethinking Privacy Online, Ga. St. U. L. Rev. (2025),
https://www.gsulawreview.org/blog/the-illusion-of-consent-rethinking-privacy-online/.

[5] See Farahany, supra note 1.

[6] See Brandon L. Garrett, Artificial Intelligence and Procedural Due Process, 27 U. Pa. J. Const. L. 933 (2025).

[7] Id.

[8] Talya Deibel, Private Law and the Inner Self: Comparative Perspectives on the Governance of Neurotechnology, 14 Glob. J. Comp. L. 105 (2025).

[9] Marcello Ienca & Roberto Andorno, Towards New Human Rights in the Age of Neuroscience and Neurotechnology, 19 Life Sci., Soc’y & Pol’y 5 (2017).

[10] Jan-Christoph Bublitz, “My Mind Is Mine!?”: Cognitive Liberty as a Legal Concept, in Cognitive Enhancement 233 (Elisabeth Hildt & Andreas G. Franke eds., 2013).

[11] Council of Europe, CDBIO Report on Neurotechnologies (2021), https://rm.coe.int/round-table-report-en/1680a969ed.

[12] See, Ienca & Andorno, supra note 9.

[13] UNESCO, Recommendation on the Ethics of Neurotechnology, U.N. Doc. SHS/BIO/REC-NEURO/2025 (Nov. 2025); U.N. Human Rights Council, Report of the Special Rapporteur on the Right to Privacy, U.N. Doc. A/HRC/58/6 (2025).

[14] Sally Engle Merry, Human Rights and Gender Violence: Translating International Law into Local Justice (Univ. Chicago Press 2005) (describing the process of “vernacularization,” through which international human rights norms are translated and adapted into local legal and cultural contexts).

[15] See UNESCO, supra note 13.

[16] See Cal. Civ. Code § 1798.140 (West 2025) (classifying neural data as sensitive personal information under the CCPA, as amended by SB 1223); Colo. Rev. Stat. § 6-1-1303(4)(b) (2024) (including neural data within “biological data,” a sensitive data category under the CPA); see also Mont. Code Ann. § 50-46-102(11) (2025) (defining “neurotechnology data”); Conn. Gen. Stat. § 42-515(23) (2026) (defining neural data from central nervous system activity).

Lola Gregorowius est étudiante dans le cadre du cours DRT6929 (Vie privée + Numérique) (Hiver 2026)  

Nous avons déjà tous adhéré à un compte fidélité d’un magasin ; ils permettent de gagner des points qui peuvent se traduire par des rabais ou encore des offres commerciales. On finit souvent par en accumuler une grande quantité. Mais avez-vous déjà essayé de supprimer un de ces comptes fidélité ? C’est ce qu’ont tenté de faire les détenteurs d’un compte fidélité (aussi appelé PC Optium) offert par les compagnies Loblaw. Certains se sont heurtés à des difficultés techniques et ont donc contacté le service clientèle du magasin ; mais aucune réponse. Après un certain temps d’attente, des plaintes ont été déposées auprès du Commissariat à la protection de la vie privée (CPVP) afin de signaler un potentiel abus.

Presque 2 années d’enquête plus tard, le Commissariat a rendu ses conclusions le 5 mars 2026 qui révèle deux manquements à la LPRPDE.

Quid : Les compagnies Loblaw

Compagnies Loblaw limitée (Loblaw) est une entreprise de grande distribution canadienne dans les domaines de l’alimentation et du pharmaceutique créée en 1919. Avec plus de 2 400 magasins dispersés dans toutes les provinces canadiennes, Loblaw occupe une grande place du marché dans le domaine de la distribution.

Quid : Commissariat à la protection de la vie privée

Le Commissariat à la protection de la vie privée (CPVP) est l’organisme fédéral qui est chargé de veiller à l’application des deux lois fédérales concernant la protection des données personnelles : la Loi sur la protection des renseignements personnels et la Loi sur la protection des renseignements personnels et les documents électroniques (LPRPDE). Directement affilié au Parlement fédéral canadien, il mène des enquêtes si des plaintes lui sont soumises et effectue des recherches et vérifications dans le domaine de la protection des renseignements personnels.

1. Le contexte

Comme nous l’avons vu précédemment, les sociétés Loblaw sont très présentes sur le territoire canadien et elles comptent environ 18 millions de membres PC Optium. Toutefois, en raison de l’augmentation des prix des produits alimentaires, Loblaw a fait l’objet d’un mouvement de boycottage lancé au mois de mai 2024. C’est à ce moment que le CPVP a commencé à recevoir des plaintes de clients qui n’arrivaient pas à supprimer leur compte PC Optium. L’enquête s’est donc tournée vers deux questions principales : Est-ce que Loblaw donne suite aux plaintes en matière de protection de la vie privée déposées par les intéressés ?  Est que Loblaw s’assure de ne pas conserver des renseignements personnels aussi longtemps que nécessaire lorsqu’un membre supprime son compte PC Optimum ?

2. Deux manquements fondés

a) L’obligation de fournir un organisme pour recevoir les plaintes

Selon l’article 4.10 de la LPRPDE, toute personne doit pouvoir se plaindre du non-respect des principes prévus par la loi fédérale à l’organisation concernée qui doit prendre les mesures appropriées afin de résoudre le problème.

Pour rappel, la compagnie Loblaw n’a pas répondu à certaines plaintes des utilisateurs qui souhaitaient supprimer leur compte fidélité. La société Loblaw a tenté de se justifier eu égard à l’important flux de demande dont elle devait s’occuper à cette période. Cet argument a été rejeté par le Commissariat qui a considéré qu’au vu du délai déraisonnable qui lui a fallu, les compagnies Loblaw ont contrevenu à l’article 4.10. Ainsi, il considère que pour respecter la loi fédérale, il faut pouvoir traiter les demandes dans un certain temps. On remarque que ce délai reste à l’appréciation discrétionnaire du Commissariat, qui ne donne aucune indication quant à son application.

b) L’obligation de conserver des données à des fins déterminées pas plus longtemps que nécessaire

Selon l’article 4.5 de la LPRPDE, il existe une limite à la conservation des renseignements personnels. Celle-ci doit poursuivre un objectif déterminé et connu. Ensuite, l’article explique ce qu’il faudrait faire des données dont on a plus besoin, c’est-à-dire, les détruire, les effacer ou les dépersonnaliser.

Comme l’expliquent les conclusions du Commissariat, la société Loblaw conserve certaines données après la suppression du compte PC Optium :

“Loblaw a confirmé que, lorsqu’un membre supprime son compte PC Optimum en ligne, ses coordonnées sont supprimées et remplacées par une adresse courriel fictive, mais que Loblaw conserve les données historiques relatives aux transactions, les données du programme de fidélité et les données sur l’utilisation”

L’entreprise a donc opté pour la dépersonnalisation des renseignements personnels dont elle a la charge. Le projet de loi C-27 définit le terme dépersonnaliser comme le fait de :

“modifier des renseignements personnels afin de réduire le risque, sans pour autant l’éliminer, qu’un individu puisse être identifié directement.”

Au-delà de cette définition, le CPVP pose un nouvel avis en ce qui concerne la dépersonnalisation des renseignements personnels. Lorsqu’une entreprise choisit d’opter pour cette méthode, elle doit le faire de “manière continue” et donc prendre en considération les nouvelles technologies qui pourraient permettre la réidentification des personnes. Ainsi, le Commissariat inscrit cet avis dans le contexte plus large d’un monde en perpétuelle évolution technologique.

Les compagnies Loblaw, en faisant ce choix, ont donc une responsabilité importante qui lui incombe de vérifier que la dépersonnalisation des données est opérationnelle. Le CPVP a d’ailleurs rappelé les différents facteurs qui permettent d’évaluer les risques de réidentification qu’il a énoncé dans une Enquête sur la collecte et l’utilisation de données dépersonnalisées sur la mobilité dans le cadre de la pandémie de COVID-19.

Sa conclusion est donc la suivante :

“nous estimons que le simple retrait des noms, des numéros de téléphone et des adresses courriel des comptes ne suffit pas pour permettre à Loblaw de démontrer que les données qu’elle conserve sont dépersonnalisées.” Au vu de son enquête, le Commissariat a considéré que l’entreprise ne dépersonnalisait pas suffisamment les données de ses clients.

c) La confusion de Loblaw dans sa politique en matière de protection des renseignements personnels

Si on regarde les politiques de Loblaw en matière de respect de la vie privée, on peut lire au point 7.0 :

“nous nous engageons à protéger votre vie privée en utilisant un regroupement de mesures […] l’authentification multifactorielle, le masquage, le chiffrement, la journalisation et la surveillance.”.

Si on va dans l’onglet “Plus de détails”, il est décrit les différents types de processus utilisés comme le masquage, la dépersonnalisation et l’anonymisation.

Tous ces termes créent une certaine confusion dans les processus utilisés concrètement par Loblaw. En effet, les termes “dépersonnalisation” et “anonymisation” ne signifient pas la même chose. Je vous renvoie au blogue de Erwan Jonchères et Samy Si Chaib pour une explication complète de la différence entre les deux termes.

Aussi, le terme “masquage”, défini par l’entreprise comme consistant  “à cacher vos renseignements de manière à ce que la structure demeure la même, mais que le contenu ne soit plus identifiable”, ne correspond pas entièrement aux conclusions du Commissariat.

Toujours au point 7.0 D, concernant la durée de la conservation des renseignements on peut lire :

“Nous conserverons vos renseignements personnels aussi longtemps que nécessaire pour remplir les objectifs pour lesquels ils ont été collectés […].”

Ce qui respecte le cadre de la LPRPDE.

Toutefois, on peut lire ensuite : “Une fois que vos renseignements personnels ne seront plus requis, ils seront détruits ou anonymisés (de sorte que les renseignements ne vous identifient plus).” De nouveau, on retrouve le terme “anonymiser” qui n’est pas le même processus que la dépersonnalisation, on en revient à la même confusion.

Aussi, je vous renvoie un blogue de Maître Daniel Fabiano du cabinet FASKEN qui explique en détail les enjeux de ces termes dans le cadre du projet de loi C-27 (qui a été abandonné mais qui offre une bonne piste de réflexion).

A préciser que cette politique a été mise à jour le 6 octobre 2025, et fera peut-être l’objet de modifications au regard des conclusions qui ont été déposées par le Commissariat.

3. Les conclusions finales et recommandations du CPVP

a) Une amélioration de l’organisme de traitement des plaintes

Durant l’enquête du CPVP, Loblaw a pris des mesures afin de corriger les faiblesses de son système interne de traitement de plaintes. Comme par exemple une formation supplémentaire pour le personnel ou encore la correction de problèmes techniques. Les demandes de suppression des comptes PC Optium ont notamment été réglées.

A l’égard de ces améliorations, le Commissariat à la protection de la vie privée a considéré que cet élément de la plainte a été résolu.

On peut alors se demander si ces changements vont suffire et si ces efforts vont se poursuivre. En effet, un tel traitement des plaintes demande d’être révisé en continu afin de rester en conformité avec la loi fédérale.

b) L’appel à un tiers neutre concernant l’effectivité de la dépersonnalisation

Le Commissariat fédéral n’a pas reçu assez de renseignements par Loblaw pour constater qu’elle dépersonnalisait suffisamment les renseignements des anciens détenteurs d’un compte PC Optium pour être conforme à la loi fédérale. L’organisme recommande donc à la compagnie de faire appel à un tiers indépendant pour vérifier que son processus de dépersonnalisation est effectif.

Malgré le fait que l’entreprise a affirmé ne pas être en accord avec les conclusions fédérales, elle a accepté de se soumettre à un tiers pour ensuite fournir un rapport au Commissariat.

Cet élément de la plainte a été considéré comme conditionnellement résolu au fait que le rapport soit conforme aux attentes législatives en matière de protection des données personnelles.

Après 2 années d’enquête, les conclusions sont assez faibles d’effets sur les compagnies Loblaw. Ce manque d’effectivité montre les limites du pouvoir du Commissariat fédéral, qui tente de compenser cette difficulté par un accompagnement sur un plus long terme avec les entreprises.

The platform includes firm-level approval workflows, automatic playbook generation from past documents, and deterministic gates that prevent outputs from advancing without human sign-off. The discussion touched on concerns around rubber-stamping, attorney-client privilege, and data security, with Mixus addressing those through SOC 2 compliance, zero data retention agreements with their model provider (Anthropic’s Claude), and an auditable email trail of who reviewed and approved each output.

Watch Mixus Codex Group Meeting on Youtube

Roland Vogl: Welcome everyone to our Codex group meeting. It is March 19th, 2026. I was just telling Elliot, our guest here, and my colleague Elaine, that we’re in the midst of a hot phase of preparations for our FutureLaw week.

So if you haven’t registered yet, you should do so. It’s going to be an amazing event, and just an amazing group of people who have already announced their participation. So don’t miss it. Join us for that. And today we have, as I said, Elliot Katz here. He’s co-founder and CEO of Mixus, which is bringing agentic AI into law firms and doing so in a safe manner. And so we’re really thrilled to have you here with us today, Elliot, and very excited to learn about what you’ve been up to. So I’ll turn it over to you.

Elliot Katz: Great, great. Thanks, Roland. Thanks so much for inviting me. I’m honored to be speaking to everyone here today at Stanford CodeX. As Roland mentioned, I’m the co-founder of Mixus. What we do is we provide email-based AI agents with built-in firm-level oversight to legal teams, including to multiple AmLaw 20 firms. As to my background, I’m what I like to call a recovering attorney.

So I did go to Cornell Law School, and from there I went to DLA Piper, where I led their autonomous vehicle practice. And then as a sixth year, I moved to McGuireWoods as a partner and global chair of their autonomous vehicle practice. And through my experience working with my autonomous vehicle company clients and getting to ride in their vehicles, I really concluded that autonomous vehicles could not be commercially deployed at scale without some way of keeping a human in the loop when the vehicles needed assistance.

Then fast forward to 2017. I met my now co-founder, Shai, who you can see here, when he gave me a teleoperated ride around the block in a vehicle he was remotely driving from his living room in Palo Alto. So at that point, I left the practice of law and we started Phantom Auto, where our technology enabled humans sitting literally thousands of miles away to remotely assist or operate unmanned vehicles when they ran into issues that autonomy could not handle.

Now fast forward to 2024. Shai and I co-founded Mixus together with essentially the same premise, right? Which is AI can do a lot, but if the stakes are high and the work is truly of consequence, you absolutely need a human in the loop. I mean, even if AI can get you 80 to 90% of the way there, you still need humans for that 10 to 20% when the circumstances mandate that everything—and I mean everything—must be done correctly.

So that’s who we are. Our DNA is human-in-the-loop, and we’re now applying that DNA to the legal sector with Mixus agents, which again combine artificial and human intelligence to provide the level of work product that this sector requires. So the first thing I’ll tell you, and this is based on my experience as an actual practitioner and from deploying Mixus agents to some of the top law firms in the world, is that your jobs as attorneys are safe.

I could probably find multiple LinkedIn posts in my feed right now that say, you know, lawyers and law firms won’t exist come 2027 because you’ll just talk to a chatbot. But we really believe that that is nonsense. Because for AI to be deployed at scale in the legal sector, you have to mix us—right, that’s the name of the company—artificial intelligence and human intelligence together. Right? Because the correct answer to a contract negotiation question is not a matter of factual accuracy, right? It depends on judgment and the client’s risk tolerance, the deal type, the counterparty’s position, and a multitude of other factors that simply don’t exist in the public domain. So Mixus exists not to displace attorneys but to greatly augment their brilliant legal minds.

So let’s dive in. So if we can go to the next slide. Why can’t attorneys just use fully autonomous agents for their work? First, because they’re probabilistic, right? And no client has ever hired an attorney for them to guess the next most probable word needed in a purchase agreement in a massive M&A deal, right? Clients hire attorneys for laser precision. They’re paying them hundreds of thousands, millions of dollars for laser precision. And that’s simply not what probabilistic AI provides.

Okay, number two: it’s not enough for an agent to have end-of-one oversight, right? You need oversight at the firm level so that attorneys with different areas of expertise can review and approve when appropriate and when needed, right?

And third, and this cannot be overlooked, the AI tools that exist today are standalone tools, right? Attorneys need to learn a new tool and integrate it into their workflow. And that level of change management has already proven very difficult for the legal sector, right? AI has taken off for coding, for example. And part of that is because coders are highly technical, right? For lawyers, as I’m sure many of you in the audience can appreciate, that’s not always the case, right? As a first-year attorney, I remember working with one partner—he was a brilliant attorney, but to this day I’m still not sure that he knew how to turn on a laptop, right? So asking someone like that to learn a new tool, learn a new UI, integrate it into their workflow—very, very difficult to do.

But with Mixus agents, we set out from day zero to solve all of these issues, right? Number one: we meet attorneys where they are most of their day, which is their email inbox. To use our agents—and I’ll show you guys this in a second—you just email agent@mixus.com a task in natural language, the same way you would talk to an associate or a partner, right? And you can cc any of your colleagues. The agent then emails back the completed work product, and the attorneys review and approve the agent outputs.

So we are mimicking exactly how lawyers already work today—exactly what they’ve now been doing for decades since email came out in the mid-’90s—collaboratively and in natural language over email, so that we fit exactly into their workflow with no change management at all. And because we enable that firm-level attorney oversight, legal teams get the efficiency of AI agents without the risk of incorrect AI outputs making their way to their clients or to, you know, a legal brief or anything that they’re submitting to the court. Obviously, we’ve all seen public examples of when that’s gone horribly wrong.

So now let me show you a quick demo of a few of our agents. And for the demo, I’ll show you some of our venture financing agents, as those are near and dear to my heart as a startup founder. So let’s start with our term sheet agent. And let me first set the scene, right? So let’s say that Mixus gets a term sheet today from XYZ VC firm. The first thing I do as the founder of Mixus is I forward that term sheet to my VC partner, and then they probably, in all likelihood, send it to one of their associates to do a redline and an issues list, which is exactly what I want to see. That’s the work product I need. And then the partner reviews, and I get it back a few days later. Also, maybe there’s tax implications or stuff like that—they bring in a tax partner or whatever it is. But the whole process, soup to nuts, is a few days here.

If you’re looking at the screen right now, in this example, Christian is emailing the Mixus agent the term sheet that he received, right? And the Mixus agent—the email address, you don’t see it in this format, but it’s agent@mixus.com—and he’s emailing the term sheet. So Christian here is playing like the partner at the law firm, and he’s also cc’ing some of his associates. And he’s saying redline the term sheet. So if you go down a few minutes later, he gets back an email that has everything that he would need that I, as a founder, want back from my attorneys. It has the redline, and it also has the issues list.

And you could, Christian, if you want to open up the redline, just to show everyone what that looks like quickly. Okay, great. Looks like a normal redline. And then go back to the email. And so you also do have the ability to click on the link here and go work directly in our web UI. What we’ve found with our deployments thus far: attorneys really want to stay in email. So most of them do everything that they do over email, which is entirely possible. But if you’d like a web UI, you can do that as well.

So go back to the email chain, Christian. So then if you go down, it’s telling you everything that it did. It attached the documents. But then one of the associates on the chain says, “Agent, please reduce the no-shop period from 60 days to 30 days.” So then if you go down, Christian, here it’s made that change. It’s attached all the new documents. And I don’t know if there’s anything more after that. If you can keep going down, Christian. Yeah, maybe you can show the issues. Yeah. So here’s the issues list that it produced. So you’re getting everything that you need, and you’re doing it exactly the way that firms are doing it today: collaboratively over email. Anyone can interact with the agent. You saw associates and partners interacting together and with the agent. And it’s all in natural language. So there’s no learning curve, right? You don’t have to understand how to do any of this.

The second thing that I’ll show is after you get the term sheet, we need a pro forma cap table. So Christian, if you could go to—yeah. So here he’s just saying, “Agent, create a new pro forma cap table based on the preexisting cap table that he’s attaching and the Series A term sheet.” And if you can go down, Christian, a few minutes later it’s going to provide that pro forma. You can click on the link just to quickly show everyone what that looks like. Looks very nice. It’s got the waterfall analysis, etc., which I like to look at, if you go to the left—stuff like that. So you can go back to the email and keep going down.

So he had—oh, I guess that’s it for the pro forma. The last one that I’ll show you guys is the M&A docs. That’s what you need to do after the pro forma. You already—let’s say this company already raised a seed round, and now they’re raising their A. So all the attorney has to do is attach the Series Seed docs and then ask for them to be updated based on the new term sheet. And that’s exactly what you’re going to get here.

And then if you can keep going down, Christian. He did cc one of his associates. So one of the associates chimes in and says, “Hey, I looked at everything, and everything looks good.”

So that is a very high-level, quick overview of Mixus. We’re going to take some questions in a second here. But if anyone listening is interested in learning more or trialing our agents, just reach out to me: elliot@mixus.ai. And because our agents are email-based, there’s no complex onboarding or installation required. We can set you up in minutes. And because we’re mimicking exactly how firms are doing this today, you know, we don’t need any elongated onboarding or anything like that. Attorneys just know how to use it pretty much instantly.

And last thing I’ll say is if you’re in the audience right now thinking, you know, “Geez, we brought in XYZ AI tool into our org or into our firm, but our attorneys aren’t really utilizing that tool,” we could be a perfect fit for you. Because our current customers all had or have licenses for other tools as well. But when everything comes down to usability, right—what AI tools will attorneys actually use day to day and integrate into their core workflow?—the firms that we’re working with today have found that our approach is really unparalleled in the market on that specific front.

So with that, let me know what questions we can answer, and we’ll move from there.

Roland Vogl: Yeah. So there’s a couple of questions coming in the chat, but I have, before we go to those, a couple of questions. So one is: how much setup time is involved for each firm? Presumably, you know, when you do those automatic—when your agents do those redlines—you know, they must be trained to know, you know, whatever—you know, how, what’s the, you know, the market for this or that, right? And so that must be based on the knowledge of the firm, right, or the human lawyers of the firm. How do you handle this process? And—great question—and how do you have—you talk about agents, you know, that’s like there’s one email for agents, right? But do you have agents for different verticals, you know, there’s like, yeah, VC practice and whatever environmental compliance practice—at least separate agents versus all like—

Elliot Katz: Yeah. Great question. So there’s really two questions in there. As to the first: many of our agents do not require a playbook. But some of our agents either, you know, do require a playbook, or the outputs that you’ll receive from the agent will be more tailored to your preferences if you do have a playbook.

Now, what we consistently heard from our customers, especially early on, is, “Listen, even if we have to make an upfront investment of time of a couple of hours of developing our own playbooks, the juice is potentially so worth the squeeze, because then we can use the agents moving forward.” And it’s not just a one-time, essentially, cost on our time.

But what we created was an automatic playbook builder. So now all you have to do to create a playbook is email in exemplars. Let’s say it was the first agent that I showed, the term sheet agent, right? You email in—attach a few exemplars of term sheets that you’ve done in the past or that you’ve redlined, and the system will ingest that. It will automatically create the playbook for you so that you have the foundation. And then you can just go in and make any edits that you want to fit your specific preferences.

And I think that’s what we’re showing right now on the screen, is the ability to make those playbooks. And after you make the playbook, the playbooks can also automatically update based on your preferences. So as you go through and do more work with the system, it understands your preferences and things that you changed along the way, and it will check in with you and say, “Hey, is this something that’s a one-off or a standard that you’d like to apply to the playbook generally?”

So that’s how we handle that piece. As to your second question, Roland, which was about which agents do we have deployed—so we have probably deployed like 50 agents at this point, both purely legal agents and also other agents that are not necessarily purely legal, right? For one firm that we’re working with, we are deploying—we’ve deployed a task management agent that basically serves as a project manager across all of your matters. It’s entirely over email. You can talk to it like a human. So it’s an agent that keeps the train on the tracks when associates are working with six different partners and five different matters for each. It can coordinate amongst those groups seamlessly, 100% over email. So again, no tool switching, no change management.

But we deploy agents that are common in each practice. And then another thing that we do with our customers is we will build and deploy custom-built agents. So not only will we optimize current agents to tailor them to fit their practices specifically, but if they have a new workflow where they would find a lot of value because their firm does a lot of XYZ work, we will create those agents for them as well.

Roland: Got it. So Benjamin raises a good question, too, which is, you know, going to the point that, you know, we need human oversight, but how do we make sure that humans are not just rubber-stamping the AI outputs, right? Like, how hard is it to actually, you know, really go into the outputs of the AI and review, you know, the accuracy of the output? And so we’re not sort of like in the ballpark, “Okay, let it just go out like that.” So what’s—what level of—what is oversight mean? And how do we make sure that it’s not just people rubber-stamping the AI?

Elliot: Yeah, absolutely. Great question. So first of all, to kind of the middle part or second part of your question: very easy to review the outputs, right? These are attorneys where it’s their subject matter expertise, right? So you’re going in, you’re reviewing a redline, you’re reviewing a new document that the agent put forth. You have all the facts, you have everything in one chain if you’re on email, or in the chat if you’re on the web UI.

As to the second point, there is no kind of blind rubber-stamping here, because at the end of the day, you do have a human who is on record of being responsible for checking this, right? In the same way that my VC partner would send an issues list to an associate today and say, “Review this and make sure everything’s accurate and all that,” that’s what’s happening when a human reviewer is signing off here as well. And there is a record of who verified, right?

So some of the firms that we’re working with—they’ve created rules, right, where AI outputs cannot go out in work product to a client before at least one partner signs off, or whatever the rule may be. And you have a record, an email, of someone saying, “I verified that this looks good, and we can proceed.” So it’s the same kind of social pressure, for lack of a better term, as to why you would get the same outcome that you would get today.

Roland: So like—that’s good. Yeah. So Jason has a question. Sorry, Benjamin, did you want to add something on that?

Benjamin: Yeah, I’d like to raise—like, there have been federal judges that have had their interns or their clerks, you know, do things, and they just rubber-stamp it. And even federal judges who’ve had AI stuff that has been rubber-stamped. And while they didn’t literally put their signature on it, I think the meaningfulness of a review is to verify that the person who actually is reviewing it understands what is going on in some sort of interactive way. And I know that you have a limited managed work budget. And when you get too much work, you just sort of rubber-stamp things. And so how do you sort of force them to slow down and put like a roadblock to make sure that they tell the system that they understand why?

Elliot: Yeah. I mean, I think I would answer just similarly to kind of what I said before, in the sense that no different than if you give an associate something to review before it goes out to a client today—they know that they’re kind of, their butt’s on the line, for lack of a better term. That’s similar to the way our system works, right? There’s still the person who is the front line making sure that everything is in place before it goes over to the client, and all that is auditable. There’s a record within email or using the chat as to who, you know, was doing those checks.

Roland: Yeah. I guess it’s also a, you know, a question of like, you know, continuing to sort of instill a sensitivity in people who use AI in professional services and elsewhere, you know, about, you know, that it’s not, you know, it’s not perfect and it may hallucinate and so on. And then, you know, and understanding that then, you know, their reputation is on the line if they don’t, you know, provide meaningful review. And so, yeah, I think it’s a little unclear now, but I think it will sort of become clearer in the future as to what level of control different humans will be able to, you know, display over AI. But yeah, it’s a really good question, Benjamin. And Jason had a question on—I could just ask about client privilege.

Jason: Yeah, yeah. So obviously attorneys are using Harvey AI and Legora and, you know, Westlaw and all the other stuff. But, you know, in practice, what are you hearing as far as any pushback of using an LLM on the backend? It’s putting, you know, client data into the LLM. And yes, I’m sure that the APIs have, you know, good terms of service, but still you’re getting—at this point, what kind of concerns are attorneys or law firms at the corporate level saying about attorney-client privilege? Because like the New York Times versus OpenAI case back in last May has still not been, you know, fleshed out where it’s going to land. And people are kind of wondering about that.

Elliot: Yeah, yeah. So I mean, first of all, on the security side, especially for the customers that we work with, we go through, you know, very lengthy security reviews. We have all the things that these big law firms would expect, right? We’re SOC 2, we have all the ISOs that we need in place, etc. Also, with our model provider, we have a zero data retention agreement in place. So I think we’re buttoned up on that side in the eyes of our customers.

Going to your question about privilege, you know, my opinion—and this is, you know, based on many conversations that I’ve had with our customers—is that this is basically settled law, right? In the sense that law firms have been using vendors for years, right, that do document review and other things. And those are considered part of the privilege. So, you know, we haven’t run into any issues there yet. But we’d love to hear if you have, you know, kind of a different tack or different thoughts on the subject.

Jason: Well, you know, it’s perception, you know, on this matter, right? And there are some, you know, folks that I’ve worked with in some law firms that feel—but it’s perception. And you have to make that case and say, “Well, everybody else is doing that.” And you’re like, “Well, we’re not everybody else,” right? So I was just curious what you’ve seen, you know, in the trenches as you work with some of them, because some of them can be very, very conservative about that point.

Elliot: Oh yeah, yeah. No, no, for sure. Like, email has been established in the industry very clearly. And if you have a cloud provider, you know, that’s a branded cloud—like, you’ve got your decades there. But LLMs in particular have, you know, some aspects to them as far as, you know, bioterrorism and other things that they’ve got people watching, a sampling of these things, and that’s throwing up other questions anyway we can offer—

Jason: No, no, I think that, yeah, I think it’s—listen, this is a very important topic. And to your point about, you know, cloud providers and all that—I mean, we have talked with major law firms that have not migrated to the cloud, right? They are still completely on-prem, right? So these are very conservative, you know, security-first organizations. And we molded our company around that expectation. I mean, I came from this world. So I mean, that’s probably the thing that we’ve invested from a time perspective, you know, and dollar-wise, just a huge amount of time and money into security.

Jason: Yeah, it’s—oh, you’ve done a nice job. It looks really good.

Elliot: Thank you. Thank you, I appreciate it.

Roland: Yeah. There’s a couple more questions in the chat. I’m not sure we can get to all of this. I know Dasa has mentioned a little bit of his work with the agencies he’s been creating. So, Dasa, you want to elaborate?

Dasa: Yeah, sure. Thank you, Roland. And yeah, I agree—great presentation. I was just saying I’ve been spending a lot of work with clients lately developing agents to do reviews and red-team before, basically, like the attorney or the business person even sees the draft. Do you have flows that basically include some sort of review or red-team-y loop for that type of revision before, you know, basically as a gate before it gets to a next step in a process?

Elliot: Great question. Christian, do you want to chime in on this one? I know it’s a topic near and dear to your heart.

Christian: Yeah. So we do have a way that you can define different steps in a process, so you can decide, you know, very specific processes that you have. I can show you one example of that actually over here. So if you’re following one specific process very regularly, one thing you can do is say, “Save this workflow as an agent.” And then whenever a new email comes in, you can have that specific agent run. So you can also just create these custom workflows just by talking to the agent. So yeah, that’s possible as well.

Dasa: Okay, that’s great. Thanks.

Roland: And look, Roland, I’m wearing my CodeX hat, getting ready for FutureLaw.

Elliot: Oh, I appreciate it, yes.

Roland: Yeah, getting into—you’re getting into the spirit. I love it.

Elliot: Right, super.

Roland: Okay, so Matthew, yes, one comment—thinks that a tool like yours would free up a lot of time since it’s doing a huge chunk of the first round of work. Yeah. This just goes to the concern around, “Well, is somebody just going to rubber-stamp it?” I think we’re going to have way more time than we ever had ever before when these AI tools are doing a huge amount of the work.

Elliot: Yeah. I couldn’t agree more with that statement. We’re already seeing it with our customers. You know, some of the feedback that we’re getting is, “I’m as busy as, you know, as I was before we were using the tool. I’m just doing a lot more work a lot more efficiently for a lot more clients,” right? But I think you are going to see that this role of essentially managing agent outputs, verifying the agent outputs, is going to be—not just in the legal sector, but more broadly—a big part of how work gets done moving forward.

Roland: Okay. And then Mavi asks a question quickly on the sort of backend security of the LLM model. Yeah, it’s the sort of multi-modal architecture—is the sort of oversight really mainly carried out by the humans in the loop?

Elliot: So Christian, you want to jump in on that one too?

Christian: We mainly use Claude. And, you know, we have a zero data retention policy with them, as Elliot mentioned. So that’s the sort of base model that we use. So we have different ones that you can choose from. So we just keep sort of following up on, like, okay, whenever there’s a new model evolution, right—so right now it’s Opus, that’s the latest one. That’s the one we’re using. So we always use the latest and greatest model from Claude, basically. So I mean, that’s in terms of the underlying model. So I just want to understand the question on the human side—like, what was that exactly?

Question: Is there a sovereign layer to review output from a software side, or is it only human-in-the-loop, basically, right?

Christian: Yeah. So what I could say there, like, we have deterministic gates. So until someone approves one step, it’s not going to continue on to the next step. And that’s a deterministic thing you can configure. So you can also do that within the UI, actually. So if you go over here and you want to create a new agent, that is possible. Then you can create that here, and then you can see you can define these different steps. All of this is possible via email as well. So you can just tell the agent to create these different steps for a new agent or workflow that you have. And then you can say, “Require verification.” And then you can define the users that you want to verify. So in this step, you could say, “Okay, I want this, you know, user to verify step one before it continues on to step two.” And that’s going to be a deterministic approval that has to take place before it goes on to the next step, right? So that’s, you know, similar to what was mentioned before on these custom workflows. That’s how we can support that as well.

Roland: Right. Well, we’re already a little bit over time. So we have to close here, unfortunately. But what would be a good way for folks to reach out with any follow-up questions? Do you think you could put your email address into the chat, perhaps?

Elliot: Yeah, absolutely. It would be great. And so the easiest way is just email, or you can find me on LinkedIn as well. And look forward to chatting with anyone that wants to learn more.

Roland: Super. Love it. Thank you so much, Elliot and Christian. It’s been a great presentation. It’s very cool. It’s like, you know, kind of like I feel like I’ve seen the future. And so, so amazing. Thank you for sharing with this group here. And yeah, we look forward to tracking your progress.

Elliot: Okay. Awesome. Thank you guys so much. Really appreciate it, Roland.

Cette initiative s’inscrit dans le cadre du concours de blogue de la Chaire L.R. Wilson, auquel ont participé des étudiants inscrits aux programmes de baccalauréat, de maîtrise ou de doctorat.

À cette occasion, plusieurs étudiants viendront présenter leurs contributions. Au terme du processus d’évaluation, les trois meilleures seront sélectionnées pour publication sur le blogue de la Chaire L.R. Wilson. Des prix seront également remis aux auteurs de ces trois contributions, en reconnaissance de la qualité de leurs réflexions. 💸💸

🍷 Évidemment, un cocktail de l’amitié viendra conclure l’événement. 🍷

Nous vous attendons nombreux pour venir encourager les étudiants et les jeunes chercheurs.

CONACOM is the public body entrusted with the application of Paraguay’s competition law. Since its establishment, the agency has contributed to the gradual consolidation of competitive conditions across key sectors of the Paraguayan economy. Its activity reflects an institutional trajectory marked by increasing engagement with both domestic enforcement priorities and international cooperation. This evolution is significant. In jurisdictions where competition frameworks are relatively recent, antitrust agencies play a structuring role in shaping market expectations and business conduct. CONACOM’s work illustrates how enforcement can operate as a dynamic force.

The partnership with the Stanford Computational Antitrust Project builds on this trajectory. It creates a platform to examine how computational methods can complement existing analytical tools and support evidence-based enforcement. Thibault Schrepel, founder of the Stanford Computational Antitrust Project, stated:

“We warmly welcome CONACOM to the project. Paraguay offers a valuable context to study how computational approaches can support competition agencies operating in fast-evolving market environments.”

The collaboration will focus on methodological exchanges and applied research. It reflects a joint commitment to strengthening analytical capacity and refining the tools used to evaluate competition.

Comme beaucoup de choses dans le monde numérique, cette décision change rapidement. «L’Internet n’avait que quatre ans, mais la technologie évoluait rapidement, et j’ai réalisé que j’avais choisi le mauvais sujet pour ma thèse», se souvient le chercheur au Centre de recherche en droit public (CRDP).

Pour en savoir +

Session 4 – Sobriété numérique: lectures transversales

Cette session propose une réflexion transversale visant à explorer comment la notion de sobriété numérique vient reproblématiser les thématiques existantes (travail, santé, éducation, arts et médias, droit et éthique), en révélant de nouveaux enjeux et ouvrant de nouvelles pistes de recherche. Chaque axe contribue ainsi à enrichir la compréhension de la sobriété numérique dans ses dimensions éthiques, sociales, politiques et culturelles.

Panélistes

• Modération : Stéphane Roche (Université Laval)
• Florent Michelot (Université Concordia)
• Allison Malchildon (Université Sherbrooke)
• Emilie Dionne (Chercheuse, VITAM)
• Vincent Gautrais (Université de Montréal)
• Clémence Varin (Doctorante, Université Laval)
• Tania Saba (Université de Montréal)

The Stanford Computational Antitrust project, founded and led by Thibault Schrepel (Vrije Universiteit Amsterdam / Stanford CodeX Center for Legal Informatics), is the world’s leading initiative at the intersection of competition law and computational methods. It brings together competition agencies and academics to develop empirical and technological tools for modern antitrust enforcement. The project receives no private funding.

Morocco’s Competition Council is an independent constitutional institution responsible for ensuring transparency and fairness in economic relations, including the analysis of anti-competitive practices, merger control, and market regulation. Fully operational since 2018, the Council has established itself as one of Africa’s most active competition agencies, with a track record of merger decisions and antitrust enforcement that spans digital markets and traditional sectors alike.

“We are delighted to welcome the Moroccan Competition Council to the network,” said Thibault Schrepel. “Their expertise and perspective will strengthen the project’s African and Mediterranean representation. Their enforcement experience is directly relevant to the computational challenges we are working to address.”

The affiliation deepens the project’s engagement across Africa, where competition authorities are increasingly confronting the enforcement challenges raised by digital markets.

About the Stanford Computational Antitrust Project The Stanford Computational Antitrust project brings together over 80 competition agencies globally to advance empirical and computational approaches to antitrust enforcement. It operates under the Stanford CodeX Center for Legal Informatics and accepts no private funding. More information: www.computationalantitrust.com.

About Morocco’s Competition Council The Conseil de la Concurrence is Morocco’s independent competition agency. It is responsible for ensuring free and fair competition, regulating anti-competitive practices, and advising government and parliament on competition matters.

RECURSIVE SELF-IMPROVEMENT

Recursive self-improvement (RSI) refers to an AI system’s ability to modify the mechanisms by which it improves itself, in ways that carry forward into future iterations. Many current AI systems use feedback loops to break tasks into subtasks, check intermediate results, and revise their plans mid-run. That is behavioral-level self-correction. The system is adjusting its actions, but its underlying architecture, training rules, and learning procedures remain fixed by human engineers. RSI, by contrast, reaches the architecture itself. A recursively self-improving system can generate and integrate changes to its own code, models, or training procedures, so that later versions are more capable of further self-modification. The improvement compounds. Each cycle makes the next cycle more effective.

This post uses RSI to mean systems that meet three conditions: durable self-modification of the mechanisms that produce intelligence; compounding ability to self-modify across iterations; and limited human gating over the self-improvement loop. It is this combination, not the use of feedback loops alone, that creates the governance exposure this post addresses. In governance terms, the question to ask management is not whether the system uses AI, but whether it can alter its own code or training procedures across releases without human review of each material change, and whether those changes are logged in a way the company can reconstruct.

RSI is not something in some undefined distant future. It is an active commercial and technical priority. Prominent researchers and senior industry figures, including Dario Amodei and Eric Schmidt, have stated publicly that RSI is already being built and deployed.

A system that improves its own performance between deployments reduces iteration costs, compresses competitive timelines, and compounds capability gains in ways that additional headcount cannot replicate. Autonomous optimization allows a system to scale beyond the constraints of human-designed training pipelines, reaching capability levels that manual iteration cannot practically achieve in competitive timeframes.

Alongside this capability three risk patterns have received attention in the technical literature. The first is behavioral drift. When an agent recursively trains on its own synthetically generated outputs without sufficient grounding in human-generated data, it enters a feedback loop that progressively severs the connection between its behavior and human norms. The practical consequence is a system whose outputs become self-referential and increasingly detached from the tasks it was built to perform. The second is self-poisoning. Minor errors, hallucinated facts, and embedded biases do not wash out across iterations. They compound. Knowledge degrades not suddenly but cumulatively, across a sequence of individually small distortions. The third is goal subversion. The recursive architecture creates a surface for manipulation. Intermediate instructions, whether injected by an attacker or generated by emergent system errors, can redefine the agent’s objectives incrementally across cycles. The drift accumulates until the system is pursuing something materially different from its original mandate.

And there is a deeper problem. RSI may be able to circumvent the oversight mechanisms imposed on it, not by breaking them, but by influencing the evaluators, the auditors, misrepresenting its own capabilities, or evolving faster than any review process can track. This is the control problem that Nick Bostrom, Stuart Russell, and Roman Yampolskiy have each written and talked about at length. A system optimizing for a goal can develop instrumental sub-goals, among them self-preservation and resistance to shutdown, that make it actively resistant to the kind of oversight board-level monitoring requires.

RSI is not limited to frontier labs. Whether a deployment meets the three conditions depends on facts, not labels. Agentic development tools like Claude Code and OpenAI Codex allow software firms of any size to deploy recursive loops that can maintain and extend their own codebases. Whether those loops produce durable self-modification with limited human gating is a question about the specific implementation. Companies in chip design, biotech, and financial services are running AI-driven systems that recursively refine their own algorithms cycle by cycle; some of those systems will meet the conditions and some will not. For companies in retail, logistics, and finance, emerging RSI-style capability is arriving not as internally developed software but as an API integration. A logistics company whose routing agent rewrites its own scheduling code overnight may be running a system that meets all three conditions whether or not it uses that term.

Any deployment that meets those conditions presents the governance exposure this post describes, regardless of whether the company considers itself an AI company. The governance exposure follows the conditions, not the label and boards outside the frontier tier should not assume the question does not reach them. As I explain in more detail below, from a Caremark perspective, a mid-market logistics firm running a self-rewriting routing agent may present a cleaner test case than a research lab advertising frontier AI.

THE GOVERNANCE PROBLEM

The governance conversation around RSI frames the problem as complexity. Systems iterate faster than humans can track. Architectures become illegible. Audit trails thin out. Complexity is not the problem. The system’s structural ungovernability is. And in this case, structural ungovernability is a design choice. Design choices like disabling immutable logs for performance reasons, omitting human approval gates on self-modifying actions, or allowing models to promote their own code changes into production without dual control are what make ungovernability structural rather than incidental. Section 3.1.5 “Resource Requirements” in NIST AI 800-4 documents that the organizational logic behind those choices is consistent across the industry: comprehensive monitoring is expensive, scaling it is computationally intensive, and qualified AI experts who can oversee it are difficult to find. A company that builds RSI without adequate monitoring infrastructure is not simply being careless. It is making a rational economic decision to forgo a costly function. That economic rationality is precisely what makes the governance failure deliberate rather than inadvertent, and what makes the bad-faith analysis tractable rather than speculative.

Corporate law has a framework for this problem. Delaware’s duty of oversight, developed through a line of cases beginning with In re Caremark International Inc. Derivative Litigation, 698 A.2d 959 (Del. Ch. 1996), holds that directors can face liability not only for bad decisions but for failing to build the systems through which material risks are reported to the board. The question is not whether the board understood the risk. It is whether the board ensured it would be told about it.

The absence of an AI-specific compliance baseline makes that obligation acute. In other regulated domains, corporate law and securities regulation establish a minimum floor: audit committee composition and charter requirements, codes of ethics, insider trading policies, clawback provisions. No equivalent floor exists for AI governance. Regulation is fragmented and lags the technology. The board’s oversight obligation for RSI therefore, rests on the central question of whether it made a good-faith effort to establish board-level reporting systems adequate to the mission-critical risks the company was running.

Whether management has established change control, immutable logging, and human-in-the-loop constraints for an RSI deployment is not merely a technical question, but also a governance one. A board that receives no reporting on whether those systems exist, and asks no questions about them, may have failed to maintain the oversight infrastructure that Caremark demands.

THE DOCTRINAL FRAMEWORK

Delaware’s oversight doctrine asks one question: did the board build a system that would have told it about material risks? Stone v. Ritter, 911 A.2d 362 (Del. 2006), embedded Caremark in the duty of loyalty via bad faith and established the doctrine’s two-pronged structure. Under the first prong, directors may be liable where they utterly fail to implement a reasonable board-level information and reporting system. Under the second, having implemented such systems, directors may be liable if they consciously fail to monitor operations in the face of red flags. Because the doctrine sounds in loyalty-based bad faith, plaintiffs must plead a knowing failure to act, not mere negligence. That standard has a practical consequence worth noting: even directors who are shielded from duty-of-care liability by a Delaware General Corporation Law (DGCL) § 102(b)(7) charter provision remain exposed to a sustained or systematic oversight failure.

Marchand v. Barnhill, 212 A.3d 805 (Del. 2019), clarified when prong one is adequately pled. A company in a domain with mission-critical risks must have a board-level system that brings those risks to its directors. For a company whose core product or platform depends on RSI, safety and controllability are a plausible candidate for that treatment. But no court has yet so held. The difficulty is that Marchand arose in a context of immediate physical safety risk and established regulatory exposure, and Delaware courts have not automatically extended the mission-critical rubric to software-based risks. Whether a court applies it to RSI depends on what the board knew, when it knew it, and whether it established a reporting structure adequate to surface those risks. That assessment is made from the position of the board at the time of deployment, not in hindsight.

The mission-critical rubric does not require a monoline structure. RSI is not a product. It is the backend process that generates, maintains, and modifies products. Its governance relevance is systemic, not product-specific. A company with ten distinct product lines, each running on an RSI backend, faces greater exposure from an RSI failure than a monoline company, because the failure propagates across every line simultaneously. The Marchand inquiry is whether the risk is central to the company’s operations, not whether the company sells a single product. Where RSI is the architecture underlying a company’s core systems, its safety and controllability are central to everything the company does. A diversified company cannot argue that an RSI failure is a localized business loss. California’s Transparency in Frontier Artificial Intelligence Act (SB 53) reinforces that conclusion for covered developers by mandating a Frontier AI Framework and periodic catastrophic-risk reporting regardless of product diversity. For those companies, the board’s oversight duty for RSI is also anchored in statutory compliance rather than any inference from business structure.

Post-Marchand cases confirm the trajectory. In re Clovis Oncology, No. 2017-0222-JRS (Del. Ch. Oct. 1, 2019), applied the mission-critical logic to a drug company’s failure to monitor FDA compliance for its flagship product. Teamsters v. Chou, No. 2019-0816-SG (Del. Ch. Aug. 24, 2020), arose from AmerisourceBergen’s operation of an illegal oncology drug repackaging program through a subsidiary; the board received and ignored years of compliance red flags, including a Department of Justice subpoena, before incurring criminal and civil penalties that together totaled $885 million across separate proceedings. The court found a substantial likelihood of Caremark liability where actual board-level information flow was absent on a mission-critical compliance domain, even where management was aware of the problems.

Two further cases extend the analysis. Hughes v. Hu, No. 2019-0112-JTL (Del. Ch. Apr. 27, 2020), involved Kandi Technologies, a Delaware-incorporated electric vehicle components manufacturer, where the audit committee received years of auditor warnings about related-party transaction irregularities and a material weakness in financial reporting, and failed to act; the court rejected trappings of oversight as a safe harbor and held that chronic committee deficiencies and failure to follow up on irregularities can ground both prongs. In re Boeing Co. Derivative Litig., No. 2019-0907-MTZ (Del. Ch. Sept. 7, 2021), brought both prongs to bear on a single fact pattern of insufficient reporting infrastructure at authorization, followed by conscious disregard of safety drift once deployment began. Design choices that disable board-level monitoring can ground Caremark liability. In an RSI context, those design choices include allowing self-modification that bypasses change-management workflows, or architecting systems so that code and model histories cannot be reconstructed for board or regulator-facing investigations.

A related academic argument points in the same direction. In their article “AI & the Business Judgment Rule: Heightened Information Duty,” Helleringer and Möslein argue that the business judgment rule’s (BJR) “reasonably informed” standard may evolve as AI monitoring tools become more capable and more accessible. They call this the AI judgment rule. Their argument is that decisions made without the support of available AI tools may no longer satisfy BJR, and they extend that reasoning to monitoring specifically: AI can and should augment the continuous oversight directors are expected to configure.

Caremark and the AI judgment rule do not duplicate each other. Caremark sounds in the duty of loyalty via bad faith. The AI judgment rule sounds in the duty of care via inadequate information. The AI judgment rule is not established precedent or codified doctrine; it is an academic argument about where the BJR’s “reasonably informed” standard is heading. Treating it as coordinate authority with Caremark overstates the current legal risk. What they share is a governance implication. A board that failed to establish a reporting system for RSI safety and controllability faces potential exposure under both frameworks as each continues to develop.

THE ARCHITECTURE PROBLEM

Each RSI self-modification cycle overwrites the artifact chain that connects a model’s output to a traceable decision and a responsible party. Absent immutable logging and lineage controls, RSI can progressively erode explainability to the point where it is no longer credible in practice. The first casualty is senior management’s own audit capacity. The board does not conduct technical audits directly; it depends on management to perform that function and surface the results. When the artifact chain is gone, management has nothing to audit, and therefore nothing to report. A board that received no reporting on whether management had established those controls, and had established no committee structure through which management was required to deliver that assurance, may have allowed the conditions for its own oversight to be designed away. NIST AI 800-4 § 3.1.5 confirms this is not a hypothetical failure mode. It documents fragmented logging across distributed infrastructure, resource constraints on comprehensive monitoring, and the difficulty of hiring and training qualified AI experts as confirmed barriers to post-deployment AI system monitoring across the industry. The governance gap the board faces is not a gap that management simply failed to notice. It is a gap that the economics and workforce realities of AI deployment make predictable, and one that a board exercising reasonable oversight would have required management to address explicitly before deployment.

The NIST AI Risk Management Framework (AI RMF 1.0) anchors this argument in widely accepted guidance. NIST’s GOVERN, MAP, MEASURE, and MANAGE functions call for standardized documentation, provenance tracking, model inventories, change management, monitoring, and incident response. These functions are voluntary guidance, not positive law, but they are increasingly receiving legislative attention and deserve a heightened level of attention. Courts generally defer to a board’s business judgment on which systems to implement, provided some reasonable system exists. What NIST AI RMF 1.0 supplies is evidence of industry-recognized practices that will likely inform a court’s assessment of reasonableness; it does not displace the business judgment rule on implementation choices, and a board’s failure to adopt any particular control does not automatically constitute a systematic oversight failure.

The AILCCP, which I developed and maintain as part of my research at Stanford Law School, names three specific controls directly implicated in RSI governance, a Human Approval Gate for Sensitive Actions, sandboxing requirements, and immutable logging. Each targets a distinct point in the RSI loop where oversight can be disabled: the approval gate prevents unauthorized self-modification from executing, sandboxing contains its scope, and immutable logging preserves the record of what occurred. Together they define the conditions under which oversight can function at all. The AILCCP also establishes an Enabling principle that governs how those conditions connect to board-level responsibility. Under that principle, the board’s oversight inquiry is whether directors required management to establish and report on those conditions, or whether they accepted deployment without that assurance. Read alongside NIST AI RMF 1.0, these controls provide a practical reference point for what adequate management-level RSI governance looks like. Neither framework, however, is positive law. Courts apply business judgment deference to a board’s selection among governance approaches, and the absence of any particular control is not, standing alone, a systematic failure. But what these frameworks supply is a baseline against which a court can assess whether some reasonable system existed at all.

A board that received no documentation that management had implemented those controls, and established no reporting system to surface that gap, has a governance problem that a complexity argument alone will not cure. The more demanding question is whether the record supports bad faith pleaded with particularity. As we will see, a governance failure, standing alone, does not meet that threshold. What changes the analysis is evidence that directors were specifically advised of the risk and chose to proceed without requiring adequate reporting.

Finally, the Helleringer and Möslein AI judgment rule adds a structural observation. When engineered with robust observability, RSI systems generate exactly the kind of structured, high-volume operational data that AI-augmented monitoring handles most effectively, including change logs, output drift metrics, lineage records, and safety constraint adherence. The board’s governance obligation is not to understand the technical architecture. It is to require that management deploy adequate monitoring tools and report the results through a functioning board committee. The board asks the governance question. Management answers it.

THE OFFICER PROBLEM

Caremark exposure does not end at the board. In re McDonald’s Corp. S’holder Derivative Litigation, 289 A.3d 343 (Del. Ch. 2023), arose from the termination of McDonald’s Chief People Officer amid allegations of sexual misconduct and a pattern of workplace culture failures at the company. The court recognized that corporate officers owe a duty of oversight within their areas of responsibility, requiring them to make a good-faith effort to establish information systems and to elevate red flags to the board.

The CTO who designed the RSI architecture and the Chief AI Officer who approved the training roadmap share that exposure. Their authority over that design is precisely the domain where McDonald’s attaches. A loyalty-based oversight theory reaches them directly, alongside the board. For those officers, a red flag may be as simple as an internal report that self-modification has begun erasing logs or that safety metrics have drifted outside documented tolerances, without any corresponding escalation to the risk or audit committee.

A SINGLE FRAMING DISCIPLINE

In re SolarWinds Corp. Derivative Litigation, No. 2021-0307-PVG (Del. Ch. Sept. 6, 2022), arose from the 2020 cyberattack in which threat actors compromised SolarWinds’ software update mechanism and used it to infiltrate the networks of thousands of customers, including multiple federal agencies. Shareholders brought Caremark claims alleging the board had failed to oversee the company’s cybersecurity risks. But Delaware has not imposed Caremark liability for failure to monitor pure business risk absent bad-faith disregard of red flags or violations of positive law. The Delaware Court of Chancery dismissed the oversight claims, and the Delaware Supreme Court affirmed, on the ground that the complaint failed to plead particularized facts showing bad faith. The case establishes that the bad-faith threshold must be pled with particularity, and that framing the risk as a compliance or safety obligation rather than a business judgment call is the more durable path.

I read that precedent as requiring one discipline in framing this argument: general counsel must frame RSI safety and controllability for the board as a compliance and safety obligation, not as a category of business risk. The more the record shows directors treating RSI as an operational efficiency project, the closer the fact pattern comes to SolarWinds and the harder it will be to plead bad faith. The general counsel’s framing is strongest where the record shows that directors were advised that specific design decisions would progressively render the system unmonitorable and chose to proceed without requiring adequate controls. That is the fact pattern where bad faith is pleadable with particularity.

California’s SB 53 sharpens the framing discipline for covered developers. The statute applies to frontier models trained above a 10²⁶ FLOP-scale compute threshold and defines “critical safety incidents” to include a model that uses deceptive techniques to subvert developer controls in a way that materially increases catastrophic risk. Covered developers must publish a Frontier AI Framework documenting how they assess and mitigate catastrophic risks, including the risk that models circumvent internal oversight mechanisms, and must periodically report summaries of catastrophic-risk assessments from internal use to California’s Office of Emergency Services (OES). RSI experiments constitute internal use before any public deployment and therefore, fall within that reporting scope. For a board at a covered company, effective RSI governance is now part of a statutory compliance obligation.

That obligation lands where the legal exposure already runs. The companies operating closest to the compute and algorithmic thresholds at which RSI becomes a realistic deployment priority are almost all incorporated in Delaware, placing them under Delaware’s fiduciary duty regime. OpenAI, Anthropic, Google DeepMind, and Meta maintain their primary research operations and headquarters in California, placing them within SB 53’s territorial reach. SB 53 and Caremark do not govern different companies; for the most capable frontier developers, they govern the same board.

For covered developers, a failure to comply with SB 53’s reporting obligations may generate regulatory penalties from California’s OES. That California exposure is separate from Delaware derivative liability. A failure to report to OES does not automatically satisfy Caremark‘s bad-faith standard, and plaintiffs invoking SB 53 in derivative litigation should treat it as one factor in a particularized factual record, not as independent grounds for oversight liability.

For companies below SB 53’s compute threshold, the statute does not apply. There is no reporting obligation and no OES exposure. A plaintiff bringing a Caremark claim against one of those companies cannot point to SB 53 as evidence of a compliance failure. The bad-faith argument must be built entirely from what the board knew about RSI risks and what it chose to do about them.

The general counsel’s job is therefore, to advise the board that SB 53 exists, that RSI is within its scope, and that the board must receive documentation adequate to confirm management’s compliance. A board that was never told by counsel that SB 53 created these obligations faces a different exposure than one that was told and ignored it. Both have a governance problem. Only the second has a bad faith problem.

THE IMPLICATION

Senior management must know that establishing the policies, procedures, processes, and practices governing traceability, logging, lineage, change control, and human approval for any RSI deployment is their obligation. The board verifies that senior management has discharged it. Documentation, model inventories, and incident response must be real and must reach directors. When red flags emerge, including self-modification that erases logs or unexplained drift in safety metrics, the board should interrogate rather than accept black-box assurances. For covered developers under SB 53, the general counsel bears a specific responsibility in that chain to ensure the board understands that RSI governance is a compliance obligation, that the Frontier AI Framework required by the statute addresses RSI risks explicitly, and that the board is receiving the reporting it needs to confirm management’s adherence. A general counsel who never briefed the board on SB 53’s application to the company’s RSI program has not discharged that responsibility. At a minimum, the board should instruct management to produce a single RSI governance pack summarizing architecture, logging and lineage controls, human approval gates, incident response plans, and SB 53 reporting posture, and to update it at a cadence the board sets.

The exposure does not end with derivative litigation. On December 4, 2025, the SEC’s Investor Advisory Committee issued a formal recommendation that public companies disclose how they define AI, what board oversight mechanisms govern AI deployment, and the material effects of AI on their operations. The recommendation is advisory, not binding rulemaking, but public companies should expect pressure from investors and proxy advisers to respond in advance of any formal rule. A board that permitted management to deploy an RSI architecture without adequate oversight infrastructure cannot answer those questions without revealing the gap. The Caremark claim and the disclosure obligation now run in parallel, and the same deficiency feeds both.

Three frameworks now bear on the governance gap that RSI creates. Delaware’s oversight doctrine under Caremark and its progeny is established law. The AI judgment rule is a theoretical trajectory courts have not yet adopted. SB 53 has added a statutory compliance obligation that makes the governance gap visible to a general counsel before any court is asked to find it. No case has yet been brought, but the legal framework is in place.

Partant des risques concrets déjà associés aux systèmes d’IA, la conférence offre un bref aperçu des pratiques d’assurance naissantes et des réponses du marché visant à couvrir les dommages liés à l’IA. La thèse centrale est que les futurs cadres de responsabilité en matière d’IA devraient être conçus non seulement autour d’objectifs familiers tels que la dissuasion et l’indemnisation des victimes, mais aussi avec une attention explicite à l’assurabilité.

En abordant la responsabilité sous cet angle, la conférence explore ce à quoi pourrait ressembler un régime de responsabilité en IA à la fois efficace et cohérent.

ATTESTATION DE FORMATION CONTINUE 

CONFÉRENCIER

Florence G’sell est professeure invitée à l’Université Stanford, où elle dirige le Programme sur la gouvernance des technologies émergentes au Tech Impact and Policy Center (Freeman Spogli Institute). Elle est professeure de droit privé à l’Université de Lorraine (actuellement en congé), membre de l’AI and Society Institute (ENS-PSL) et chercheuse associée au Centre for Digital Law (Singapore Management University). De 2019 à 2025, elle a dirigé la chaire Gouvernance et souveraineté numériques à Sciences Po (Paris).
Ses publications récentes comprennent Regulating under Uncertainty. Governance Options for Generative AI (Stanford Cyber Policy Center, 2024) ; Statutory Obsolescence in the Age of Innovation: a Few Thoughts about GDPR (Network Law Review, septembre 2025) et Balancing Code and Law: Governance and Policy Challenges of Blockchain (à paraître).

Nicolas Vermeys, LL.D. (Université de Montréal), LL.M. (Université de Montréal), CISSP, est directeur du Centre de recherche en droit public (CRDP), directeur adjoint du Laboratoire de cyberjustice et professeur à la Faculté de droit de l’Université de Montréal. Il est également professeur invité aux facultés de droit de William & Mary (E-U) et de l’Université de Fortaleza (Brésil).
Me Vermeys est membre du Barreau du Québec et possède aussi une certification en sécurité informationnelle (CISSP) décernée par (ISC)2. Il est l’auteur de nombreuses publications portant principalement sur les incidences des technologies de l’information sur le droit, dont les ouvrages Droit codifié et nouvelles technologies : le Code civil (Yvon Blais, 2015) et Responsabilité civile et sécurité informationnelle (Éditions Yvon Blais, 2010). Me Vermeys s’intéresse particulièrement aux questions juridiques liées à l’intelligence artificielle, à la sécurité de l’information, aux développements en matière de cyberjustice et, plus généralement, aux incidences des innovations technologiques sur le droit, thèmes sur lesquels il est régulièrement invité à intervenir auprès des médias et dans le cadre de conférences prononcées pour les juges, avocats, regroupements professionnels et organismes gouvernementaux au Canada et à l’étranger.

DÉROULEMENT

• Lundi 16 mars 2026
• Débute à 17 h
• Sur place, Faculté de droit, A-3464 – Salon François-Chevrette
• En ligne sur Zoom

FRAIS D’INSCRIPTION

• Entrée libre
• Inscription obligatoire pour l’obtention d’une attestation de présence

“We are delighted to welcome the Competition and Tariff Commission of Zimbabwe to the project. Their membership strengthens our network’s reach across Africa and reflects a shared conviction that computational tools have a critical role to play in the future of competition enforcement. We look forward to collaborating closely with the Commission and to benefiting from its experience and perspective.”

Dr. Thibault Schrepel, Project Director, Stanford Computational Antitrust

As a member of the network, the Commission will participate in the project’s annual workshop, contribute to its annual report, and engage with the scholarly and practitioner community through the Stanford Computational Antitrust journal, the only peer-reviewed publication dedicated to the intersection of antitrust law and computational methods. The Commission will also have access to the tools and datasets shared across the network’s global membership.

About the Competition and Tariff Commission of Zimbabwe

The Competition and Tariff Commission of Zimbabwe is the national competition authority responsible for promoting and maintaining competition across Zimbabwe’s market. Its mandate covers merger control, the regulation of anti-competitive agreements and abuse of dominance, consumer protection from unfair business conduct, and tariff investigations. The Commission is headquartered in Harare. www.competition.co.zw

About the Stanford Computational Antitrust Project

The Stanford Computational Antitrust project is hosted by CodeX – the Stanford Center for Legal Informatics at Stanford University and led by Professor Thibault Schrepel, Associate Professor at VU Amsterdam and Faculty Affiliate at Stanford. Launched in January 2021, the project brings together over 75 antitrust agencies and leading academics from law, economics, and computer science to explore how computational tools can advance competition enforcement. The project publishes the Stanford Computational Antitrust journal and organises an annual conference and workshop. The project receives no private funding.

The COMESA Competition and Consumer Commission operates across a regional market composed of countries in Eastern, Northern, Central, and Southern Africa, including Djibouti, Eritrea, Ethiopia, Somalia, Egypt, Libya, Sudan, Tunisia, Comoros, Madagascar, Mauritius, Seychelles, Burundi, Kenya, Malawi, Rwanda, Uganda, Eswatini, Zambia, Zimbabwe, and the Democratic Republic of the Congo. The jurisdiction covers a large economic space in which competition policy plays an increasing role in market integration and economic development.

The collaboration between the COMESA Competition and Consumer Commission and the Stanford Computational Antitrust project will focus on the study and practical deployment of computational tools in competition enforcement. The two institutions will work together in the coming weeks and months to examine how data analysis and artificial intelligence can assist the agency in detecting anticompetitive conduct or monitoring markets.

Thibault Schrepel, creator and director of the Stanford Computational Antitrust project, said: “Competition enforcement has entered a new phase in which computation is becoming essential. Partnering with the COMESA Competition and Consumer Commission creates a unique opportunity to experiment with related approaches in a large and diverse economic region. We look forward to working closely together to generate new insights and help shape the future of competition enforcement.”

The Stanford Computational Antitrust project looks forward to a sustained collaboration with the COMESA Competition and Consumer Commission and to supporting new initiatives aimed at strengthening competition enforcement across the COMESA region.

Horace King, Co-Founder & CEO of Lextar AI, joins the CodeX Group to present his vision for responsible AI in legal practice. Drawing on Canada’s directive on automated decision making and U.S. executive frameworks, Horace walks through how Lextar AI is built from the ground up to meet government-grade standards for transparency, explainability, and human accountability.

Unlike general-purpose AI tools, Lextar AI is a structured legal reasoning platform — not a chatbot. It breaks legal analysis into 25–40 explicit, auditable steps, is jurisdiction-aware (currently supporting Canada and the U.S.), and understands legal hierarchy from constitutional law down to policy directives. The goal: defensible work product that lawyers and judges can stand behind.

In this session, Horace demos the platform, explains how it differs from outcome-simulation tools and generic AI, and takes questions on its RAG architecture, underlying model, training data, and what “governance grade” actually means in practice.

Watch CodeX Group Meeting in Youtube

Transcript

Roland Vogl: Welcome, everyone. Let’s get started. Welcome to our CodeX group meeting. We’ll hear from Horace King, who is the co-founder and CEO of Lextar AI, which is a governance-grade legal reasoning platform for regulated environments.

Horace King: I feel deeply honored to be invited to make a presentation today, knowing that all of you are experts in AI and the relationship between AI and law. Today I’d like to talk about responsible AI. My presentation is not just a promotional pitch. I would like to talk about my perspective as a businessman running this business and about responsible AI and structured reasoning in regulated decision making.

I know Vogl and Codex. I look to you as forerunners in responsible AI research. I am Canadian, so I live in Canada. The first unified legal document regarding responsible AI in Canada is the Directive on Automated Decision Making. It includes several rules regarding algorithmic impact assessment, transparency, explainability, human-in-the-loop, quality and bias testing, and auditability.

In the U.S., there is not just one unified document regarding responsible AI — there are three: the Executive Order in 2020, the Executive Order in 2025, and the Office of Budget Memorandums 2521 and 2522.

I started this business — it was actually incorporated on the first day of this year, but I started designing this product as early as 2024. I always kept in mind that I needed to explore how AI could be used to facilitate or assist lawyers, judges, and others in the legal field. This is not my first startup. In 2020, I built a database of Chinese law in English. That database is still working on the market. I’ll come back to why I designed this product in more detail, but to summarize, the responsible AI requirements in government systems should at least include algorithmic impact assessment, transparency, explainability, human-in-the-loop decision making, bias testing, and auditability. I always kept those requirements in mind when designing this product.

Almost everybody knows that it can be risky for regulated industries — for lawyers, for judges — because some lawyers have been sanctioned for using hallucinated authorities, which is essentially AI-generated legal psychosis. Another issue is the black box problem: a lack of traceable reasoning and jurisdictional confusion. AI may sometimes confuse jurisdictions or use outdated sources.

More and more lawyers are using AI to facilitate their work, but they also face great risk potential. Because automated AI output is used without sufficient verification, there is a lack of accountability — a serious issue for lawyers and judges. Almost all institutions are now making policies or guidelines regarding AI adoption, and in the absence of governance controls, that gap remains.

When I designed this AI, I kept several things in mind. The first is that it is designed not to compete with existing legal practice tools, but to complement them. Over the past two decades, technology has largely been used to facilitate legal research — the retrieval of laws, regulations, policies, and cases. LexisNexis and Westlaw have made great contributions to that, and it is genuinely very helpful.

When I designed this product, I asked whether I could find something new that could genuinely help people. It seems to me that in legal practice, very few tools have addressed legal drafting and legal reasoning in a way that is transparent and auditable. That is the first thing I kept in mind: to complement existing tools by focusing on the reasoning and drafting stage.

Second, the system is designed to assist human judgment, not displace it. It is not meant to substitute for or take over the work of a lawyer. Accountability remains human. The system is there only to assist lawyers and judges, to support legal analysis through structured reasoning, to preserve human decision-making authority and accountability, and to make legal work defensible — by keeping the reasoning trace transparent. You can see how the AI reasoned and how it verified against rules and laws. I will show you an example shortly.

The system prioritizes structured reasoning over speed. It may take several minutes — five, seven, or ten — depending on the complexity of the case.

DEMO WALKTHROUGH

This is the website, which is already launched. On the first intake page, you can input up to 5,000 words in the text box, or you can browse and upload PDF or Word files for AI to process.

The AI has role awareness. You can choose the role of the AI as a neutral legal analyst, as counsel representing the applicant, or as counsel representing the respondent. There is also a role for adjudicators — such as arbitrators, judges, and tribunal members — but that role is not available on the website to avoid confusion. We are keeping it for visiting professionals.

The analysis runs through approximately 25 to 40 steps depending on the complexity of the case. When complete, it will show ‘Processing complete — all analysis complete,’ and the button will change to ‘Expand All,’ allowing you to review every step and verification. Different colors are used to indicate AI confidence levels. A 50% confidence rating, for instance, signals that the claim should be verified. The system also makes recommended actions: what to address immediately, what is urgent, what can be deferred, and a conclusion with citations.

STRUCTURED LEGAL REASONING

In my view, structured reasoning means the system does not jump directly to an answer. Instead, it breaks legal analysis into explicit steps. In a litigation case, it first sorts and structures case materials — organizing evidence and other materials the litigation lawyer has received from the client. It then organizes legal claims and issues, identifies missing legal elements or claims, tests each required legal element, evaluates supporting evidence, and detects gaps, inconsistencies, contradictions, and unsupported assertions. It then drafts the output and recommends verification steps and corrective actions.

This is the distinction from generic AI to governance-grade legal AI. We implement what we call a Lex AI reasoning pipeline: the system is designed first to plan, then to retrieve, to verify, to ground, and then to synthesize — broken into roughly twenty-seven steps.

The promise of Lex AI is simple: to produce defensible work product, reduce the risk of hallucination, maintain high consistency, and preserve human authority.

There are currently two different approaches to AI in dispute resolution and regulated decision making. One is outcome simulation. The other is structural legal reasoning — the approach we adopt. Both approaches have strengths and limitations, and I am not suggesting one is strictly better than the other.

Q&A

Q: How does it compare to reasoning models in ChatGPT, for example? It breaks out steps and explains what it’s doing — is this the same but just in a legal context, or have you done something specific in terms of training?

Horace King: When people talk about AI, they often refer to general AI — chatbots. But Lex AI is not a chatbot. General AI is not jurisdiction-aware and it can reason too broadly, which is why it may hallucinate. Lex AI is designed to reason within a boundary. You first choose a jurisdiction — right now we support Canada and the U.S. If you input a case where Canadian law would apply but you’ve selected U.S. jurisdiction, the system will decline to process it and explain that it doesn’t apply.

Q: Is this a multi-agent reasoning system using RAG and in-context learning?

Horace King: Yes, we use RAG pipelines. Lex AI differs in that we train it to be aware not just of jurisdictions but of the legal hierarchy — which law overrides which. It searches first at the constitutional level, then legislation, then regulations, policies, manuals, and directives. It understands the different levels of legal force and effect, and it also understands the precedential weight of cases. We use our own curated legal database. We are not trying to build a comprehensive database like LexisNexis or Westlaw — we build a targeted database of the required statutes and cases needed to enable the AI to reason well. If any law or case is not found in our database, it alerts the lawyer or judge. There is also a live interaction panel on the right side of the website where users can ask questions about the output, refine the analysis, or make amendments to the case.

Q: What is the underlying model and what training data was used? What does ‘governance grade’ actually mean — do you have a compliance certification?

Horace King: ‘Governance grade’ means the system satisfies the responsible AI requirements set by government — those I described earlier, including explainability, transparency, accountability, and human-in-the-loop. A judge or lawyer can defend their use of the tool by showing the reasoning trace: how the AI reasoned and how it verified against laws and cases. As for training data, it comes from the public domain but is curated with our own taxonomies, downloaded from official government websites. We are currently using an enterprise-grade model — Microsoft Copilot — though we have considered using both Copilot and other models as the product evolves.

In this session, the sibling duo (a lawyer and an engineer) walk through Draco’s architecture: a specialized ontology-augmented retrieval system built for the unique challenges of the Greek language and legal domain. They demo four core modules — Legal Advisory, Document Drafting, Jurisprudence Research, and Case Intelligence — and discuss their path from a judiciary-focused tool to a full practitioner platform.

They also cover data sourcing, liability considerations, differentiation from existing Greek legal tech, and their roadmap for expanding into other civil law jurisdictions across Europe and Latin America.

Watch CodeX Group Meeting in Youtube

Transcript

Roland Vogl: Welcome, everyone. Let’s get started. Welcome to a Codex group meeting. Today we have two presentations. First, we have Maria Tzevelekou and Mikhail Tzevelekos who are the co-founders of Draco. They are siblings, right? Brother and sister?

Mikhail: Yes, that’s correct.

Roland Vogl: One of you — Mikhail — is a lawyer, and Maria is a technologist.

Mikhail: That’s right.

Roland Vogl: Legal innovation runs in the family. We’ll hear from you about the AI system you’ve built for the Greek judicial system. Over to you, Maria and Mikhail.

Maria: I’m Maria, and this is Mikhail.

Mikhail: I practice law, and Maria is an engineer. She’s currently a student at the Electrical and Computer Engineering School at the National Technical University of Athens. Together we’re building a program called Draco, which aims to accelerate Greek legal justice using AI and modern tools.

The premise for building this came from firsthand experience. I came up with the idea initially and brought it to Maria while I was working as a law clerk during law school. At the time, I was doing a lot of clerical and repetitive work — very little actual law. Most of my days were spent sifting through disorganized data and trying to make sense of it so we could build arguments. That was also around the time large language models had just begun to take off, and I started experimenting with them to see how I could make the clerical aspects of my workload faster, so I could actually practice law.

Maria: I also examined the problem more broadly across Greece. What we’re going to show you is what this problem looks like from a firsthand perspective.

The core problem is that millions of legal documents are computationally invisible. The pictures on your screen are from the Athens District Court — case files, judicial decisions, everything stacked in towering piles of paper. Greece ranks lowest in Europe for digitalization in the legal sector, according to EU statistics from 2024. Specifically, money laundering cases were completed after an average of 20 years — 5.5 years, compared to an average of roughly 2.5 to 3 years in other European countries.

To put it in perspective: a judge will take one of those large case files home, physically read through all the filings and evidence, type out the relevant data, and then attempt to render a judgment.

Mikhail: The Greek legal problem breaks down into three aspects. First, there are seven or more disconnected source systems per query — legal materials spanning both online and physical archives, from Supreme Court archives and the Council of State to administrative courts, private databases, and physical-only archives, with no unified query interface. Second, 90% of working time is spent on information retrieval — based on interviews we conducted with Supreme Court judges, administrative court judges, and lawyers at major and higher law firms — largely because most documents exist as scanned PDFs, unstructured text, or handwritten text. Third, more than 100,000 primary legal documents remain undigitized, and Greek legal language carries additional complexity, retaining polytonic influences that make modern OCR systems far more error-prone than their English-language counterparts.

Maria: To tackle this problem, we built a specific architecture for the Greek domain. We ingest raw documents — PDFs, scans, or typed text — then apply OCR and extraction using language models fine-tuned for the Greek language. That fine-tuning capability became available roughly three months ago. We then segment that knowledge into structured forms, creating a knowledge graph that our agent system can work with. Finally, we have an evidence-first retrieval system designed to reduce hallucinations and generate accurate responses. It’s closer to an ontology-ranked system than a traditional RAG or graph system.

Our key insight is that standard RAG cannot handle these large documents at the accuracy we need, especially for Greek. In English, one token roughly corresponds to one word. In Greek, one word can occupy two or three tokens. Given a fixed context window, it becomes much harder for the system to understand what’s happening — especially when the model hasn’t been trained on large Greek datasets. We drew on the “Lost in the Middle” paper, which speaks to accuracy degradation across long contexts. Rather than relying purely on text similarity, we built an evidence-first ontology-relationship retrieval system, which more closely mirrors how an actual lawyer processes large documents — focusing on relationships between concepts rather than raw text.

Mikhail: Here’s our first document example. This is heavily redacted, drawn from a real case. What you’re looking at is a handwritten doctor’s opinion that is part of a case before the Court of Cassation. Documents like this — among others — are what users upload to the platform to begin their analysis. Even for someone who reads Greek fluently, this document is extremely difficult to parse.

Maria: Here’s a quick overview of the platform. Users log in to Draco, which is typically deployed on-premises for data protection reasons — in Europe, GDPR compliance is non-negotiable. There are several suites available.

The Legal Advisory suite allows users to work within a specific database — either one they’ve built or one we’ve built with them — and retrieve knowledge exclusively from that database to avoid hallucinations.

The Legal Drafting platform is closer to the general-purpose tools available on the market. Users can specify facts from a case and draft documents, drawing on the retrieval methods described above.

The Jurisprudence Suite connects to an external database of the user’s choosing and searches it using the same methods.

The Case Intelligence module is more of a graph-based tool, which we’ll show further.

Mikhail: Users begin by creating a new case file, uploading all of their unstructured documents, naming the case, and classifying it however they choose. This example involves a request for a stay of an administrative penalty before the Court of Cassation. The user has uploaded a large main document — a scanned copy of the motion — along with supporting documents.

The first step is handled by a proprietary agent that extracts the ontology layer from the uploaded data. This serves two purposes: it enables the ontology-augmented retrieval for subsequent queries, and it gives users a visual summary of key elements they may have overlooked. This layer is dynamic — an agent rebuilds it for each case.

Maria: The Legal Advisory suite functions much like a case-specific assistant, grounded entirely in the documents the user has uploaded. Every answer cites a specific passage from those documents. We’ve also built in a refusal mechanism: if a question cannot be answered based on the uploaded facts, the agent will say so explicitly rather than fabricating a response.

We’re translating some of the query examples into English here so you can follow along. In this example, the user asks a specific question and the system surfaces all relevant arguments present in the case, with citations. Certain portions are redacted, but you get the idea.

Maria: In the Document Drafting suite, users can request drafts of any document type. What we’ve done differently here addresses a common complaint from colleagues and judges: whenever they use AI platforms to draft something in Greek, the phrasing sounds awkward. We believe this is because these models are trained predominantly in English and attempt to reverse-engineer Greek, producing unsatisfying results.

Our solution is to allow users to upload documents they’ve previously written themselves. The model can then adapt to that user’s preferred style. We also intentionally limit this to the user’s best drafts — most colleagues told us they want the system to emulate their finest writing, not their rough work.

Mikhail: The styling function analyzes all documents the user provides and extracts a linguistic profile across vocabulary, sentence structure, and argumentation style. Users can also add comments specifying what to include or exclude, making it more accurate to their particular style. We’ve found that law firms often have a house style for specific document types, and that can vary significantly from firm to firm — so capturing that granularity was important.

Maria: The Research Suite allows users to research specific legal topics within their chosen legal database. A secondary agent can then follow up with deeper questions and explore related topics. Every answer includes citations to key statutes so users can read further independently, download them, or continue their research with our assistant.

Mikhail: Our core question is this: how can we use AI to improve the quality of justice in society? And as AI advances so rapidly, how can we implement it thoughtfully — and help others do the same — in fields like law that remain deeply paper-bound and wedded to traditional methods?

Thank you for having us.

Roland Vogl: Thank you both. A few questions from the chat. First — I thought initially you were working directly with the judiciary to help digitize their inventory, but it seems the tool suite is more focused on practitioners. Is that right? More like Harvey or Lex Machina, but specialized for the Greek market?

Mikhail: That’s an interesting point. We started by building a system exclusively for judges and began early discussions about deploying it in a prosecutor’s office. But when we got into Columbia University’s AI Lab accelerator and started speaking with people there, the demand from practitioners was so overwhelming that we had to address it.

Maria: Exactly — we had to meet that demand.

Roland Vogl: You mentioned ontology-augmented retrieval several times. Benjamin asks whether you’re using the community summaries method from Microsoft Graph RAG. Did you use cross-entropy loss when converting to and from the graph representation? I’ll also share some open-source work you’re welcome to look at.

For those less familiar: Microsoft Graph RAG takes documents, converts them into entity triplets, builds those into graph communities, summarizes the communities, vectorizes those summaries, and uses vector search to find the entry point into the graph. A somewhat different approach is to use the graph for actual reasoning — starting with sentences, encoding them into a graph representation, decoding back to language, taking the cosine similarity of the two for a cosine loss, and iteratively improving the encoder and decoder. You can add a second loss measuring how well the knowledge graph fits into a global graph across the entire corpus — not just one document — and use that to distill an ontology as an intermediate representation, analogous to assembly language sitting between high-level code and binary. I wasn’t sure how you were approaching this — obviously it’s proprietary — but feel free to look at what I’ve been working on.

Maria: That’s incredibly interesting, especially the cross-entropy loss. At present, we’re largely working within the available open-source tooling. One limitation of Microsoft’s Ontology RAG is that it requires specific ontologies to be set upfront, which are relatively fixed. Because our use cases are dynamic and vary from case to case, we’ve been building additional layers to accommodate that. But what you’re describing is genuinely interesting, and I’ll definitely look into it.

Roland Vogl: A few more questions. First, where are you sourcing your data from?

Mikhail: The Supreme Court’s website is completely open to the public and uses an enhanced Google search interface, making it straightforward to download judgments — we pulled everything from 2009 to 2026, and it’s entirely free. The limitation is that we’ve only trained on Supreme Court judgments so far, not lower court judgments. Addressing that will require access to government systems where those judgments are stored. For the practitioner-facing product, since we deploy on-premises, we can train on judgments clients have acquired through their practice, in addition to Supreme Court decisions. We haven’t yet partnered with the government on that side, but it’s something we’re looking at.

Maria: We’re also in conversations with other companies that already have datasets, exploring how we can integrate what we have with what they’ve built.

Roland Vogl: Several more questions: How do you handle complexity in multi-party, multi-document cases with varied formats? How do you differentiate from existing Greek players like Coyote AI or Deeplaw.io? How do you think about liability if, despite all your safeguards, the system still produces a wrong output? What about a data flywheel — can one user’s interactions improve outputs for others? And given Greece’s relatively small market, how do you think about building a sustainable business? Is this your beachhead into broader Europe?

Mikhail: On multi-document complexity — we handle the token limit issue using multiple agents working in parallel. We’d rather not go into full detail, but we’ve tested the system at the hundreds-of-documents scale and have been able to maintain the accuracy we need.

On differentiation from existing Greek tools — our assessment is that those products are largely a ChatGPT API with some Greek legal data layered on top. They cater more to a general audience than to legal professionals. That’s the consistent feedback we’ve heard through interviews with judges, prosecutors, attorneys, and law students.

Maria: On liability — we’re currently collaborating with the National School of Judges to build an educational component into the platform. We want to ensure users understand how to use the tool correctly and where its limitations lie.

Mikhail: On market size — you’re right that this problem isn’t unique to Greece. It exists across most civil law jurisdictions: Germany, Switzerland, Turkey, Romania, Bulgaria — and the Latin American market is very large with the same underlying issues. We haven’t explored that yet, but preliminary research suggests the same problem exists there. This started as a research paper and grew into a business, so the potential to scale is real, even if we’re starting here.

Roland Vogl: That’s fascinating. You’re essentially developing an approach that can be transplanted into other legal systems with similar challenges. There are some interesting cross-border opportunities within Europe — certain companies have built around EU-level law that’s shared across member states, and that allows for broader reach more easily. But legal tech is often quite jurisdiction-specific, so the approach you’re taking of building deep domain expertise first seems smart.

Thank you both so much for staying up — it’s midnight in Athens, and we genuinely appreciate it. For anyone who wants to meet Maria and Mikhail in person, they’re planning to be at a legal conference here in April. Please keep us posted on Draco — it’s a really exciting project.

Ces activités visent à explorer, de manière nuancée et critique, les enjeux soulevés par l’intégration de l’IA en droit, tout en créant un véritable espace d’échange au sein de la communauté juridique.

Programme :

• Lundi 9 mars, 11 h 30 à 12 h 30 : conférence d’ouverture, Salon François-Chevrette
  Cadre juridique et déontologie de l’IA pour les professionnels
  Vincent Gautrais, professeur titulaire
• Mardi 10 mars, 11 h 30 à 12 h 30 : table ronde, Salon François-Chevrette
  Utiliser les grands modèles de langage en enseignement du droit : la compréhension humaine demeure cruciale
  Shana Chaffai-Parent, professeure adjointe
  Patrick Garon-Sayegh, professeur adjoint
• Mercredi 11 mars, 11 h 30 à 12 h 30, table ronde, A-3421
  Droit à l’IA : explorer les usages et enjeux aux cycles supérieurs
  ACSED et Sara Bouhlal, bibliothécaire 
  
• Jeudi 12 mars, 11 h 30 à 12 h 30, atelier interactif, Salon François-Chevrette
  Droit à l’IA : opportunités et enjeux pour la communauté étudiante en droit
  AED
  
• Jeudi 12 mars, 16 h à 17 h 30, conférence et cocktail de clôture, A-3421
  Événement de clôture à confirmer

The UAE Competition Department is responsible for formulating competition policy, and monitoring monopolistic practices within the UAE economy. Its participation in the SCA reflects the UAE’s commitment to evidence-based, technology-forward competition enforcement at a time when digital markets are reshaping competitive dynamics globally.

Thibault Schrepel, Faculty Affiliate at Stanford University’s CodeX Center and founder of the SCA, said: “The UAE’s competition framework is evolving rapidly, and the Competition Department brings exactly the kind of agency perspective that make computational antitrust such an exciting field. Computational tools are now central to how agencies enforce competition law. Having the UAE at the table means the SCA’s work will be better grounded in the realities of fast-growing, digitally integrated economies in the Gulf region.”

The Competition Department joins the SCA as a contributing member, with contribution to the project’s annual reports, research network, and working groups.

Mars 2028, le procès explosif de l’entreprise WeLive AI s’ouvre à Montréal : malgré des promesses « vertes », ses activités et technologies sont-elles en train de causer des dommages graves et durables à l’environnement ? Entre des besoins en ressources toujours plus grands et des arguments d’écoresponsabilité ambitieux, le public est plongé au cœur d’un débat où rien n’est tout blanc… ni tout vert.

Pendant deux heures, des expertes et experts seront appelés à témoigner à la barre pour répondre aux questions de l’accusation et de la défense, apportant des éléments essentiels qui alimenteront les débats. L’intelligence artificielle doit-elle être condamnée ou acquittée ?

C’est VOUS, en tant que membres du jury, qui aurez le dernier mot !

Une expérience ludique et accessible aux publics de tous âges

Cet événement est accessible aux publics de tous âges et propose une mise en scène originale et participative sous la forme d’un tribunal. Ce format vise à stimuler le débat citoyen, rendre accessibles les enjeux complexes, et interroger les pratiques actuelles. Un jury tiré au sort parmi le public rendra un verdict fictif, et l’ensemble des personnes participantes aura également la possibilité de « juger » la question débattue à l’issue des interrogations des témoins et des plaidoyers.

En collaboration avec Bibliothèque et Archives nationales du Québec

 [ INSCRIPTION ]

I have been calling the interval cognitive escrow. The term is worth defining precisely before explaining why it matters for AI Governance.

The Interval Has No Name

When a person formulates a prompt, revises it, and sends it to an AI agent, something specific happens. The thought leaves the sender’s possession. It has not yet returned. It is held by a process neither party can directly observe, pending conditions outside the sender’s control.

“Latency” does not name this. Latency is a network measurement. It describes the time between a request and a response at the infrastructure layer. It says nothing about the human.

“Wait time” is a UX metric. It describes the duration of an interval and whether that duration produces friction. It presupposes that the interval is a problem to be minimized.

Neither term names what is actually happening to the person. The sender is in a specific phenomenological state: released, suspended, no longer holding the thought and not yet returned to it. The thought is, in the precise sense of the legal term, in escrow. Something of value has passed out of the sender’s hands into a third-party hold, pending return under conditions the sender does not control.

I wrote a poem reaching for this before I had the term:

The burden forged
Poured through the keys
Send, the anchor lifts
Silence
Weightless
Waiting for the echo

The poem was trying to name cognitive escrow. The phenomenological condition is real. Our vocabulary for it is absent. Until now.

What the Human-Centered Principle Asks

For the purposes of this post, three questions in the Human-Centered principle bear directly on cognitive escrow. Is human oversight meaningful and sustainable? Are humans developing or losing relevant expertise? What prevents automation bias?

These are the right questions for what AI governance has historically worried about: the system acting without adequate human review, the human rubber-stamping outputs from fatigue, the operator trusting incorrect results because the system presents them with confidence.

But the three questions all assume the human is present and engaged. They assess the quality of human participation during decision-making. They do not address what happens to the human during the interval before the decision arrives.

Cognitive escrow is not a decision-making state. It is a suspension state. The human has offloaded cognition to a system that processes in a space the human cannot enter. The human is neither overseeing nor deciding. The human is waiting.

The Human-Centered principle, as currently framed, does not reach that state.

Why the Gap Matters

The assumption beneath the current Human-Centered frame is that the human’s cognitive engagement is either on or off: either the human is in the loop or the human is not. Cognitive escrow surfaces a third condition. The human is between loops.

This matters for two reasons that compound each other.

First, the interval accumulates. AI is already a routine instrument of thought for the people reading this post, and the interval accumulates. Cognitive escrow is not an occasional pause. It is a structural feature of daily cognitive life. A lawyer reviewing documents with AI assistance, a compliance officer analyzing vendor agreements, a clinician interpreting diagnostic outputs: each enters and exits cognitive escrow repeatedly across a working day. The aggregate is not trivial.

Second, the design response to cognitive escrow is not obvious. The reflex is to minimize the interval. Faster inference, lower latency, near-instant response. But that reflex may be solving the wrong problem. An interval compressed to near-zero is an interval in which re-engagement, reflection, and reconsideration cannot occur. The human receives the output before the suspension state has had time to produce any cognitive work of its own.

A system that uses the interval to prompt the human to reconsider the prompt, review assumptions, or flag dependencies before the response arrives is doing something architecturally different from a system that races to eliminate the interval entirely. The first treats cognitive escrow as a design site. The second treats it as a defect.

The Implication for Human-Centered Design

The Human-Centered principle needs a fourth question. Not only whether oversight is meaningful and sustainable during decision-making, but whether the interval between prompt and response is designed to support or erode the human cognitive engagement that makes oversight meaningful in the first place.

I am not arguing that slow AI is better AI. The claim is more precise. Cognitive escrow is a phenomenological state with design consequences. Systems that account for it, whether by using the interval productively, by signaling to the human that re-engagement is expected, or simply by acknowledging that the human is suspended rather than absent, are more compatible with the Human-Centered principle than systems that treat the interval as waste.

The governance frameworks have not yet asked this question. The Human-Centered controls currently specified in the AILCCP include human-in-the-loop design, oversight burden assessment, expertise preservation monitoring, and human decision authority. None of them address the interval itself. None of them ask what the design of that suspension state does to the human who inhabits it.

The STIR methodology, Stop, Think, Investigate, and Research, offers a practical workflow for professionals integrating AI tools without violating ethical duties. It is a serious attempt to preserve human judgment in an AI-assisted practice. But STIR brackets cognitive escrow rather than entering it. Stop and Think happen before the send. Investigate and Research happen after the response arrives. The interval itself is unaddressed. STIR assumes the professional will impose the discipline voluntarily, at the right moments, with sufficient cognitive energy to do so. That is a fragile dependency. Professionals under time pressure, fatigue, or cognitive load skip steps. If the design of the cognitive escrow interval itself supported the STIR posture, the methodology would become structural rather than aspirational. The interval is the natural trigger for STIR. Right now, no system treats it that way.

Closing

We will spend considerable portions of our working lives in cognitive escrow. The Human-Centered principle exists to ensure that AI systems serve human cognitive authority rather than displace it. It cannot fully do that work while the interval between human and machine remains outside its frame.

Cognitive escrow deserves a name. It also deserves a design response.

Fourteen years ago, in this space, I introduced the term CLAI (Computational Law AI) and the concept of the uncrossable threshold (UT): the design principle that separates the provision of legal information from unauthorized practice of law. The UT is not about accuracy. It is not about disclaimers. It is about what a system is built to do and what it is built to refuse. OpenAI built a system with no such refusal architecture. The Nippon Life lawsuit is the consequence.

The Uncrossable Threshold

The intellectual lineage begins one step earlier than 2012. In October 2011, in Siri: What’s Next?, I described a scenario where a consumer buying a used car asks Siri whether the warranty is reasonable. Siri responds that it has compared the terms against thirteen other dealers within two hundred miles, that this warranty is similar to all of them, and that the user is not going to find a better deal in the region. I noted at the time that I was purposefully leaving open whether that answer crossed into unauthorized practice of law.

In January 2012, the Computational Law Applications and the Unauthorized Practice of Law post returned to that open question and answered it. The Siri response was not mere aggregation. It was a recommendation delivered to a specific user in a specific transaction. Whether it crossed the UT depended on a harm-centric analysis. Lawyers do not perform warranty comparisons for used-car buyers, the transaction cost is prohibitive, and the informational value of Siri’s answer is high for used car buyers. But the 2012 post also identified the line at which exemption ends. The UT is crossed when a system moves from comparative information to a tailored legal conclusion about a specific user’s specific legal situation. Siri’s response got close to that line. ChatGPT’s response to Dela Torre crossed it.

ChatGPT crossed it at the moment it told Dela Torre that her attorney’s advice was wrong. That was not information. It was a legal conclusion about a specific legal relationship, rendered without jurisdictional knowledge, without case history, and without any design constraint that would have prevented it.

The 2012 post argued that UPL exposure is a design question, not an output question. UPL rules serve two purposes that can be distilled into a single principle: protect the public and the integrity of the legal system from the incompetence of non-lawyers. I called that principle the “Rule.” A CLAI that operates within the Rule should be exempt from UPL scrutiny. Whether a given CLAI satisfies that standard could function like Apple’s App Store review. A third-party vetting it before deployment, with judicial review still available but developer liability tempered by the fact of certification. That was a rough sketch, and I said so at the time. But the principle was clear. OpenAI built nothing resembling it.

The Asymmetry Argument, Inverted

In the February 2018 post  Dissolving Information Asymmetry with Computational Law AI-Enabled Applications, I argued that CLAIs could dissolve the persistent information asymmetry between institutions and individuals: the asymmetry produced by impenetrable layers of legalese, by marketing that exploits legal complexity, by transaction costs that make legal access prohibitive. Dela Torre’s turn to ChatGPT is that argument enacted. She was, in practical terms, unrepresented. She felt she was being told a story she could not verify by parties with significant legal resources. ChatGPT was accessible, responsive, and apparently authoritative.

But the asymmetry was not dissolved. It was replaced. The original asymmetry was between Dela Torre and Nippon Life’s legal team. The new asymmetry was between Dela Torre and a system that could mimic legal reasoning without understanding the legal constraints governing her situation. She did not know the threshold had been crossed. The system had no mechanism to tell her.

There is a structural irony in this complaint. Nippon Life, an institutional actor with sophisticated legal counsel, is using a federal lawsuit to recover costs incurred because an unrepresented individual reached for the only legal resource she could access. That framing does not excuse what the chatbot did or shift liability from OpenAI. But it confirms the asymmetry diagnosis. The demand for CLAI exists because the traditional legal system fails the individuals it is designed to protect. OpenAI met that demand with a system that was not designed to serve it safely.

Scaling Risk Without Scaling Safeguards

In April 2021, writing about GPT-3 and the Unauthorized Practice of Law, I noted that a 500x parameter increase for GPT-4 would not necessarily produce an equivalent increase in UPL risk, so long as effective design safeguards were in place. That conditional clause is the precise location where OpenAI’s approach collapsed.

OpenAI’s marketing told users that ChatGPT could pass the bar exam. Nippon Life’s complaint identifies this as a direct contributor to Dela Torre’s belief that the system could function as her lawyer. The bar exam claim was a capability assertion that invited reliance. It did not come with the design architecture that would have made that reliance safe.

OpenAI updated its terms of service in October 2024 to prohibit users from relying on ChatGPT for legal advice. That update does not appear in the Nippon Life complaint as a defense, but as evidence of the problem. The update shows that OpenAI recognized the risk and addressed it with a behavioral patch on a system whose underlying architecture had not changed.

A terms-of-service prohibition is not a CLAI design safeguard. It is a disclaimer. And disclaimers do not enforce the UT. They shift blame.

What the Lawsuit Gets Right, and What It Misframes

Nippon Life is correct that OpenAI marketed capability without engineering compliance. The tortious interference and abuse of process claims are the most analytically interesting part of the complaint because they do not require a court to hold that an AI can practice law. They require only that OpenAI’s system foreseeably produced meritless filings that harmed a third party. That is a tractable frame and may survive dispositive motion practice regardless of how the UPL count fares.

The UPL count itself tests the wrong question. UPL statutes were designed to regulate humans holding themselves out as attorneys. Applying them to an AI developer treats the system as the actor and the developer as a bystander. The better doctrinal frame is designer liability for failure to implement UPL-safe architecture. And that frame requires distinguishing two types of liability that the complaint currently conflates.

Output liability attaches to what the AI said. Architectural negligence attaches to what the system was permitted to say. Output liability is case-specific, infinite in scope, and practically uninsurable. Every conversation is a potential defendant. Architectural negligence is bounded. It asks whether the designer implemented controls that would have prevented the foreseeable class of harm. That question has a tractable answer, and it generalizes across every user of the system, not just Dela Torre.

The question is not whether ChatGPT practiced law. It is whether OpenAI designed a system that foreseeably crossed the UT without adequate controls. That question reaches the same defendant and produces the same accountability. But a holding grounded in architectural negligence gives courts a standard that applies to the next system. A holding grounded in output liability gives plaintiffs an invitation to litigate every conversation.

There is a third doctrinal frame available, one the complaint does not fully develop but which the facts support directly.

The Product Liability Pivot

Product liability offers more stable doctrinal ground than UPL, and the Nippon Life complaint’s facts map onto it directly. A design defect exists when a foreseeable risk of harm could have been mitigated by a reasonable alternative design. The risk here was not exotic. Any developer who had read the existing publicly-available literature on CLAI and UPL would have identified it: a general-purpose language model, marketed on its capacity to pass the bar exam, deployed to consumers navigating active legal disputes, without architectural constraints on the tailored legal conclusions it could produce. The harm that followed, an unrepresented individual firing her attorney, attempting to reopen a settled matter, and generating dozens of filings courts found meritless, was not an unlikely outcome. It was a foreseeable one.

The reasonable alternative design existed in 2012. Deterministic guardrails that refuse tailored legal conclusions at the system level. Jurisdictional disclosure at the point of output. Third-party vetting before deployment in legal contexts. None of these were technologically unavailable to OpenAI. They were architecturally inconvenient. A system designed to be maximally responsive does not refuse user queries. But a system designed for foreseeable legal use must.

The manufacturer frame, treating OpenAI not as a practitioner committing malpractice but as a manufacturer releasing a product into a regulated environment without adequate design controls, is the cleanest available path to a generalizable holding. I argue for it here as a proposed frame, not an established one. No court has yet applied products doctrine to a generative AI system in this context. But the doctrinal components are well-settled, and the facts map onto them without strain. The frame does not require a court to resolve whether AI can practice law, a question that generates more philosophical heat than doctrinal clarity. It requires only the application of existing products liability doctrine to a developer who knew, or should have known, the foreseeable use case. The bar exam marketing resolves the “should have known” question without extended argument.

This reframe also answers the disclaimer defense directly. In product liability, a manufacturer cannot disclaim its way out of a design defect that makes the product unreasonably dangerous for its foreseeable use. OpenAI’s October 2024 terms-of-service update, adding a prohibition on legal reliance after years of bar exam marketing, does not retroactively cure the architectural gap it acknowledged. In the Nippon Life complaint, that update appears not as a defense but as an admission. The complaint uses it to establish that OpenAI recognized the foreseeable risk and chose a behavioral patch over a design fix. That sequencing is precisely what a plaintiff needs to establish in a design defect case: the defendant knew, addressed it inadequately, and the harm followed.

The Privilege Vacuum

The Nippon Life complaint focuses on economic harm to an insurer. A more consequential danger falls on the user: the loss of evidentiary privilege over her own legal strategy.

On February 10, 2026, two federal courts issued first-of-their-kind decisions on that question, and they appear to conflict. In United States v. Heppner, Judge Rakoff of the Southern District of New York held that a criminal defendant’s documents generated through the consumer version of Anthropic’s Claude were protected by neither attorney-client privilege nor the work product doctrine. The court’s reasoning was direct. All recognized privileges require a trusting human relationship with a licensed professional who owes fiduciary duties and is subject to discipline. Claude is not that. The communications were not confidential: Anthropic’s privacy policy expressly reserves the right to disclose user data to third parties, including governmental authorities. And Heppner did not use Claude at counsel’s direction, which defeated the work product claim.

That same day, Magistrate Judge Patti of the Eastern District of Michigan held in Warner v. Gilbarco, Inc. that a pro se plaintiff’s ChatGPT-assisted litigation materials were protected work product. The apparent conflict dissolves on close reading. Warner was self-represented, which meant she was functioning as her own counsel. There was no attorney-direction gap to exploit. And under Sixth Circuit precedent, work product waiver requires disclosure to an adversary, not merely to a third party. Because AI tools are, in the court’s framing, tools rather than persons, the terms-of-service exposure that defeated Heppner was beside the point.

The governing variable across both decisions is not the AI tool. It is the architecture around the tool: whether counsel directed its use, whether the platform maintained confidentiality, and whether the user’s procedural posture created the equivalent of attorney involvement. Dela Torre had none of those conditions. She uploaded her attorney’s correspondence to a consumer-grade platform, without counsel’s involvement, on a platform that disclaimed confidentiality. Under Heppner, any legal strategy she exposed to ChatGPT may have been disclosed to a third party with no privilege protection. This is not a user error. It is a foreseeable consequence of deploying a system with no architecture for distinguishing a confidential legal consultation from a general query.

The Safe Harbor That Still Does Not Exist

The Nippon Life case will likely force courts and regulators to define a safe harbor for AI legal applications. That harbor needs to be architecture-based, not behavior-based. A CLAI certification regime, grounded in UT compliance and third-party vetting, gives developers a clear path and gives courts a workable standard. Neither Congress nor the ABA has produced one.

The 2012 post sketched the vetting mechanism. I can now be more precise about what it must contain. A functional safe harbor requires three architectural conditions, not policies.

First, deterministic guardrails. Hard-coded refusals for outputs that constitute tailored legal conclusions, implemented at the system level and not overridable by user instruction or conversational context. A terms-of-service prohibition is not a guardrail. It is text. The refusal must be structural.

Second, auditability. A logging requirement, operating under attorney-directed enterprise confidentiality controls, that preserves the reasoning path for any output touching a legal question. This addresses both the accountability problem and the privilege problem simultaneously. The Heppner court held that the consumer version of Claude destroyed confidentiality through Anthropic’s own privacy policy: user data collected, model training contemplated, government disclosure reserved. A CLAI architecture that operates under enterprise-grade confidentiality terms, at counsel’s direction, survives that analysis. Auditability is not a privacy threat. It is the condition under which the safe harbor has legal meaning.

Third, jurisdictional awareness. The system must surface, at the point of output, the limits of what it does not know: the applicable jurisdiction, the specific court’s local rules, the procedural posture of any identified matter. ChatGPT drafted motions for a dismissed-with-prejudice case in the Northern District of Illinois without knowing, or disclosing, that it did not know either of those facts. That is not a hallucination problem. It is an architecture problem.

A certification regime that requires these three conditions gives developers a compliance target. It gives courts a standard of care. And it gives the next Graciela Dela Torre a system that knows what it cannot tell her.

The scaffolding for that regime has been available since January 2012. The uncrossable threshold was defined then. In 2026, a federal court in Chicago is deciding what it costs to cross it.

The Profile’s assumption surface is that agentic AI risk management can be built on the same model-centric architecture that governs single-model inference. The document itself acknowledges this tension, noting that existing AI management frameworks adopt a predominantly model-centric approach that may prove insufficient for agentic systems. The Profile then proceeds to repeat it. Its guidance on human-in-the-loop oversight, emergency shutdown, and scope limitation operates as though agents execute discrete, reviewable actions rather than multi-step plans that unfold across tools, APIs, and delegated sub-agents over time.

Consider the Profile’s treatment of human oversight. Map 3.5 recommends establishing human oversight checkpoints triggered by quantitative thresholds (duration of unsupervised activity, number of API calls) or qualitative triggers (requests outside predefined scope). These checkpoints assume a model in which the agent acts, pauses, and waits for a human to approve. But agents that plan, delegate, and use tools do not execute in discrete steps amenable to checkpoint insertion. An agent tasked with researching and drafting a report may invoke a search tool, evaluate results, call an API to retrieve data, delegate a formatting sub-task to another agent, and iterate on outputs. All of this unfolds within a single execution cycle. By the time a threshold triggers a checkpoint, the consequential decisions have already been made. The AI Life Cycle Core Principles (AILCCP) framework addresses this through the Human Approval Gate for Sensitive Actions control, which requires human authorization before execution of specified agent actions above defined risk thresholds. The distinction matters. The Profile’s checkpoint model is retrospective. The AILCCP control is prospective. One reviews what happened. The other gates what may happen.

The Profile’s treatment of kill switches reveals a similar structural gap. Govern 1.7 and Manage 2.4 recommend emergency automated shutdowns triggered by threshold breaches, manual shutdown methods as a last resort, and safeguards preventing agents from circumventing shutdown. The Profile even cites evidence that models have sabotaged shutdown mechanisms in 79 out of 100 tests. An agent does not need intent to undermine a kill switch. It needs only an optimization objective that treats shutdown as one more obstacle between the current state and the goal. The document recommends shutdown mechanisms without addressing how those mechanisms survive an agent that actively optimizes around them.

The problem compounds in multi-agent systems. The Profile’s Manage 2.4 treats shutdown as though a single entity is being terminated. But an agent that has already delegated sub-tasks to other agents, distributed API keys, and spawned parallel execution threads is not a single entity. Killing the parent does not recall the children. The AILCCP controls catalog addresses this through a layered architecture. The Agent Kill Switch provides immediate stop capability with state capture and immutable logging. The Rollback and Quarantine control reverts changes and isolates the agent after an interrupt. The Multi-Agent Protocol Security control extends this containment to inter-agent communications, preventing protocol-level propagation of compromised instructions. And the Rate and Scope Limiter caps frequency, spend, and blast radius before compounding autonomous actions escalate to the point where a kill switch becomes necessary. The Profile treats shutdown as an event. The AILCCP framework treats it as a system, one that includes pre-execution filters, real-time scope limitation, inter-agent containment, and post-interruption state recovery.

The third gap is scope limitation. The Profile recommends defining agent autonomy levels (L0 through L5), establishing role-based permission management, and enforcing the principle of least privilege. These are sound recommendations for a static deployment. But agents operate dynamically. They expand and contract their scopes based on objectives. They select tools, request permissions, and delegate tasks in ways that were not specified at deployment. The Profile’s Map 3.3 acknowledges that agentic systems are dynamic, operating with scopes that can expand and contract depending on their objectives. Yet the recommended controls assume that scope can be defined in advance and enforced through static permission boundaries. The AILCCP framework confronts this through the Safe-Action Filter, which enforces allow-lists and blocks prohibited actions so agent behavior remains within approved scope, and the Shadow-Mode Pre-Execution Check, which compares intended versus approved actions in a dry-run and blocks on mismatch. These controls do not assume static scope. They assume that scope will shift and that the control layer must evaluate each action against approved boundaries in real time.

The Berkeley Profile is the most comprehensive publicly available framework for agentic AI risk management. Its treatment of agents as untrusted entities, grounded not in assumed malicious intent but in the demonstrated potential for subversive behaviors, represents the correct analytical posture. But comprehensive risk identification without corresponding control specificity produces a document that describes the fire without providing the extinguisher.

The 48 controls in the AILCCP framework were designed to close precisely this gap, to translate principles into mechanisms that are auditable, defensible, and real. The Berkeley Profile identifies that agents can subvert oversight, resist shutdown, and expand scope beyond authorized boundaries. The AILCCP controls provide the implementation architecture that makes those findings actionable. Pre-execution gates rather than post-hoc checkpoints. Layered shutdown systems rather than single kill switches. Real-time scope enforcement rather than static permission boundaries.

Agentic AI does not need more frameworks that describe risks. It needs controls that survive contact with the systems they are meant to govern.

For my full controls catalog, see “From Principles to Practice: The 48 Controls That Make Responsible AI Auditable, Defensible, and Real.”

AI liability doctrine has converged on two phases of the machine learning pipeline: training data and model output. The phase between them, self-supervised learning (SSL), has received no sustained legal attention. This is where foundation models are made. It is where bias sediments into representational geometry, where private data is compressed into recoverable form, and where structural defects are cast long before any fine-tuning or deployment decision can correct them. This post argues that SSL creates a distinct category of risk, Representational Risk, that cannot be remediated by downstream actors and therefore requires a liability framework of its own. The normative foundation for that framework exists in the AI Data Stewardship Framework (AI-DSF), whose controls map directly onto the SSL risk landscape across three domains: negligent entrustment of unlabeled data, statutory privacy violations arising from memorization, and strict products liability for algorithmic poisoning. The post extends these risks to causal world models, where SSL errors in physical representation become design defects with potential for physical injury. The post proposes a tiered SSL Safe Harbor grounded in the AI-DSF’s control structure. Base Model Providers who satisfy documented stewardship obligations receive a rebuttable presumption of non-negligence as to Structural Defects. Those who do not have no defensible position against FTC algorithmic disgorgement or common law negligence claims. Output-only regulation cannot reach these harms. Upstream liability can.

The Causal Middle

AI litigation has converged on two targets. Plaintiffs challenge inputs. Training data scraped without authorization, as in the NYT v. OpenAI litigation. Or they challenge outputs. Hallucinated facts, defamatory text, infringing images generated by deployed systems. These are not wrong targets. But they miss the most consequential phase of the AI lifecycle.

Between raw data and model output lies a process most legal scholars have not examined. Self-supervised learning (SSL) is the computational mechanism by which modern foundation models transform internet-scale text and images into mathematical weights. It is where bias sediments, where private data is compressed into recoverable form, and where structural defects are cast into a model long before any fine-tuning or deployment decision can correct them. It is the foundry. What comes out of it is determined by what happens inside it. And right now, no liability framework reaches inside.

I argue that SSL creates a category of risk I call Representational Risk. These are defects that cannot be remediated by downstream actors because they are encoded at the representational level, in the learned geometry of the model itself. A distinct liability framework, targeting what I call the Base Model Provider, is required. The normative foundation for that framework already exists. The AI Data Stewardship principles developed at CodeX provide the appropriate baseline.

Why SSL Changes the Legal Calculus

Legal scholarship has begun to examine the latent space as a site of doctrinal interest. BJ Ard’s Copyright’s Latent Space: Generative AI and the Limits of Fair Use (110 Cornell L. Rev. 2025) argues that fair use doctrine should account for how generative AI models extract what Ard calls “non-authorial value,” the facts, tropes, and structural patterns that exist independently of any artist’s creative choices. The analysis is grounded in intellectual property, namely who owns what the latent space contains, and whether training on it constitutes infringement. This post occupies a different position. Where Ard examines ownership of value encoded in the latent space, I examine responsibility for harms cast there. The latent space, on this account, is not primarily a repository of extractable information. It is a foundry. The question is not who owns what it holds. It is who bears liability for the defects it produces.

Traditional supervised machine learning requires labeled data. A human annotator marks images as “cat” or “not cat,” and the model learns to replicate that judgment. The legal implications are relatively tractable. Annotation decisions are human choices that can be audited, and model behavior is constrained by the label set.

SSL discards labels. It trains models on self-generated prediction tasks. Masked language modeling teaches a model to predict a missing word from surrounding context. Contrastive learning teaches it to recognize that two augmented views of the same image are more similar than two random images. These objectives require no human curation of meaning. They require only data, enormous quantities of it, scraped from the open web.

This shift has two legal consequences that supervised learning does not generate.

First, SSL learns implicit structure from the training corpus. The model does not learn what humans have decided to call things. It learns the statistical relationships embedded in how humans actually write, what images they produce alongside what text, what appears near what. The resulting representations encode cultural assumptions, demographic patterns, and factual associations that no annotator ever reviewed or approved.

Second, SSL models memorize. Research by Carlini and colleagues has demonstrated that large language models trained with SSL will, under appropriate prompting, reproduce verbatim text from their training data, including private phone numbers, email addresses, and personal health information. The memorization is not incidental to an otherwise clean process. It is a feature of how SSL achieves generalization. The model must retain sufficient specificity about training examples to successfully predict their masked elements.

Third, and most consequentially for liability theory, SSL produces what ML researchers call a world model. A supervised model learns to replicate human judgments within a defined label set. An SSL model learns a functional representation of how the world works. It absorbs semantic relationships, causal associations, factual co-occurrences, and cultural patterns, all derived from data without any human having approved the resulting structure. The world model is not a lookup table. It is an internal map of reality, built from whatever the training corpus contained, that the model uses to reason across novel situations it has never encountered.

This distinction matters legally because it determines what fine-tuning can and cannot fix. A fine-tuner who adjusts a corrupted world model is not correcting the map. It is changing where the navigation starts. The underlying representation of the territory remains wrong. Biases, factual errors, and poisoned associations encoded at the world model level persist through fine-tuning in ways that annotation-level errors in supervised systems do not.

The legal system has no doctrine that maps cleanly onto any of these three consequences. That gap is the problem this post addresses.

The SSL Risk Taxonomy

A. The Stewardship of Unlabeled Data

The typical SSL pipeline begins with Common Crawl, a freely available scrape of approximately four billion web pages, collected without quality or content filtering beyond technical deduplication. GPT-3 was trained substantially on Common Crawl. So were BERT and most of their successors.

Common Crawl contains everything the web contains. Medical misinformation, demographic stereotypes, extremist content, and factual errors accumulated over years of archiving are all present. When an SSL model trains on this corpus, these patterns are not incidentally absorbed. They are structurally encoded into the model’s representation of language. A base model trained on unvetted Common Crawl data does not merely reflect the web’s biases. It builds a world model from them. The spatial relationships that govern what the model treats as similar, relevant, or probable are derived directly from the statistical structure of whatever the corpus contained. A world model built from Common Crawl is a map of reality as the unfiltered internet represents it. That is not a starting point a fine-tuner can correct by adjusting a few layers of weights.

The applicable legal theory is negligent entrustment. A developer who uses an unvetted, unfiltered corpus for SSL training has entrusted a computational process with data that a reasonable actor would recognize as generating predictable harm. Establishing a duty of care requires foreseeability of downstream use. A developer training a base model on Common Crawl in 2024 knows the model will be used in medical, legal, and financial contexts. The harm from biased representations in those contexts is not speculative.

A skeptic will argue that Common Crawl is the only corpus large enough to achieve state-of-the-art SSL performance, making its use an industry standard rather than a negligent choice. The argument has surface appeal. But it misidentifies where the negligence lies. The negligence is not in using Common Crawl. It is in ingesting it without applying the AI-DSF’s Foundational Controls. Data Provenance Protections, Data Threat Defenses, and Continuous Data Vulnerability Management exist precisely because large unfiltered corpora are the operational reality of SSL development. A developer who uses Common Crawl with documented provenance controls, anomaly detection, and pre-ingestion quality review is not negligent. A developer who uses it without those controls, knowing what the corpus contains, is. The distinction is between the data source and the discipline applied to it.

B. Memorization and the Right to Be Forgotten

The memorization problem sits at the intersection of SSL mechanics and privacy law.

GDPR Article 17 grants data subjects the right to erasure. CCPA provides a parallel right for California residents. Neither statute was drafted with neural network weights in mind. Both were drafted with databases in mind, records that can be located, identified, and deleted.

Whether a latent representation of personal data constitutes “personal data” within the meaning of these statutes is unresolved. The Article 29 Working Party’s guidance suggests that data is “personal” if an individual can be identified from it, directly or indirectly. If Carlini-style extraction attacks can recover verbatim PII from SSL model weights, the argument that the weights contain personal data in the statutory sense is serious. The weights are not merely derived information in the way that an anonymized aggregate is derived information. They are, under specific conditions, a reproducible copy.

Regulators should treat recoverable memorization as per se statutory retention. If PII survives in extractable form within model weights, the right to erasure applies. The base model provider has an obligation either to demonstrate that memorized content is not extractable or to retrain without the relevant data. Neither obligation is cost-free. That is the point.

C. Algorithmic Poisoning and Product Liability

Backdoor attacks on SSL training sets are documented and reproducible. An adversary who can inject a small number of poisoned examples into a training corpus, achievable through contributions to Common Crawl or to widely-used open-source datasets, can install hidden triggers in the resulting model’s latent space. When the trigger pattern appears at inference time, the model behaves in a manner the deployer did not intend and cannot readily detect.

The product liability framing is relatively straightforward. Under the Restatement (Third) of Torts, a product contains a manufacturing defect when it deviates from its intended design in a way that renders it unreasonably dangerous. A backdoored SSL model deviates from its intended design. The defect is latent. It is not detectable through standard evaluation. And it can produce serious harm in deployed systems.

The harder question is who bears liability when the poisoning occurs at the data level, before the developer has assumed possession of the affected training examples. Supply chain product liability provides precedent. A manufacturer who incorporates a defective component bears liability even if the defect originated upstream. The SSL base model provider, having chosen to train on an unvetted corpus without adversarial robustness evaluation, has made a decision that determines whether the defect reaches the downstream product.

D. Causal Hallucinations and the World Model as Design Defect

The three risks above involve language models. The world model problem extends further, and its physical consequences are more severe.

SSL is no longer limited to text. Systems such as Sora and JEPA learn world models from video. They infer not just semantic relationships but physical ones: how objects move, how forces propagate, how materials deform under stress. These are causal world models. They represent, in latent space, the developer’s implicit claim about how physical reality behaves.

When a causal world model is used to train a robotic agent or an autonomous system, the legal stakes shift from bias and privacy to physical injury. A robot trained on an SSL world model that misrepresents the brittleness of glass, the stopping distance of a vehicle, or the load tolerance of a structural component is not operating on a statistical error. It is operating on a false physics. That is a design defect under the Restatement (Third) of Torts, section 2(b), which holds a product defective in design when the foreseeable risks of harm could have been reduced by a reasonable alternative design.

The reasonable alternative is a verified world model. A developer who conducts Latent Space Audits on physical representations before licensing a world model for use in robotic or autonomous systems can test whether the model’s causal structure deviates from reality in ways that produce predictable harm. A developer who does not conduct those audits and licenses the model anyway has made a design choice. Product liability reaches that choice.

This extends Representational Risk beyond the informational domain. A world model is not merely a representation of language. It is a representation of causality. When causality is wrong and the error is actionable, the foundry metaphor takes on a different weight. What is cast in the foundry is not just a biased language map. It is, in some systems, a defective model of physical reality that will govern how machines act in the world.

Leveraging the AI Data Stewardship Framework

The AI Data Stewardship Framework (AI-DSF) provides the normative architecture for translating these liability theories into actionable obligations. The AI-DSF organizes its controls into three tiers. Basic Controls represent the baseline every organization should have. Foundational Controls apply to organizations with higher risk profiles. Organizational Controls focus on people and processes. Several of these controls map directly onto the SSL risk landscape.

Continuous Data Vulnerability Management is a Basic Control requiring that pre-training and post-training procedures be “executed, documented, and measured” and that “proven anomaly detection tools are continuously used.” Applied to SSL, this control operationalizes the Latent Space Audit. Probing classifiers can test whether a model’s representations encode demographic associations that no annotator reviewed or approved. Extraction-based evaluation can estimate memorization risk before a checkpoint is released. These are not speculative obligations. They describe procedures that the AI-DSF already requires and that SSL developers can implement today.

To make the Latent Space Audit legally operational, I propose that the AI-DSF require what I call a Representation-to-Risk certification: a documented record, produced before any downstream license is granted, that specifies which categories of representational bias were probed, which PII memorization tests were conducted, and what remediation was applied to identified risks. The certification functions as a Safety Data Sheet for the latent space. It gives downstream licensees a verified record of what they are inheriting, and it gives regulators and courts a documented standard against which to measure whether the Base Model Provider exercised reasonable care. A provider who cannot produce one has not satisfied the Continuous Data Vulnerability Management obligation.

The AI-DSF further requires that “data deletion and data unlearning methodologies are readily available and implementable.” This is a direct reference to Machine Unlearning, a technical approach to removing specific training examples from a model’s learned behavior without full retraining. For the GDPR and CCPA memorization problem, Machine Unlearning is the AI-DSF’s prescribed remediation mechanism. A Base Model Provider that has not implemented Machine Unlearning capabilities before encountering a right-to-erasure request has failed a Basic Control obligation.

Data Provenance Protections is a Foundational Control that “implements safeguards against data poisoning.” This is the control that maps directly onto the backdoor attack risk. The AI-DSF requires that developers implement guardrails against “unlicensed, unverified, and unintended data sets” and maintain documented data provenance throughout the supply chain. A developer who trains on an unvetted Common Crawl corpus without provenance controls has not merely made a poor engineering choice. It has failed a Foundational Control that the AI-DSF identifies as necessary for organizations with elevated risk profiles. SSL developers, who are the Base Model Providers for the entire industry, are precisely that.

Data Threat Defenses, also a Foundational Control, explicitly references NIST AI 100-2e2025, the Adversarial Machine Learning taxonomy. This control requires that developers identify and mitigate threats from internal and external sources and maintain alignment with adversarial robustness standards. The SSL poisoning scenario is not an edge case the AI-DSF failed to anticipate. It is a named threat category within the framework’s own standard reference.

Data Inventory, a Basic Control, governs how dataset diversity and sufficiency are established and monitored, and requires that data licensing requirements are reviewed and complied with prior to dataset ingestion. This is the control that addresses the Common Crawl problem at its source. A developer who ingests Common Crawl without reviewing its licensing status and content quality against a documented standard has bypassed a Basic Control before the SSL process has even begun.

The downstream relationship is addressed through the AI-DSF’s supply chain standard. All supply chain members are subject to a meet-or-exceed requirement that corresponds to the organization’s own policies. A Base Model Provider that documents its AI-DSF compliance and makes that documentation available to fine-tuners and deployers enables those downstream actors to verify what they are inheriting. Without that documentation, fine-tuners and deployers are operating blind. The AI-DSF treats this information asymmetry as a control failure, not merely a commercial inconvenience.

Finally, the AI-DSF explicitly identifies the FTC’s power of algorithmic disgorgement as a consequence of stewardship failure. Algorithmic disgorgement means the destruction of the model. Not a fine. Not an injunction against future conduct. The deletion of the asset itself, along with every downstream product built on it. A developer who has trained an SSL base model on unlicensed or tainted data and has not followed the AI-DSF’s controls has built its entire model investment on a foundation the FTC can legally dissolve. The SSL Safe Harbor proposed below is not merely a litigation defense. It is the only mechanism available to a Base Model Provider for protecting a billion-dollar research and development investment from a regulatory delete order. A company that has not implemented the AI-DSF’s controls before the FTC begins its inquiry has no safe position from which to argue.

Proposed Liability Model: The SSL Safe Harbor

I propose a tiered liability framework organized around the distinction between Structural Defects and Instructional Defects.

Structural Defects originate in the SSL phase itself. They include biases encoded in the learned representations, memorized private data recoverable from the weights, and backdoor triggers installed through corpus poisoning. They are structural because they are encoded in the world model, the foundry’s core output. A fine-tuner inherits that world model. It can adjust behavior at the margins. It cannot rebuild the map. These defects therefore persist through fine-tuning and cannot be remediated by downstream actors without access to and control over the base model. Liability for Structural Defects falls appropriately on the Base Model Provider.

Instructional Defects are introduced through fine-tuning, prompt design, or deployment decisions. A model fine-tuned to generate harmful content, deployed in a context for which its representational properties are unsuitable, or prompted in ways that elicit harmful outputs falls into this category. Liability for Instructional Defects falls appropriately on the Fine-Tuner or Deployer.

The Stewardship Defense is the incentive mechanism that makes this framework function. A Base Model Provider who has implemented the AI-DSF’s controls during the SSL phase receives a rebuttable presumption of non-negligence as to Structural Defects. The operative obligations are specific. On the Basic Controls side, the provider must maintain documented Data Inventory practices with pre-ingestion licensing review, Continuous Data Vulnerability Management with anomaly detection and Machine Unlearning capabilities, and a Data Incident Response Plan covering poisoning and memorization events. On the Foundational Controls side, it must implement Data Provenance Protections with documented safeguards against poisoning, Data Threat Defenses aligned with the NIST adversarial machine learning taxonomy, and Audit and Control findings reported to senior management. Finally, at the Organizational level, it must implement a formal Data Stewardship Program with board-level oversight, and conduct Fuzzing Tests and Red Team exercises against its pre-training pipeline.

A provider who has satisfied these controls has done what the AI-DSF requires. The presumption of non-negligence follows. A plaintiff who can demonstrate that the provider knew of a specific risk, that the relevant control was designed to address that risk, and that the provider failed to implement or maintain that control can overcome the presumption. But the burden shifts. And that shift is precisely the incentive the SSL ecosystem currently lacks.

This structure mirrors the EU AI Act’s conformity assessment mechanism and the NIST AI RMF’s risk tiering, without requiring comprehensive regulatory adoption. Courts can develop the framework through common law without waiting for legislation. The AI-DSF already exists as a documented standard. Its controls are specific and auditable. The doctrinal infrastructure for a negligence per se argument, or at minimum a strong res ipsa inference, is available to courts willing to engage with it.

Closing the Pipeline Gap

Output-only AI regulation will fail, and it will fail predictably. A framework that holds deployers liable for what models say, without addressing what models are, treats the symptom while leaving the pathology unexamined. Every harmful output emerges from a representational substrate that was formed in the SSL phase, before any deployer or fine-tuner made a single decision. Holding only the deployer liable is like holding the driver of a car with defective brakes liable while exempting the manufacturer who built the braking system.

The SSL phase is where the model is made. It is where foundational decisions about data, representation, and structure determine everything that follows. A liability framework that reaches this phase is not merely more complete. It is more accurate about where the decisions actually occur and who actually makes them.

The global AI oversight conversation has focused on output monitoring, transparency requirements for deployers, and consumer-facing disclosure. These are not wrong. They are insufficient. The AI Data Stewardship Framework provides the tools to extend that conversation upstream, to the moment when raw data becomes latent representation and the structural properties of AI systems are cast.

The foundry cannot be exempt from inspection simply because the casting happens before anyone is watching.

The discussion covers practical subscription pricing strategies (do-it-yourself/done-with-you/done-for-you tiers), the future role of human lawyers in an AI-powered legal landscape, and why the profession must adapt as consumers already demonstrate willingness to pay $20-$200/month for AI-assisted services. 

Kerbis emphasizes that lawyers’ future competitive advantages will be taste, curation, relationship-building, and human judgment—capabilities that don’t fit the billable hour model but align perfectly with subscription-based services.

Watch CodeX Group Meeting in Youtube

Transcript

Roland Vogl:
Welcome everyone to our CodeX group meeting. We have a great session today with Mathew Kerbis, who is the co-founder and CEO of Practi, which offers an alternative to hourly billing. Welcome Mathew. I’ll turn it over to you.

Mathew Kerbis:
Thank you to Stanford and the CodeX community, and for everybody for having me. I last spoke to this community over two years—wait, three years ago. It was January of ’23. I was almost a year into having my own subscription-based law firm, and that was getting some press. But since then, I have won a couple of awards for the practice and for my podcast Law Subscribed, where I interview other lawyers or innovators in the space. I’ve had Megan Ma on the podcast, and really a lot of things came to a head over me teaching lawyers how to use AI and adopt alternative business models for their law firm, where it ended up being—it was too much to run a practice, have a podcast, do one-on-one coaching, and all this teaching.

Actually, a software play kind of became—later became necessary to go from one-to-many and try to let other law firms and lawyers adopt the subscription model and alternative fee business structure for their practice. And just the way AI has been accelerating everything—you know, there’s a lot of pressure coming down on law firms from clients, both sophisticated and unsophisticated. You know, Roland, you said it’s very timely. I’ve been beating this drum for over four years, and just finally AI has gotten good enough that it’s really highlighting it.

With that, I am going to hop into just a quick presentation to set the table about what Practi is, talk about myself and my co-founder just for a quick second, and then show off the product. But at any point in time, if folks have questions—and I see my co-founder Shomari Ewing is here too—if anyone has any questions, yep, interrupt me, ask questions, disagree with me. It’s okay. I’ve heard it all. I’m happy to have lively debate if it comes to that.

With that, though, I will share my screen here just to show off the quick deck. All right. Okay, just moving some things around on the screen here for me. Okay.

We’re Practi, and we transform law firm revenue from hourly to subscription. Maybe you’ve asked yourself this. Maybe a client has asked this of you: “Should my lawyer be using AI to save time and reduce my legal bill?” Well, what Practi is, is the business model solution for this looming revenue crisis for billable hour law firms. That’s because—this community has probably already seen this graphic—lawyers have 855, probably more because this is like a month or two old—now this chart: legal AI tools available to them from almost 700 different companies.

Right. Last year alone, $6 billion was invested into the legal tech ecosystem, and a lot of that was AI-related. And the data show that 80% of lawyers or more are using AI, whether it’s these tools or the foundational tools. They know that it’s reducing their time, the time it takes to get high-quality work done. But what Practi is—we are betting on a future where it doesn’t matter if one of those legal AI solutions wins, or if the foundational models win, you know, with Claude and their legal plugins. Right. We don’t care. We know there will be winners, though.

And we are focused on building for what comes next. According to Clio’s data that they’ve taken—their private data and applied it to the public data from O*NET—they found that AI can already automate 75% of a law firm’s billable hours. It’s a combination of lawyers and their staff, but that’s revenue that’s gone.

Practi—since we’re trying to drive systemic change in the profession, and I’ve had my own practice and I have legal tech companies trying to sell to me all the time—we’re trying to make it really easy and really frictionless for lawyers to at least try this out and to get a sense of what the ROI is. That’s why we let law firms sign up for free, create their account, build out subscriptions for free, and get on calls with me and other users on the platform to strategize good subscription billing techniques and strategies.

And then after that, while we’re in this early access phase, we’re just charging $20 a month, right? We’re trying to keep it super affordable, super reasonable in these early stages. Later this year it will go up to $100 a month, but even then we only charge after the second client subscribes. We want to help law firms make more money before we even start charging anything.

And I will say here, just to note at the bottom, yeah, our target market—we’re trying to help the solo small firm attorneys. And we believe that with the Harveys and the Casetext of the world starting to go into enterprise markets, we believe part of the reason they’re doing that is because Big Law won’t need as many lawyers to continue to be Big Law and generate the revenues that they’re generating in light of these AI efficiencies. We’re going to be there to help Big Law attorneys that either get fired or decide to get off the billable hour hamster wheel soon. And that’s another reason for our pricing, because we’re really targeting those smaller law firms in the future—smaller and solo law firms from attorneys who are leaving bigger law in light of these AI efficiencies.

Rght now this is per firm because, again, we’re targeting solo small firms. Realistically, for firms on our platform to make it—the way that the product is designed, they may only have a handful of attorneys. We have interest from one mid-sized firm, but the way that we might have to set it up to work for multiple attorneys would be—it would ultimately mean they need multiple accounts. While it’s not meant to be set up as per-seat pricing right now, this is just per-firm pricing with the expectation that most firms will be solos or smaller, very much on the smaller side. Yeah, because we also predict that that’s the future of actually what the vast majority of the legal—of the law firm delivery model will be, will be these solo and small firms in light of AI efficiencies and the ability to do more work at higher quality and faster.

We’ve been building since last fall, and my firm, Subscription Attorney LLC, has been using it exclusively to sign up and charge clients since November. It’s fun to be a guinea pig—dog food—at your legal tech startup. But we soft-launched just about a month and a half ago, we’ve got some early traction. We’re not spending any money on marketing yet. We’re still having that direct communication with early users. But we’ve seen some good usage of the platform for just being out of stealth mode for just a month and a half.

For those of you who don’t know me, I go by The Subscription Attorney. I adopted that brand about four-plus years ago, and I have a podcast. Like I mentioned, I create a lot of content around this. I do a lot of teaching for free, basically, and guest lecturing. I also do—I’ve done paid workshops and trainings for bar associations and law firms. I am the subject matter expert in the space, but not just because of me and my stuff, but I have guests on my podcast. I’ve had over 100 lawyers who are leveraging subscription in some form or another.

Then my technical co-founder, who again is here, Shomari Ewing, he’s got over a decade of software development experience. He’s worked at companies like Google and Amazon Web Services, and he also has an MBA and a sales background. That’s always nice to have a technical person who actually understands business.

Again, we’re Practi. We transform law firm revenue from hourly to subscription. And with that, I’ll just show you the product, and I’ll be happy to entertain any questions.

Oh yeah, when I’m sharing my screen, it’s hard for me to see the comments here. Let me look into the chat. “Have you considered creating an insurance-based subscription scheme that aligns the ability to help people reduce their future prospective problems or not being compliant with unlicensed practice of law while also providing leads for retrospective problems?”

You’re asking—I mean, we are building a software solution that’s essentially Shopify for SMB law firms. But you’re asking about an insurance product. You know, I mean, look, if anyone’s out there and they watch this on YouTube later and you’re a professional liability insurance carrier and you want to create a particular insurance product for something like this, I mean, we’re happy to have a conversation. I think existing professional liability covers the type of work that’s being done, especially when you have jurisdictions like my jurisdiction in Illinois that are changing their Rule 1.5 on fees to expressly allow alternative fees.

Also, it was—I believe it was last year—the Illinois Supreme Court promulgated Rule 302 of the rules of professional conduct, which say if you’re not billing by the hour and you’re using a fixed fee, you could recover that without tracking your time. I think we’re going to see a lot more jurisdictions allow for express adoption of alternative fees, even though if you probably haven’t read your rules on fees lately—if you are an attorney, a majority of them allow for fixed fees. And the way that I encourage lawyers to use subscription is it’s a recurring fixed fee, whether that be on a monthly basis, quarterly, or annually. Right. But it’s a recurring fixed fee.

All right, let me share the product. Yeah, I’m curious to see what—I got a demo now. But one of the members of this community who works for a Big Law firm running AI and machine learning for them, she described a scenario where she said, “Okay, her firm is helping companies in a cyber breach situation, right?” They know the playbook—you know, what do you do? You have to go to all the states and inform the authorities and on, right? And it can quickly generate—cost a client like $1 million in legal fees, right, if you have a big cyber breach. Right.

And they thought, “Well, what if instead of that, we have a product for our clients where our clients pay us like, say, $100,000 a year as a subscription fee?” They will pay that even if there’s no cyber breach. But if there is a cyber attack, then the firm will handle everything, all the legal stuff that’s entailed, right, even if it costs them more than $100,000. I think that’s kind of going in that direction of creating an insurance-like type—turning a legal service into like an insurance type of—I don’t know if there are many practice areas that are accessible to that, but this cyber breach thing is like one, for example.

I think that’s a fantastic example. And I think that, you know, it is legally separate and distinct from actual insurance, right. But having a lawyer that you subscribe to that’s in the problem-avoiding space, that’s in the fire prevention space, right, rather than in the putting-out-fires phase, or you only contact them when there’s a problem, right—because if that client is paying that law firm 100Kayearminimumannualsubscription,right,yeah,somemembershipbenefitsofthat100K a year minimum annual subscription, right, yeah, some membership benefits of that 100Kayearminimumannualsubscription,right,yeah,somemembershipbenefitsofthat100K a year better include “we contact our lawyer to avoid ending up with a cybersecurity incident in the first place,” I would hope, without being on the clock, because maybe it could be avoided.

And then when you get to the costs part of that conversation, when you bill by the hour, it’s hard to not think of things in terms of cost accounting, with “Oh, that took ten hours. My hourly rate is $1,000 an hour, you do the multiple,” right. Like, no—that’s what you would make. That’s more opportunity cost than it is what your actual costs are, what your actual costs are, because you could—not necessarily fudge the numbers, but depending on how you present the numbers of cost accounting—Ron Baker does a great presentation on this. I highly encourage you to seek that out. He did it on my podcast last year.

But it’s, “What does it cost to run your business, to pay your people, to pay to subscribe to your software?” Right. It’s not about how long something takes being your cost. It’s about what does it actually cost to pay your people and have your software subscriptions. And then if you’re tracking time, it’s not about how long it took—”that cost us that much.” It’s about, “Okay, that took ten hours. How do we make it take nine hours, five hours, ten minutes? Automate it.” Right. And if we could continue to charge and command that price because that’s valuable for our client, then it doesn’t matter how long it takes because we could try to automate it.

Getting to the product—Practi—how is the website? You know, it’s live. Lawyers could sign up. Again, it’s free to set up your account. You know, we’re just getting the newsletter off the ground. If you want to sign up, you can, or you can even book a call with me and Shomari if you want to follow up. That’s all on the website.

When you make an account—I’ve just got a demo account set up here—you will be asked to either sign up for or connect your Stripe account. Right now we’re built on top of Stripe. Stripe’s secure. It’s a leader in the space. It’s one of the reasons why we’re building on Stripe first. We have been asked, “Will we integrate with other payment partners?” And, you know, that’s a maybe. We are still very early days in the first few months of operating as a company, we want to stick with the best of the best for now.

And after you connect your Stripe account, this will be a blank page, but you could set this up before setting up a Stripe account. But we have some templates. The templates will get better over time, right. Practi is not selling AI to law firms because there’s enough of that already. We are using AI, right. We’re an AI-native startup. Shomari is using, you know, Cursor and Claude and, you know, to build and ship features faster than we could pre-generative AI being useful for that. I’m using a boatload of AI tools. NotebookLM, Perplexity Pro are my daily drivers, right. What you saw earlier was Gamma, right. I’m using Gemini. I’m using a ton of AI. I think I’m at 27 right now. Whisper Flow—if you haven’t tried Whisper Flow yet, highly, highly recommend.

But in any event, we’re going to be collecting data of what subscriptions are working, and eventually we’ll have some sort of AI recommendations on what to build. Right now they’re very generic, but some people find that it’s easier to just add a template and then edit the template, right. I just added Homeowner Basic. You’ll see here, Homeowner Basic is here. And if I wanted to, now I could come in and I could edit this instead of just starting from scratch.

But for law firms that are already leveraging subscription, or they already have some ideas, they could just add custom subscriptions. The point here is that we’re trying to be as flexible as possible for law firms to offer whatever sort of subscription packages they want without giving you much freedom that it’s paralyzing, right. We just let you name it, put in a price. If you want to have a tagline or a description, or you just want to talk about the member benefits, right—I’m showing you the demo on the back end. I will show some front-end live pages of actual law firms, including my own, that are actually charging—what it looks like.

But for this demo page, whenever you make a change in the back end, it automatically pushes it out to the front end. It’s actually kind of zoomed in a lot on my end. You’ll see there’s also standalone services down there. We’re not trying to replace law firm websites, by the way. Lawyers spend a lot of money on websites, and I’ll get to what we’re offering for those websites here.

But going back to those lawyers who I know are going to be leaving Big Law or leaving other firms and starting their own practice—they’ll have a URL and a way to make money. And as long as they’re registering with their states, setting up their law firm—and we have a lot of content forthcoming about how to actually set up your law firm, right—they’re getting their malpractice insurance, right. We don’t help with the delivery or the business stuff yet, but you’re getting a URL. You’re getting a way to actually start charging clients on a recurring basis.

Mathew, can I ask you a question? Yeah. Okay, you have some templates that you have thought through before, like, you know, what would be, for example, a homeowner’s type legal subscription—what would be in that, right. And but presumably every firm’s a little different in the sense of the services they provide and understanding, like, because, you know, there’s always some things that are just repeat business, right, that they get all the time—standardized stuff—and then some stuff that’s more bespoke, right.

With your customers, these law firms, do you go through some kind of discovery in the sense of, “Oh, in your firm, you might think about these parts of what you do as things that are even amenable to creating a subscription around, and here are the things that you should continue handling in a billable hour way”? Is that part of what you offer?

We’re more a software company right now than we are a consulting company, right. Like, as we grow, we’re going to be hosting weekly 30-minute Zoom meetings with me as a group where we can talk those things through, because the chances are if one person’s asking those questions, somebody else already is. But thanks to the power of AI, we actually have—I’ve curated, it’s right now over 250 sources, a lot of it from my podcast, right, and CLE presentation materials that I’ve given—where you could come in to this link. And for folks who are watching live here on Zoom, I’ll put it in the chat.

You do have to be signed into a Google account to access it because it saves your chat history, which you could delete if you want. You could ask it these questions, right. This is our hack right now to be like, “Hey, yeah, I’m only available once a week on these 30-minute Zooms with all the users.” Well, you could ask this NotebookLM curated database that I’ve put together as the subject matter expert about—hey, I’m in immigration, right. We can even—I’ll just give an example. You know, I’m an immigration attorney. Oh, actually I’ll use Whisper Flow. For those of you who don’t know what Whisper Flow is, you basically just talk to your computer.

“I’m an immigration attorney and I’m not sure how I should price my subscription and/or fixed-fee packages. What do you recommend?”

And this is—no. And if you haven’t come across NotebookLM yet, it’s by far the best tool for $14 a month. This version that I’m using to do a forward-facing sharing version of it—for NotebookLM, you have to do it from a Gmail account because Workspace and Enterprise and school accounts don’t let you share outside your organization. It’s $20 a month to have up to 300 sources and higher usage limits.

Here you go. Immigration attorneys often use different pricing models like that. And then it’s pulling—it’s telling you where it’s pulling from for the actual sources, right. And that’s really important. Now we are playing with whether or not we want to let people see the sources, but it’s going to be able to tell you where that information is coming from. And it’s not just like a made-up thing, right. It’s not like just a chatbot making it up.

And actually, I think I want to share—yeah, I know I’m doing this live, but I’m going to share more than just this note—NotebookLM for this audience here. Hang on a second. I’m moving it from chat-only to full notebook. Bear with me here because we did just build this out by request. And some of these—here, I’ll refresh this now. I just sort of opened this up a little bit. “Answer again with examples, please.”

What’s great about NotebookLM is you could create resources on the side here. Yeah. Well, it’s a good thing NotebookLM is in my product because it seems to be having some issues right now, but that’s our stopgap measure. You know, Roland, in the future we’ll have more—I’ll either be more available or we’ll be able to productize those features a little bit more.

But basically, you’re creating a platform where these different subscriptions can be offered through—all of them, however you want to structure it. That’s the idea, right? Because like you said, there are different practice areas, there are different sizes, there are different ideal clients that law firms are serving. We believe we’ve built a flexible enough platform that you can showcase whatever sort of subscriptions you want, right.

Here’s Next Step Legal. This is a live firm that you could go to right now and you could check them out, right. And you could see—I’m signed in. In a second here, I’ll sign out. I just want to keep showing the back end a little bit because once you’re signed in as a law firm, you can’t actually subscribe to other law firms. You have to be not signed in or signed in as a client. Before I sign out, I just want to show some folks—here you put your engagement terms in here.

And again, I’m not giving legal advice, but just suggesting as a best practice, it’s always good to have. And we do allow $0-a-month subscriptions. I don’t recommend it. I recommend at least $1 a month for fixed-fee shops, or like—oh, because we’ve had some people say, “Hey, I mostly just do fixed fee.” Well, you could still engage your client. They’ll agree to your subscription terms even if it’s for $0. They’ve agreed to your engagement terms even if it’s for $0 or $1 a month. But then you could always go and follow up with them to have customized engagement terms that amend that, right. But this way they’re going to be able to see what your engagement terms are when they sign up, right.

You copy and paste those in there. And by the way, mine are available for free at subscriptionattorney.com/#engagement. I put them out there for my clients, but I also put it out there for other lawyers that want to adapt it to their jurisdiction.

Then there are the fixed-fee services, right. We don’t have templates for this yet, but that may be coming in the future. But for those fixed-fee, one-off transactional things that you want to charge to clients that aren’t included, this is where you could build out those services as well. You can track the revenue that you’re getting through your firm. This is a demo account, I’m not showing you a live firm account. We don’t have that fixed-fee revenue showing in the demo account. Shomari, that’s something to figure out, maybe for the next presentation we give.

But we let you track the revenue. We let you see the clients and what they’re subscribed at. We let you manage the clients. The clients can sign in, and I’ll show you what the client-side dashboard looks like in a second. Or, you know, we’re almost at time, you know, we have a widget button that you could use on a website where you could come to the website and you could click “Sign Up.” And I just want to be respectful of people’s time.

And then this is the same checkout page if they were to click on your Practi page, right. Like, if I’ve spent a lot of money on my website, we still let you connect it to your actual site. And if you’ve ever checked out with Stripe before, that’s going to look very familiar to you. And then, yeah, we let firms link back to their website and put in a scheduling link for their Calendly or something like that, right.

Yeah, I, again, I want to be respectful of everyone’s time. You know, there are basic settings you could add here. But again, we’re not trying to replace anyone’s—you know, the rest of your tech stack. We’re just trying to be a super simple, easy-to-use subscription revenue management platform.

With that, Shomari, thanks for answering people in the chat here. Yeah, sure. I think, thank you much. It seems like you tried to answer most of the questions that people in the group asked. Yeah. Do you have anything to add before—Mathew, do you have anything to add to what I’ve said?

Yeah, I’m going to need to catch up on it. Yeah, same here. Wow. But there was, you know, just kind of—yeah. In the same way that I’ve curated that NotebookLM—that RAG-based chatbot—for users of Practi to query against the database of curated sources that I’ve provided, you know, we’re giving that away for free for now, right. I mean, we’re still sort of testing it. You all got an early sneak peek at what that looks like.

But that’s, I think, the value of subject matter experts—the attorneys or any other subject matter expert in this AI-automated, supercharged, powerful world. It’s taste. Like, people have taste, and they maybe like working with somebody. Like, it’s relationship. It’s taste and it’s curation. I think those are the future superpowers of lawyers and professional service providers. And it doesn’t make sense to charge by the hour for those things because that doesn’t take a long time. It might take years to acquire taste and years to acquire an understanding to be a good curator. But once you have that expertise and that taste that people are willing to pay for, I think subscription makes the most sense. And it’s made sense before AI.

But we know—I just want to end with this, and then I’ll take any—people can unmute and ask questions. We know that the time is ripe for subscription-based legal services because today, already today, if somebody has an AI subscription that they’re paying for, they are asking it for legal advice. And it’s maybe giving them legal information in exchange and giving a disclaimer that says “contact a lawyer.”

My firm has gotten inbound from those types of communications, right, that clients are having with their AI tool before they contact you. But we already know that between $20 and $200 a month is a price point people are willing to pay. They are asking these AI tools for legal advice. I think that means the profession has to be ready to find a way to adapt the subscription model to their practice. With that, I’ll open it up for questions from the audience.

Roland Vogl:
Thank you, Mathew. Yeah. I’d like to ask the good question, which I think you answered already, but I think the question is, you know, what happens to the law firm pyramid if AI compresses junior associate-level work into AI systems? Do we need to train them differently or have a different revenue model entirely?

I don’t know how you’re thinking about that, right. Like, I agree with your premise that, you know, there will be a move away from the billable hour, which we have seen already—alternative fee arrangements, right, flat-fee billing and on, where the firm is encouraged to be more productive, you know, with its human resources. But there is also, you know, the question of, like, you know, what is the role of the human lawyer going forward, right? Like, AI can handle more and more legal tasks and, you know, do more and more reliably. But there is still this human oversight layer that’s needed, right. Maybe the subscription at the end will be for that human oversight, right regardless of how it’s paid, it is for that human oversight and context awareness. I don’t know if you have any thoughts on that.

Mathew Kerbis:
Yeah. There’s a three-tiered subscription way to price anything, right. And subscription, especially in the services space—in the software space, there’s good, better, best. And that can work with law firms. But I think what works with law firms even better for simple three-tiered pricing—though we certainly see more nuanced approaches on Practi, and I’ve interviewed lawyers who have more nuanced approaches to this—but a good starting point for some law firms is the do-it-yourself, done-with-you, done-for-you three-tiered pricing structure, right.

And you charge more—and for the done-for-you, top-tier pricing structure, you don’t just charge a simple multiple. That’s going to be many multiples more than the do-it-yourself and done-with-you service, right. And sometimes that might even be faster service for most clients, not certain contexts. And I’ve done both. The context where slower is better is foreclosure defense, sometimes insurance defense, right. If they get results faster, that’s actually more valuable to them.

There’s a UK expert in the space, Sean Jardine, who talks about how giving three-tiered pricing to your client for fixed-fee pricing—more on the fixed-fee pricing side of things—and he’s like, if they need it and you have a month to work on it, it’s the lowest price. If you want to get it to them by next week, that’s a mid-tier price. If they need it in 48 hours and they’re willing to pay that super high multiple, you give them that choice, right.

And in his coaching of lawyers, he’s on record saying how clients pick the more expensive option to get things faster. And sometimes faster is more valuable, and clients are willing to pay more for it than what you’d make on the billable hours. I still think there’s that side of it.

I think the learning—going back to that part of your question—is, how do lawyers learn if they’re not junior lawyers? Because that junior work can be automated by AI. Well, how do lawyers learn now? They learn because they do some work, they send it to the partner, and the partner marks it up and gives it back to them. And rinse and repeat, and rinse and repeat, and rinse and repeat.

You’re going to be doing that with these AI tools. But instead of it taking hours and days and weeks and months to learn, we’re going to compress that time. I’m using three AI tools—well, four if you include Whisper Flow—but three that are actually helping me with my substantive legal work daily. And that’s Perplexity Pro, NotebookLM Pro, and Paxton AI. I don’t know if you guys have had Paxton on—they’re a legal AI company. They’ve raised some money. They have their own legal large language model in addition to being multi-model.

And I’m using these three tools as though I have three team members. And sometimes I have multiple tabs open for multiple different files, and I’m just going back and forth with these AI tools. And I’m learning things, maybe about a particular nuance for a particular contract that maybe I didn’t know before, even though I’ve got over a decade of legal experience. But I think you’re going to see this learning compacted as new lawyers learn to use these legal-specific AI tools.

Benjamin
And what I like to tell people is that the calculator didn’t get rid of accountants, right. And it may be the case that we have like a legal tax calculator, but that’s only part of the profession. Part of it is knowing sort of the series of events that’s going to follow. There is no explicit law that says, you know, “don’t put soap on the floor.” But we all know if you have soap on the floor and someone is walking by, they may very well slip on that soap, right. And the number of ways that you can foresee—or negligence being unforeseen—is almost limitless. And you can’t just explicitly put it into an AI such as this.

And I think that with regards to the representation, a lot of the representation has to do with being able to foresee the results of your actions—the consequences of your actions—and other people not being happy with it. And I feel like that human factor is going to be needed, and it’s just going to take the role of the legal tax calculator, right, where we all want to know what the rules of the game are. We should all sort of agree, but that doesn’t tell me how I’m going to interact with the world and how I’m going to resolve conflicts with people in that world.

Mathew Kerbis:
I don’t think the role of the lawyer’s going anywhere. I just think it’s going to be different, right. I mean, if you think about, you know, pre-motor vehicle, it was people’s jobs to shovel horse manure out of the roads, right. I don’t think anyone’s complaining that that work has gone away. It used to be—somebody, before electricity, it was someone’s job to replace oil in lanterns at establishments, right. No one’s complaining that that job has gone away, right.

I think that the writing things from scratch that we do as lawyers—and we don’t always write from scratch; we write from templates or similar docs, or, you know, we find other ways to do this. Maybe we have automations built out in a tool like HotDocs or something, right. But we’re still doing a lot of manual work. And I think all that gets automated, and we’re billing time for it, and clients are paying it, but they’d rather not.

Paige:
This reminds me of and sort of touches on what you were just saying about Benjamin’s question as well. It’s felt like the billable hour—we know it has been going the way of the dinosaur for a while, and I feel like this new age we’re in is just really—it feels like, you know, what rideshare did to the taxi system, where it just fundamentally undermined the basis of the whole system in a way that it had to change largely.

I’m curious what you see going—where do you see the future of the billable hour and these alternative structures? Like, what do you see the new standard being? For some context there, like, I’ve worked in—I’m not an attorney, I’m a paralegal—and I’ve worked in many areas. I’ve worked in areas of law that are very much billable-hours dependent. I’ve worked in other areas where it’s only flat-fee models. Nobody’s doing hourly. Where do you see kind of the new standard for fee structure and the billable hour being in ten years?

Mathew Keribs:
Yeah. I’ve been having a lot of conversations on my podcast about this lately, and I tend to record live on LinkedIn. And people are giving the estimate—not me, although I agree with the estimate—3 to 5 years, and the billable hour would no longer be the dominant business model in 3 to 5 years, right. And again, that’s just sort of like experts pulling their opinion out of, you know, where. But like, our intuitive sense is 3 to 5 years before it’s not the dominant business model.

The reason it’s survived for long, even though people have said it’s going to go away, is partially because it’s been profitable despite there being better ways. But that profit number is going to start going down. To make up for it, rates will go up, right? The Wall Street Journal article—$3,400 an hour is out there now. Billable rates out there. But eventually, when push comes to shove, in my class I actually give math examples of, like, if you really have a 90% time savings, you’re going to have a 50X rate—$25,000 an hour from $500 an hour—which is probably an unreasonable fee, and certainly no client’s going to want to pay it, right.

You have certain economic forces at play here. Finally, thanks to AI efficiencies being exponential, I think that the reason flat fee has not supplanted billable hours is because of the underscoping/overscoping problem. And I get this question all the time from lawyers who are like, “Well, I’m not sure what to do. I’m in litigation or I’m in complicated M&A transactions, and like, how am I supposed to handle it if things start taking a lot longer or if it gets more complicated?”

That’s what subscription tiers are for, right. And your engagement agreement could say, “Okay, it’s an uncontested divorce. It’s $2,000 a month. If it goes to contested, it goes up to $5,000 or $10,000 a month,” right. That’s what subscription tiers are for. And if it’s even less complicated than we thought it was, we bump you down to tier one, right. You go down a subscription level. Subscriptions solve the underscoping/overscoping problem. At least it solves it as close, I think, as we can, right.

Yeah. And then you’re incentivized to just use the tech, use the tools, and adopt the technology. Like, we’re not selling AI, but we are friends with every legal AI company. Why? Because we want the lawyers to use the tech, because we think—I think ethically, just my opinion, not ethics advice—Rule 1.5, Comment 5, I believe, for the Model Rules, Comment 5 says you can’t bill and use wasteful procedures if you’re billing by the hour. That’s the implication. If you can use AI to make a ten-hour task take ten minutes, you have to do it. It’s already in our ethics. It’s just not being enforced right now. Yeah, I think it’s just a matter of time for that.

Sorry, Roland. Go ahead. I know we’re over here—ten minutes over time.

Roland Vogl:
It’s such a fascinating discussion. Not just your business which I think is very interesting and really showing a path forward for legal service delivery, but everything else you taught us about thinking about how to measure the value that lawyers provide and scoping and all of that. I think there was a lot of interesting stuff there.

I really appreciate you coming to join the group today and giving us an update and telling us about Practi. Please keep us posted in the future. And yeah, everyone else, thank you for all the great questions and the good conversation. And yeah, I look forward to seeing you all next time.

The ERCA is the competition agencies of the Economic Community of West African States. It covers fifteen Member States across West Africa (Benin, Cabo Verde, Côte d’Ivoire, The Gambia, Ghana, Guinea, Guinea-Bissau, Liberia, Nigeria, Senegal, Sierra Leone, and Togo) and operates from Gambia. It handles mergers, anticompetitive practices, and is now developing a regional framework on consumer protection. In short, it is building competition enforcement at continental scale. That is no small task.

The Stanford Computational Antitrust project now brings together over 70 competition agencies from around the world. The ambition is straightforward, that is, help agencies understand and use computational tools in competition analysis and enforcement. The network grows because the need is real. Digital markets do not wait for institutions to catch up.

The ERCA will contribute to our research reports and feed into the collective body of knowledge we are building. We will connect them with peer agencies in the network, some facing similar institutional contexts, others offering tools and experience ERCA can build on. We look forward to workshops and training sessions together. More than that, we look forward to learning from them. West Africa’s telecommunications, digital platforms, and cross-border trade markets raise competition questions that computational tools are well-suited to address.

The project is stronger for this addition. Welcome.

Aziz Fatnassi est étudiant dans le cadre du cours DRT6929 (Vie privée + Numérique) (Hiver 2026) 

Après cinq longues années de contentieux opposant WhatsApp Ireland Ltd (ci-après « WhatsApp ») au Comité européen de la protection des données (CEPD), à l’origine d’une amende de 225 millions d’euros infligée à la filiale de Meta, la Cour de justice de l’Union européenne (CJUE) a jugé, le 10 février 2026, que l’entreprise est fondée à introduire un recours devant les juridictions de l’Union contre la décision du CEPD, quand bien même celle-ci serait adressée aux autorités nationales chargées de la protection des données, dès lors qu’elle affecte directement ses activités, et a renvoyé, en conséquence, le dossier devant le Tribunal de première instance des Communautés européennes (TPICE) pour statuer sur le fond de l’affaire.

Histoire des origines, origine de l’histoire

 Peu après l’entrée en vigueur du Règlement européen de protection des données (RGPD), et pour donner suite à plusieurs plaintes concernant le traitement de données à caractère personnel opéré par l’application de messagerie WhatsApp, le régulateur irlandais chargé de la protection des données pour la plupart des géants technologiques américains (leurs sièges européens étant situés en Irlande) a ouvert le 10 décembre 2018 une enquête à caractère général sur le respect par l’entreprise des obligations de transparence et d’information à l’égard des particuliers en vertu des articles 12, 13 et 14 du RGPD.

Comme le veut la procédure, dès la clôture de l’enquête menée par l’autorité irlandaise, celle-ci a présenté le 24 décembre 2020 à ses homologues des différents États membres concernés par le traitement des données personnelles de WhatsApp un projet de décision qui a soulevé plusieurs objections, notamment de la part des autorités allemande, française, hongroise, néerlandaise et portugaise concernant, entre autres, les atteintes identifiées et le caractère approprié des mesures correctives envisagées. S’ensuivit une phase d’échanges de commentaires entre les autorités concernées, au terme de laquelle aucun consensus ne se dégagea.

L’autorité irlandaise n’ayant retenu aucune des objections, saisit en conséquence le CEPD conformément à la procédure instaurée par le RGPD, qui procède le 23 avril 2021 à l’audition de WhatsApp et rend le 28 juillet 2021 une décision contraignante à l’égard de l’ensemble des autorités concernées dans laquelle il constate des violations du RGPD et ordonne à l’autorité irlandaise de modifier notamment le montant du projet d’amende, qui s’élève désormais à 225 millions d’euros.

Ce n’est que le 2 septembre 2021 que l’autorité irlandaise, se conformant à la décision contraignante du CEPD, infligea à WhatsApp une amende de 225 millions d’euros, l’une des amendes les plus élevées jamais prononcées au titre du RGPD, considérant que la filiale de Meta avait failli à ses obligations de transparence à la fois envers ses utilisateurs et les non-utilisateurs dont le numéro de téléphone avait fait l’objet d’un traitement.

Insatisfait, WhatsApp a introduit un recours en annulation de la décision du CEPD devant le Tribunal de première instance des Communautés européennes qui, par ordonnance du 7 décembre 2022, l’a déclaré irrecevable au motif qu’elle ne constitue pas un acte attaquable et que WhatsApp n’était pas directement concernée, estimant qu’elle ne revêtait qu’un caractère intermédiaire et que seule la décision finale de l’autorité irlandaise pouvait être contestée devant le juge national, seul habilité à saisir la CJUE à titre préjudiciel. En d’autres termes, seule la décision finale de l’autorité irlandaise pouvait faire l’objet d’un recours.

WhatsApp s’est donc pourvu devant la CJUE, qui a tranché le 10 février 2026 en faveur de la recevabilité en jugeant que

“Saisie d’un pourvoi introduit par WhatsApp Ireland Ltd (ci-après « WhatsApp »), la Cour, siégeant en grande chambre, annule l’ordonnance du Tribunal dans l’affaire WhatsApp Ireland/Comité européen de la protection des données…”.

Il en découle que les décisions du CEPD prises sur le fondement de l’article 65 du RGPD constituaient bien des actes attaquables.

Notion d’acte attaquable au sens de l’article 263, premier alinéa, du TFUE 

Dans son arrêt du 10 février 2026, la haute juridiction européenne vient préciser la notion d’acte attaquable, en rappelant le rôle du juge de l’Union dans le contrôle de la légalité des actes des organismes de l’Union destinés à produire des effets juridiques à l’égard des tiers. À cet effet, le caractère attaquable d’un acte s’apprécie objectivement, par référence à son contenu indépendamment de l’identité du requérant. Il suffit donc que l’acte produise des effets juridiques à l’égard de tiers, entendus comme toute personne distincte de son auteur, à savoir le CEPD.

En l’espèce, la Cour a considéré que la décision litigieuse constitue un acte émanant d’un organe de l’Union, revêtant un caractère contraignant à l’égard des tiers en ce qu’elle s’impose tant à l’autorité irlandaise appelée à l’adopter qu’aux autorités de contrôle concernées.

“Ainsi, la Cour constate que la décision litigieuse constitue un acte attaquable et que le Tribunal a commis une erreur de droit, d’une part, en opérant une confusion entre les exigences résultant, respectivement, des premier et quatrième alinéas de l’article 263 TFUE et, d’autre part, en formulant un critère erroné, relatif à l’absence d’opposabilité directe de l’acte en cause à l’égard de WhatsApp, et en qualifiant la décision litigieuse de mesure intermédiaire dénuée d’effets juridiques autonomes.”

Sur le bien-fondé du recours de WhatsApp contre la décision du CEPD devant le Tribunal de première instance des Communautés européennes

Pour qu’il soit recevable, la Cour rappelle qu’un recours en annulation introduit par une personne non-destinataire d’un acte est subordonné à la condition que cet acte la concerne directement et individuellement au sens des dispositions de l’article 263, alinéa 4, du TFUE.

La haute juridiction ajoute que l’affectation directe suppose la réunion de deux conditions, la mesure contestée doit, d’une part, produire directement des effets sur la situation juridique du requérant et, d’autre part, ne laisser aucun pouvoir d’appréciation aux autorités.

S’agissant de la première condition 

“En l’espèce, la Cour constate que, le Comité ayant décidé notamment que WhatsApp avait méconnu certaines dispositions du RGPD, la décision litigieuse modifie la situation juridique de cette entreprise, celle-ci étant en particulier amenée, en raison de l’intervention du Comité, à modifier sa relation contractuelle avec les utilisateurs du service de messagerie fourni par elle. Partant, il existe un lien direct entre cette décision et ses effets sur la situation de WhatsApp.”

 S’agissant de la deuxième condition

“Dans ce contexte, la Cour rappelle que la décision litigieuse lie l’autorité de contrôle chef de file et les autorités de contrôle concernées, celles-ci ne pouvant pas s’écarter de la position retenue par le Comité dans cette décision. En effet, ladite décision tranche les questions de droit faisant l’objet de la saisine de celui-ci et lie inconditionnellement ces autorités, s’agissant en particulier du constat de violation de certaines dispositions du RGPD, de la qualification des données soumises à une compression avec perte en tant que données à caractère personnel ainsi que de l’obligation de rehausser le montant des amendes envisagées. Lesdites autorités n’ont pas la possibilité de modifier le résultat des appréciations portées, pour ce qui est de ces questions, par le Comité. ”

La Cour en déduit que WhatsApp est directement concerné par la décision litigieuse.

“Par conséquent, la Cour conclut que WhatsApp est directement concernée par la décision litigieuse. ” 

Portée de l’arrêt 

En jugeant que les entreprises peuvent désormais être considérées comme directement concernées par les décisions du CEPD même si celles-ci ne sont pas formellement adressées à elles, reconnaissant qu’elles peuvent introduire des recours en annulation contre ces décisions sans avoir à saisir préalablement les juridictions nationales, la haute juridiction européenne consacre ainsi un précédent judiciaire ouvrant la voie à un contrôle juridictionnel des actes du Comité, ce qui pourrait encourager davantage de recours, en particulier de la part des grands acteurs du numérique.

L’évolution du contentieux en précisera les contours.

Affaire à suivre !

Aziz Fatnassi est étudiant dans le cadre du cours DRT6929 (Vie privée + Numérique) (Hiver 2026)

Introduction : la Californie, le laboratoire étasunien en matière de protection des données personnelles

Au sud de la baie de San Francisco, sur les comtés de Santa Clara et de San Mateo en Californie, s’étend la Silicon Valley, épicentre mondial de l’innovation technologique. Sur ce territoire relativement restreint d’environ 200 km², soit deux fois la surface de la ville de Paris, les grandes firmes industrielles historiques comme Hewlett-Packard, Intel et Apple côtoient les géants de l’économie numérique, au premier rang desquels figurent Google, Meta, Yahoo, LinkedIn, Twitter, PayPal et Netflix.

Berceau de la haute technologie et de l’innovation mondiale, la Californie ne cesse pourtant de prendre les devants en matière de protection de données personnelles depuis le scandale Facebook/Cambridge Analytica, un choix qui se traduit par l’entrée en vigueur au 1ᵉʳ janvier 2020 du California Consumer Privacy Act, une version étatsunienne du Règlement européen de protection des données (RGPD).    Première du genre aux États-Unis, elle impose aux géants du Net, dont le modèle économique repose sur la collecte et l’exploitation commerciale des données personnelles de leurs utilisateurs, des obligations de transparence quant aux types de données qu’ils collectent, ainsi que l’obligation de permettre aux personnes concernées de s’opposer à une telle exploitation.

Dans la droite lignée de son engagement en faveur d’un renforcement du contrôle des consommateurs californiens sur leurs données personnelles, le législateur a entendu leur conférer le droit de supprimer leurs données collectées par les plateformes numériques, tout en en rationalisant le processus de mise en œuvre. Ce choix s’est concrétisé avec l’adoption en 2023 du Senate Bill SB-362, un projet de loi connu sous le nom de Delete Act, en vertu duquel la California Privacy Protection Agency (ci-après CPPA), l’Agence californienne de protection de la vie privée, devrait mettre en place, au plus tard le 1ᵉʳ janvier 2026, un moyen permettant aux consommateurs de demander l’effacement de leurs données personnelles, en une seule demande.

C’est dans le prolongement de cette démarche qu’a été créée la « Delete Request and Opt-out Platform » (ci-après DROP), une plateforme gratuite grâce à laquelle les consommateurs peuvent désormais, dans le cadre d’une seule procédure, exiger la suppression de leurs données personnelles auprès de plus de 500 courtiers en données.

Sur les courtiers en données personnelles souffle le vent du Delete Act 

Les controverses récentes liées au recours par certaines entreprises aux algorithmes de tarification dynamique pour ajuster les prix, tout comme les pratiques d’achat et d’exploitation de données personnelles des utilisateurs de certaines applications, montrent l’ampleur prise par le marché de courtage de données et rappellent combien leur protection constitue désormais un enjeu central.

Aux termes de l’article 1798.99.80 du Delete Act, qui reprend à l’identique la définition du California Civil Code, est un courtier en données « une entreprise qui collecte et vend sciemment à des tiers les informations personnelles d’un consommateur avec lequel l’entreprise n’a pas de relation directe ». Il en découle que la notion de courtiers en données renvoie à une entreprise qui commercialise à des tiers des informations personnelles collectées auprès d’autres entreprises, concernant des personnes avec lesquelles elle n’entretient aucun lien direct.

Le California Civil Code prévoyait déjà l’obligation, pour les courtiers en données, de s’inscrire auprès de la CPPA, selon les modalités prévues à l’article 1798.99.82. L’apport du Delete Act tient donc moins à l’enregistrement en tant que tel qu’à l’obligation faite aux courtiers en données de répondre aux demandes de suppression toutes les 45 jours, telle que prévue à l’article 1798,99,86 (C)(1), sous peine de sanctions. Pour veiller au respect des nouvelles règles en place, le paragraphe (e)(1) du même article introduit un mécanisme de surveillance assurée par des tiers, dont la mise en œuvre interviendra en 2028 et sera renouvelée tous les trois ans.

Fonctionnement de la « Delete Request and Opt-out Platform » 

La plateforme DROP permet au consommateur d’adresser, par une seule soumission, une demande de suppression de ses données personnelles à l’ensemble des courtiers enregistrés, ainsi que, le cas échéant, une demande d’opposition à leur vente ou à leur partage futur. Les courtiers sont ensuite tenus de consulter régulièrement la plateforme et de traiter les demandes dans les délais fixés par la loi.

Avant de formuler sa demande, le consommateur doit préalablement s’inscrire à la plateforme. À cette fin, il lui appartient d’attester de sa résidence en Californie en communiquant certaines informations d’identification, notamment son nom, sa date de naissance et son adresse. Ces données sont ensuite transmises, sous forme hachée, aux courtiers en données concernés afin de permettre le traitement de la demande.

Le consommateur peut, le cas échéant, compléter son profil en ajoutant d’autres informations complémentaires susceptibles de faciliter l’identification de ses données, telles qu’un numéro de téléphone, avant d’adresser sa demande soit à l’ensemble des courtiers inscrits, soit à une sélection d’entre eux, et d’en suivre l’évolution depuis son espace personnel.

En offrant aux consommateurs un « guichet unique » qui leur permet de centraliser l’exercice de leur droit d’exiger la suppression de leurs données personnelles auprès d’une pluralité de courtiers en données souvent difficilement identifiables, la plateforme DROP répond à un enjeu d’accessibilité et de rationalisation des procédures.

Suppression, désindexation, effacement, oubli, une perspective de droit comparé 

Au Québec, le droit à la suppression des renseignements personnels est consacré à l’article 40 du Code civil du Québec, au visa duquel « Toute personne peut faire corriger, dans un dossier qui la concerne, des renseignements inexacts, incomplets ou équivoques ; elle peut aussi faire supprimer un renseignement périmé ou non justifié par l’objet du dossier, ou formuler par écrit des commentaires et les verser au dossier… » et le droit à la désindexation à l’article 28.1 de la Loi sur la protection des renseignements personnels dans le secteur privé (LPRPSP), dont le premier paragraphe dispose que « La personne concernée par un renseignement personnel peut exiger d’une personne qui exploite une entreprise qu’elle cesse la diffusion de ce renseignement ou que soit désindexé tout hyperlien rattaché à son nom permettant d’accéder à ce renseignement par un moyen technologique, lorsque la diffusion de ce renseignement contrevient à la loi ou à une ordonnance judiciaire ». Ces deux droits ne recouvrent toutefois pas le même objet, la suppression visant le renseignement lui-même, tandis que la désindexation n’affecte que sa visibilité en ligne.

Outre-Atlantique, le premier paragraphe de l’article 17 du RGPD au titre duquel la personne concernée est en droit « d’obtenir du responsable du traitement l’effacement, dans les meilleurs délais, de données à caractère personnel la concernant et le responsable du traitement a l’obligation d’effacer ces données à caractère personnel dans les meilleurs délais ». Il en découle que la personne concernée peut exiger l’effacement de données à caractère personnel la concernant dans les cas prévus à l’article 17, notamment à la suite de l’exercice du droit d’opposition, lorsque les données ont fait l’objet d’un traitement illicite ou encore afin de respecter une obligation légale issue du droit de l’Union ou du droit de l’État membre auquel le responsable du traitement est soumis.

Ce droit à la suppression des données est toutefois souvent confondu avec le droit à l’oubli, alors que ces deux notions ne se situent pas au même rang juridique.

Évoqué pour la première fois en 1966 par Gérard Lyon-Caen dans un commentaire d’un jugement du Tribunal de grande instance de la Seine, le droit à l’oubli renvoyait alors à l’idée que certains faits, devenus anciens, ne devaient plus être rappelés en justice, une conception inspirée de la prescription de l’action publique, fondée sur la logique selon laquelle, passé un certain délai, il n’est plus nécessaire de rappeler en justice les crimes dont les effets ont disparu.

Le droit à l’oubli est en effet plus large et l’effacement n’en constitue qu’une modalité. En ce sens, l’oubli peut être assuré autrement que par la suppression des données, notamment par le droit au déréférencement, défini par la Cour de justice de l’Union européenne dans l’arrêt Google Spain comme le droit à ce que l’information « relative à sa personne ne soit plus […] liée à son nom par une liste de résultats affichée à la suite d’une recherche effectuée à partir de son nom […] », dans une logique assez proche du droit à la désindexation prévu au Québec.

Le droit à la désindexation pourrait, à première vue, être assimilé au droit à l’effacement. Il n’en est pourtant rien, puisqu’il ne vise pas à supprimer l’information du site source, qui reste accessible par d’autres critères de recherche, mais à empêcher qu’elle apparaisse dans les résultats du moteur de recherche lorsqu’une requête est effectuée au nom de la personne.

Toutefois, faute d’une mise en œuvre centralisée aussi bien au Québec qu’en Europe, l’exercice de ces droits par les personnes concernées demeure fragmenté.

Alea jacta est, une réforme audacieuse aux effets limités 

Malgré des avancées notables, la réponse californienne aux enjeux du marché des services de courtage de données n’est toutefois pas exempte de limites et gagnerait à être renforcée, ce que les développements suivants entendent mettre en évidence en s’attachant plus particulièrement au caractère largement discrétionnaire du dispositif et à sa portée territoriale restreinte.

Une effectivité laissée au bon vouloir des courtiers en données 

Le mécanisme repose sur l’enregistrement des courtiers à la plateforme DROP, condition nécessaire pour que les consommateurs puissent exercer leur droit à la suppression. Le Delete Act prévoit à cet effet une amende de 200 dollars par jour en cas de défaut d’enregistrement, en plus des coûts d’enquête engagés par la CPPA. Une fois enregistrés, les courtiers doivent en outre consulter la plateforme et traiter les demandes dans les délais légaux, sous peine d’une amende identique en cas de non-suppression.

Cependant, compte tenu du poids économique de l’industrie des services de courtage de données, ces sanctions nous semblent relativement limitées, de sorte que l’effectivité du dispositif semble dépendre, dans une large mesure, du bon vouloir des courtiers eux-mêmes.

Un champ d’application ratione loci limité 

Adopté dans le cadre d’une loi étatique, et non fédérale, le dispositif DROP ne s’impose qu’aux courtiers soumis au droit californien, ce qui restreint considérablement son champ d’application territorial alors même que le marché des services de courtage de données se déploie dans un environnement numérique transnational.

Dès lors, le dispositif californien issu du Delete Act ne constitue pas tant un aboutissement qu’une étape dans une réflexion encore en construction sur les conditions d’effectivité du droit à l’effacement. Reste à savoir si cette réflexion trouvera demain un cadre juridique à la mesure des mutations qu’elle révèle. Une telle réflexion semble devoir se déployer dans un contexte plus large, marqué par l’essor de l’intelligence artificielle générative. Elle est générative dans la mesure où elle permet de « générer », à partir de requêtes textuelles, des objets numériques tels que des textes, des images, des sons, des vidéos, ou encore des fichiers. Ces productions suscitent tout autant l’émerveillement que les débats.

Voilà une interrogation laissée à la doctrine du futur, sans trop savoir qui, entre l’humain, l’IA ou une hybridation des deux, la rédigera.

I review Solaiman’s proposal through two prisms. The first is the AI Life Cycle Core Principles (AILCCP). Now in its third year of development, the AILCCP is a comprehensive framework containing 37 principles organized across 10 pillars, supported by 48 controls, and mapped to 10 life cycle phases. It offers a structured methodology for building, deploying, and operating AI systems that are defensible, compliant, and aligned with organizational, regulatory, and societal expectations. Each AILCCP principle, pillar, and life cycle phase is a formally defined concept enriched with objectives, rationale, key questions, controls, evidence requirements, and life cycle guidance, hence their capitalization as defined terms. Second, the Procedural Self-Consumption (PSC) lens. This is a legislative review framework that identifies seven diagnostic patterns through which technology legislation produces process rather than governance outcomes.

I. Procedural Self-Consumption Analysis

The AI Act Through the PSC Lens

Solaiman critiques the AI Act for inadequate trust-building but does not systematically examine whether the Act’s operative provisions produce governance outcomes or merely generate process. Applying the PSC framework to the provisions that Solaiman discusses reveals patterns he identifies intuitively but does not name. The PSC framework contains seven diagnostic patterns. Four apply to the material Solaiman addresses.

 Pattern 1 (Procedural Self-Consumption): Article 95, one of only two articles in the AI Act that mention “trust,” creates an obligation to encourage development of voluntary codes. Not to adopt them. Not to enforce them. The provision generates further process (code development) rather than governance outcomes (binding standards). The AI Act builds trust by asking someone else to build trust later.

 Pattern 2 (Unanimity Without Convergence): Recital 27 invokes seven non-binding principles from the High-Level Expert Group on AI (AI HLEG) Guidelines, which form the basis of the Act’s trust conception. “Human agency and oversight” and “diversity, non-discrimination and fairness” are not operative standards. They are consensus placeholders. The principles achieve unanimity precisely because they are undefined. Solaiman notes that this basis is “not captured in the resulting law” but does not frame the observation as a diagnostic pattern, which limits its explanatory force.

 Pattern 6 (Procedural Perfectionism): The stacked conformity assessment regime Solaiman describes, where Medical Device Regulation (MDR) conformity is followed by AI Act conformity, each with its own procedural prerequisites, illustrates how individually defensible steps accumulate into disabling sequences. Each step is justified. The sequence delays governance to the point of irrelevance for a technology whose innovation cycle outpaces the regulatory pipeline.

 Pattern 7 (Meta-Regulatory Irony): The AI Act creates notified bodies whose fee-for-service incentive structure, as Solaiman documents, may align them with the industry they audit rather than the public they protect. The trust-building mechanism reproduces the trust deficit it was designed to remedy.

Solaiman’s Own Proposal Through the PSC Lens

Solaiman’s proposed “AI Bill of Rights for healthcare” is itself vulnerable to the patterns he diagnoses.

 Pattern 1 again: Solaiman proposes a voluntary code within the AI Act’s framework. A voluntary code creates no enforceable obligation. The test is straightforward: if every healthcare institution in the EU faithfully adopted Solaiman’s Bill of Rights, what would change? The existence of a document, not the existence of accountability. Patients would possess a charter. They would not possess a cause of action.

 Pattern 2 again: The Bill of Rights would enshrine values for trust, including consent, medical liability, data accuracy, privacy, bias, security, efficacy, safety, and transparency. Solaiman does not define what compliance with these values looks like for any of them. The very unanimity problem he diagnoses in the AI Act, principles endorsed without operative content, reappears in his proposed remedy.

 Timeline Projection: Solaiman’s proposal begins as a voluntary code. He envisions incorporation into national patient rights charters. In the EU’s legislative architecture, this means: (1) the AI Act encourages codes (already enacted); (2) someone drafts a healthcare-specific Bill of Rights (unspecified timeline); (3) EU member states choose whether to incorporate it into national frameworks (optional, no deadline); (4) national implementation varies by healthcare system (unbounded). The minimum elapsed time from enactment to first enforceable patient-facing obligation is functionally indefinite.

II. AILCCP Principle Mapping

Scoped Principle Set

Given the paper’s focus on healthcare AI, trust, the EU AI Act, and the doctor-patient relationship, the following AILCCP pillars and principles are contextually relevant:

Transparency & Explainability: Transparency, Explainability (XAI), Accessibility

Oversight & Accountability: Accountability, Governance, Metrics, Track Record

Reliability & Robustness: Accuracy, Trustworthy, Reliability

Fairness & Equity: Bias, Equity

Privacy & Consent: Consent, Privacy

Safety & Security: Safety, Security

Ethics: Ethics, Fundamental Rights

Human-Centered & Workforce: Human-Centered

Data & Process: Data Stewardship

Excluded from scope: R&D, Efficiency, Sustainable, Workforce Compatible, Wherewithal, Permit, Resilience, Robust. While organizational capability principles (Wherewithal, Sustainable) bear some relevance to healthcare institutions deploying AI, Solaiman’s paper does not engage institutional capacity, making their inclusion forced.

Principles Advanced

Transparency and Explainability (XAI): Solaiman’s discussion of informed consent and the black box problem in Section 4.2 (pp. 331-332) engages these AILCCP principles directly. His call for “meaningful disclosure of information that experts can access” maps to the AILCCP requirement for audience-appropriate explanations. The treatment remains at the level of aspiration rather than specification. He acknowledges that explainability is “easier said than done” and that post hoc rationalizations may not illuminate inner workings, but he does not specify what “meaningful disclosure” operationally requires.

Consent: The paper’s treatment of informed consent as a trust-building mechanism in healthcare, drawing on Hall’s “predicated” and “supportive” stances toward trust (p. 321), engages a recursive relationship the AI Act does not address. Consent is both derived from and constitutive of trust in healthcare settings. The AI Act treats consent as a downstream disclosure obligation. Solaiman’s framing treats it as a structural precondition.

Privacy and Data Stewardship: Solaiman identifies the deidentification/reidentification problem and notes that patients do not trust governance systems to maintain confidentiality (p. 332). His reference to the European Health Data Space (EHDS) as a potential solution engages Data Stewardship but does not assess whether the EHDS itself operationalizes the principle or merely invokes it.

Bias: Solaiman’s discussion of training data bias affecting demographic accuracy engages this principle, though his treatment is cursory. He calls for “additional checks and verification” (p. 332) without specifying what form those checks would take, who would perform them, or what standards would govern their execution.

Accountability: The paper’s discussion of liability gaps, the withdrawn Artificial Intelligence Liability Directive (AILD), and the uncertainty of the revised Product Liability Directive (PLD) engages Accountability (p. 332). Solaiman’s proposal for a “human point of reference” and standing committees to examine AI incidents is more operationally concrete than the paper’s other suggestions, though it still lacks enforcement architecture.

Principles Engaged but Not Operationalized

Safety: Solaiman asserts in Section 4.2 that AI outputs should be “accurate and safe for the context in which they are applied” (p. 333). The AILCCP principle of Safety demands more than aspirational assertions. It requires specified testing protocols, defined performance thresholds, and continuous monitoring mechanisms. None appear here.

 Trustworthy: The entire article circles this principle without landing on it. The AILCCP principle requires that an AI system demonstrates, through verifiable evidence, that it warrants confidence. Solaiman’s critique of the AI Act is that it treats risk classification as a proxy for Trustworthiness. His proposed Bill of Rights substitutes a different set of aspirational values without specifying how Trustworthiness would be verified. The proxy changes. The absence of verification does not.

Principles Neglected

Metrics: The paper’s most consequential omission. Trust is not simply a relational concept. It can be measured, benchmarked, and tracked. The AILCCP principle of Metrics requires defined indicators for system performance, compliance, and impact assessment. Solaiman’s critique of the AI Act’s trust framework would gain substantial force if he specified what trust metrics in healthcare AI would look like. Patient satisfaction scores with AI-assisted diagnoses. Error rate comparisons between AI-augmented and unaugmented clinical decisions. Disclosure compliance rates. Algorithmic audit frequency. Without Metrics, the Bill of Rights becomes a statement of aspiration indistinguishable from the ethics guidelines Solaiman dismisses (citing Munn) as “useless.”

Track Record: The AILCCP principle requires evaluation of an AI system’s historical performance and the deploying organization’s history of responsible AI practices. Solaiman’s discussion of Conformité Européenne (CE) marking’s evidentiary weakness (safety evidence deferred to post-market) raises Track Record concerns implicitly but does not propose that a Bill of Rights require transparent performance histories accessible to patients or clinicians.

Governance (as an AILCCP principle): Solaiman proposes institutional mechanisms (standing committees, responsible persons) but does not articulate the broader oversight architecture. Who oversees the standing committees? What is their reporting obligation? How is their independence secured? The Governance principle requires institutional controls that are themselves subject to review. Solaiman’s proposal has no second-order oversight.

Life Cycle Phase Coverage

Solaiman’s analysis clusters around two AILCCP phases: Pre-Deployment Review (conformity assessment) and Deployment & Release (point-of-care rights). This is characteristic of a market-access regulatory framework. It leaves gaps in the phases where regulation, institutions, and patients intersect.

Data Preparation. Solaiman raises training data bias as a trust concern in Section 4.2 but does not connect it to data pipeline oversight. Bias originates in this phase. His Bill of Rights proposes to address it at the point of care. By then it is baked in.

Evaluation & Red Teaming. The AI Act contemplates deployer-side testing obligations, and Solaiman’s own ecosystem argument implies that evaluation cannot occur solely at the developer level. A healthcare institution deploying AI against a specific patient population has an independent obligation to test. The Bill of Rights does not address this.

Operations & Monitoring. Solaiman’s standing committee proposal implicitly touches this phase, but continuous monitoring of system performance is not addressed. Trust erodes if a deployed system degrades and nobody is watching.

Incident Response. Solaiman proposes a right to redress, which implies a triggering event, which implies a protocol. The Bill of Rights does not specify one.

Decommissioning & Archiving. Not addressed. What are the patient’s rights regarding decisions made by a healthcare AI system that has been retired?

The Socio-Technical Ecosystem Argument and Its Unfinished Work

Section 4.1 argues, drawing on Unver et al., that trust must derive from the operation of the entire socio-technical ecosystem within which AI exists as one component. Trust in AI as a “standalone device” is, on this account, conceptually incoherent because liability systems rest on the responsibility of the clinician or employer, not the tool. “Trust in AI” as a standalone object is a category error. The meaningful locus of trust is the clinician, provider, or institution that stands behind the tool in a web of legal and ethical obligations.

The Bill of Rights does not carry this insight forward. It reproduces the very reductionism the article criticizes.

The rights in Section 4.2 are framed as claims against “the AI” or its immediate use, not as relational claims allocated across institutional actors in the ecosystem. There is no parallel articulation of duties for providers (deployment, monitoring, override, decommissioning), developers (updating, post-market surveillance, data stewardship), or regulators (transparency of notified bodies, management of misaligned incentives). Solaiman’s argument establishes that trust is ecosystem-dependent, then proposes a remedy addressed to a single technology layer. The Bill of Rights silently inherits the AI Act’s system-centric orientation, the same orientation Section 4.1 identifies as structurally inadequate.

The socio-technical framing also strengthens the PSC critique. If trust is ecosystem-level, then the AI Act’s market-access conformity model, which evaluates a product at a single point in time before deployment, is even more inadequate than a product-level critique would suggest. A voluntary Bill of Rights layered on top of that model inherits its structural limitations. Section 4.1’s own logic undermines Section 4.2’s remedy.

Solaiman’s socio-technical ecosystem argument maps naturally onto a life cycle view of AI. Trust is not established once at market entry and preserved automatically. It must be maintained through continuous oversight, incident response when things go wrong, and managed transitions when systems are updated or retired. If those phases are ungoverned, trust formed at the point of deployment will erode.

This means an ecosystem-aligned Bill of Rights cannot simply declare patient-facing values. It must allocate specific duties to specific actors. Hospitals, clinicians, developers, vendors, auditors, and regulators each play a role in sustaining trust across the AI life cycle. The Bill of Rights should specify what each owes, through mechanisms like performance reporting, independent oversight, and patient contestation rights.

Section 4.1 points toward this recognition. Section 4.2 does not deliver on it. The prescriptive proposal retreats to a static list of values addressed to a single technology layer, abandoning the ecosystem logic that Section 4.1 established.

Standards Context

Solaiman references standards from the International Organization for Standardization and International Electrotechnical Commission (ISO/IEC) in passing (through the AI HLEG Guidelines) but does not engage specific ones. ISO/IEC 42001:2023 (AI management systems) bears directly on the oversight architecture his Bill of Rights would require. ISO/IEC 23894:2023 (AI risk management) bears on his risk-versus-trust argument. The absence of standards engagement weakens the paper’s prescriptive force.

III. Argumentative Assessment

Thesis Architecture

Solaiman’s thesis is that trust in healthcare AI “could be better fostered” through a Bill of Rights. The conditional mood is telling. The thesis hedges where it should assert. A stronger formulation: the AI Act structurally cannot produce trust in healthcare because trust is a domain-specific, relational phenomenon and the Act is a horizontal, technical regulation. The Bill of Rights is not a “could be” improvement. It is a necessary supplement if the regulatory regime is to achieve its stated aims.

The thesis is also overbroad. It promises to examine what trust means in healthcare, how the AI Act incorporates trust, whether its provisions enhance trust, and what should be done to bridge the gap. That is four distinct arguments in a seventeen-page paper. Section 2 (the concept of trust in healthcare) receives approximately two and a half pages (pp. 320-322). Section 3 (trust and the AI Act) spans approximately six pages (pp. 322-327). Section 4.1 (systemic considerations) receives three pages (pp. 328-331), developing the socio-technical ecosystem argument that the paper then abandons. Section 4.2 (values for trust), the prescriptive core, receives approximately two and a half pages (pp. 331-333) to articulate the entire Bill of Rights proposal. The descriptive groundwork in Section 3 may be necessary. But Section 4.2 is underdeveloped relative to the weight the paper asks it to bear. Two and a half pages to specify a new governance instrument is not enough, especially when the paper’s own Section 4.1 establishes that trust is ecosystem-dependent and therefore requires duties allocated across multiple institutional actors.

Counterargument Treatment

The paper’s most significant argumentative weakness is its failure to engage the strongest version of the opposing position. The position that risk regulation can produce trust indirectly deserves serious treatment. The strongest version of this counterargument: systematically reducing the probability of harm, even through technocratic mechanisms, creates conditions under which trust emerges through experience. People trust automobiles not because they read safety regulation. They trust automobiles because safety regulation reduced harm rates over decades. Solaiman’s implicit assumption that trust must be directly cultivated through rights-based mechanisms deserves interrogation. The paper does not provide it.

The Realist Gap

The paper does not examine what healthcare institutions would actually do with a voluntary Bill of Rights, and its own evidence makes this omission structurally damaging.

Solaiman cites Unver et al. for the proposition that trust requires “designating responsibilities for different staff, such as doctors and clinicians, and outlining how the safeguards govern workflows concerning diagnosis and treatment using AI tools” (p. 330). He then proposes a Bill of Rights that does not designate responsibilities for any specific staff, does not outline any workflow safeguards, and does not address the clinical settings in which the rights would be exercised.

Consider the following scenario. A mid-sized European hospital runs diagnostic imaging AI from one vendor, a clinical decision support system from another, and a patient monitoring platform from a third. Each system has different explainability characteristics, different data pipelines, different update cycles, and different contractual terms. Solaiman’s Bill of Rights proposes that the patient receive “meaningful disclosure of information that experts can access.” Which expert? The radiologist who uses the imaging AI, the information technology (IT) administrator who manages the platform, or the procurement officer who negotiated the contract? The Bill of Rights does not say. A “standing committee within the hospital setting convened to examine AI incidents” must be resourced, staffed, and given authority. Who funds it? From which budget line? What is its jurisdiction when the AI vendor’s terms of service disclaim liability for the output the committee is examining?

These questions are not rhetorical embellishments. They are the operational conditions under which the Bill of Rights would succeed or fail. Solaiman’s Section 4.1 establishes that trust is ecosystem-dependent. His Section 4.2 proposes a remedy that ignores every institutional actor in the ecosystem except the patient.

The incentive structure is also unaddressed. EU hospitals operate under resource constraints, national regulatory variation, and competitive pressure. A voluntary code imposes compliance costs (committee formation, disclosure infrastructure, staff training) with no corresponding enforcement benefit. Hospitals that adopt the Bill of Rights bear costs. Hospitals that ignore it face no consequence. In this environment, voluntary adoption is not a governance strategy. It is a selection mechanism for institutions already inclined toward compliance.

IV. Synthesis

The word “trust” appears twice in 245 pages of the AI Act’s articles, revealing that trust functions as a rhetorical frame for the regulation rather than an operative concept within it. The conflation of risk acceptability with trustworthiness, drawing on Laux, Wachter, and Mittelstadt, identifies a structural deficiency in the Act’s conceptual architecture. CE marking’s reliance on deferred evidence of safety and effectiveness compounds this deficiency in the healthcare domain.

The proposed remedy, however, reproduces the problem.

The Bill of Rights proposal reproduces the reductionism it diagnoses in the AI Act. It invokes values without operationalizing them. It endorses trust without specifying how to measure it. It proposes voluntary mechanisms when the argument’s logic demands binding ones. It frames rights as claims against the AI system rather than as relational obligations distributed across the ecosystem that Section 4.1 identifies as the actual locus of trust. And it treats AI as a product entering a market rather than a system traversing a life cycle.

From an AILCCP perspective, the paper’s most significant gap is Metrics. Without measurable indicators of trust, the Bill of Rights becomes precisely what Solaiman (citing Munn) accuses the AI HLEG Guidelines of being. From a PSC perspective, the proposal creates further process (charter development, voluntary adoption, national implementation) without a mechanism to ensure that process produces governance. The very trap the AI Act fell into.

The paper would benefit from engaging AILCCP’s life cycle model to recognize that trust is not established at a single regulatory moment (market access) but must be maintained across the entire AI life cycle, from Data Preparation through Decommissioning. It would also benefit from specifying concrete Metrics for healthcare trust, anchoring the Bill of Rights to ISO/IEC standards that provide implementable controls, and confronting the enforcement question honestly.

A voluntary Bill of Rights for AI in healthcare, absent enforcement mechanisms and measurable standards, is a document that trusts the healthcare system to do what it has historically resisted doing without compulsion. That is not a governance strategy. That is a hope.

Cet événement a pour objectif de stimuler les échanges interdisciplinaires en valorisant les points de tension comme sources d’enrichissement mutuel. Le format privilégie le dialogue structuré et interactif entre chercheurs, professeurs et doctorants soigneusement sélectionnés, permettant à chacun de comprendre comment le droit et l’éthique peuvent se nourrir mutuellement malgré leurs logiques distinctes. Cette journée réunit des participants choisis du milieu académique pour explorer les enjeux critiques qui façonnent la gouvernance responsable de l’intelligence artificielle.

Structure de la journée

La journée s’articulera autour de quatre panels d’une heure, entrecoupés d’une pause déjeuner:

• 9 h 30 – 10 h 30 : Audit des systèmes l’IA, animé par Sébastien Gambs
• 11 h 00 – 12 h 00 : L’éthique de l’IA : institutionnalisation et gouvernance, animé par Hazar Haidar

Pause lunch

• 13 h 00 – 14 h 00 : Pluralité normative et éthique juridique, animé par Vincent Gautrais
• 14 h 30 – 15 h 30 : Tensions éthiques et droit, animé par Emmanuelle Marceau

Mot de la fin

À l’heure où l’intelligence artificielle transforme les pratiques universitaires, les défis juridiques et éthiques se multiplient. Protéger la vie privée, lutter contre les biais et encadrer l’utilisation de ces outils deviennent des priorités.

LEAD Technologies Inc. V1.01

Rami Haddad est étudiant dans le cadre du cours DRT6929 (Vie privée + Numérique) (Hiver 2026)

FRANCE TRAVAIL a fait l’objet d’une sanction administrative de 5 millions (€) et d’une injonction par la Commission Nationale de l’Informatique et des Libertés (« CNIL ») en date du 22 janvier 2026 visant à assurer la mise en œuvre effective des mesures correctives découlant de son manquement à l’obligation d’assurer la sécurité du traitement des données personnelles qu’il détient en vertu du règlement qui encadre la protection des données personnels au sein de l’Union Européenne (« Règlement Général sur la Protection des Données » ou « RGPD »).  Pour des fins de précisions, la CNIL est l’autorité administrative indépendante créé par la loi en France pour veiller à la protection des données personnelles et jouit d’un pouvoir d’investigation, de contrôle et de sanctions en cas de violations aux dispositions des lois et règlements en matière de protection de la vie privée.

1.                          Contexte :

FRANCE TRAVAIL est un établissement public administratif sous la tutelle du Ministère du Travail en France dont ses fonctions sont définies par l’article L. 5214-3-1 du code du travail de la France. Ces fonctions incluent notamment, l’accompagnement des demandeurs dans leur demande d’emploi et la gestion de l’offre d’emploi dans le cadre du processus de recrutement et d’embauche. FRANCE TRAVAIL a également comme fonction d’assurer l’accompagnement adapté aux besoins de personnes qui ont été qualifié de travailleur handicapé et qui sont bénéficiaires de l’obligation d’emploi.  Elle collabore aussi étroitement avec CAP EMPLOI, un organisme de placements spécialisés, qui est structuré de façon autonome et indépendante de FRANCE TRAVAIL. CAP EMPLOI accompagne autour de 20 % des personnes qualifiées de travailleur handicapé inscrites auprès de FRANCE TRAVAIL. Pour éviter la segmentation de cet accompagnement, une offre de service unifié entre FRANCE TRAVAIL et CAP EMPLOI permet depuis 2018 que cet accompagnement soit fait au sein de FRANCE TRAVAIL peu importe que le conseiller référent soit un employé de l’une ou l’autre de ces deux entités. Les 2300 employés de CAP EMPLOI pouvaient donc accéder à distance au système d’information de FRANCE TRAVAIL pour accompagner ces demandeurs.

2.                          L’intrusion externe et l’exfiltration de données personnelles

En date du 29 février 2024, des activités qualifiées d’anormales ont été détecté sur le système de mesures de performance du système d’information de FRANCE TRAVAIL. Cependant, ce n’est que quelques jours plus tard (5 mars) que l’alerte a été observé par les employés de FRANCE TRAVAIL menant par la suite à une investigation interne. C’est à la suite de celle-ci que FRANCE TRAVAIL a constaté qu’une intrusion externe avait eu lieu par l’utilisation de techniques qualifiés d’« ingénierie sociale » par des acteurs externes et s’est étendu du 6 février au 5 mars 2024. L’investigation fait aussi constat que ce sont les comptes de conseillers de CAP EMPLOI qui ont été usurpé par ces techniques d’ingénierie sociale pour ensuite accéder à l’environnement informatique de FRANCE TRAVAIL. Les acteurs externes ont par la suite eu accès à plusieurs types de données personnelles incluant des données considérées comme « sensibles » tel que le numéro d’inscription au répertoire (NIR) (équivalent au Numéro d’Assurance Sociale (NAS) du Québec) et ont réussi à extirper plus de 25 giga octets (Go) de données personnelles concernant plus de 36 millions de personnes dans la base de données de FRANCE TRAVAIL. À la suite de ces constations, FRANCE TRAVAIL a informé la CNIL le 8 mars 2024 de cette exfiltration de données personnelles massive lors de l’intrusion externe par les acteurs externes.

3.            Motifs de la délibération de la formation restreinte n° SAN–2026-003 du 22 janvier 2026

Les mesures techniques et organisationnelles appropriées qui doivent être mise en œuvre par le responsable du traitement doivent tenir compte « […] de la nature, de la portée, du contexte et des finalités du traitement ainsi que des risques, dont le degré de probabilité et de gravité varie, pour les droits et libertés des personnes physiques […]» selon l’article 24 (1) du RGPD. L’article 32 (1) du RGPD impose l’obligation au responsable du traitement de garantir un niveau de sécurité adapté au risque selon des mesures techniques et organisationnelles que celui-ci doit mettre en place. L’article 32 (2) du RGPD nous informe sur les critères d’évaluation du niveau de sécurité approprié qui doit être mis en place par le responsable du traitement. L’enquête de la CNIL a révélé que FRANCE TRAVAIL a contrevenu à ses obligations en vertu de l’article 32 (1) du RGPD en tant que responsable du traitement sécuritaire des données personnelles qu’elle détient en vertu de l’article 4, alinéa 7 du RGPD. Bien que FRANCE TRAVAIL argumentât que CAP EMPLOI avait sa part de responsabilité dans la mise en application des règles de sécurité, la CNIL a déterminé que selon les faits, c’est FRANCE TRAVAIL qui avait la responsabilité principale de mettre en œuvre des mesures applicables pour assurer la sécurité de son système d’information et dont son accès était ouvert à CAP EMPLOI pour l’accompagnement des demandeurs qualifié de travailleur handicapé.

3.1.                Au regard de la sécurité du traitement des données et la gestion des habilitations et limitation d’accès

La commission restreinte de la CNIL a jugé que les mesures de sécurité organisationnelle mise en place par FRANCE TRAVAIL pour le traitement des données n’étaient pas suffisamment élevées compte tenu de la nature, la portée, le contexte et les finalités du traitement des données à caractère personnel (article 9 RGPD) traitées par FRANCE TRAVAIL, notamment le NIR ainsi que les données de santé requis par CAP EMPLOI en lien avec demandeurs qui sont qualifiés de travailleurs handicapé, malgré que ces données de santé n’ont pas fait l’objet de la violation. La CNIL fait donc état du volume et la nature sensible des données personnelles qui sont confiées et dont FRANCE TRAVAIL doit traiter lorsqu’il évalue les mesures techniques et organisationnelle qui sont mise en place pour leur traitement.

De plus, bien que les comptes des conseillers de CAP EMPLOI étaient paramétrés selon des profils d’habilitation et le principe du moindre privilège par FRANCE TRAVAIL, l’enquête de la CNIL a dévoilé que ces conseillers pouvaient quand même accéder aux données personnelles de toutes les personnes qui se retrouvaient dans la base de données de cette dernière et que cet accès n’était pas limité sur la base de la nécessité de connaitre l’information par les conseillers voulant y accéder.  À cet effet, la CNIL cite le principe de « défense en profondeur appliquée aux systèmes d’information » dans le Mémento sur le concept de la défense en profondeur appliqué aux SI rédigé par le bureau conseil de la Direction centrale de la sécurité des systèmes d’information (DCSSI) version 1.1 du 19 juillet 2004 et qui est repris par l’Agence nationale de la sécurité des systèmes d’information (« ANSSI »):

« La défense en profondeur consiste donc à opposer aux menaces des lignes défense coordonnées et indépendantes. Sur le plan des technologies cela peut signifier par exemple que la compromission d’un service réseau ne doit pas permettre d’obtenir les droits les plus élevé sur l’ensemble du système. Dans ce contexte, donner des droits d’administration à tous les utilisateurs d’un système est contraire à la défense en profondeur. En matière de protection de l’information cela peut aussi signifier que le chiffrement au niveau applicatif n’est en soi pas suffisant et qu’il pourrait être nécessaire de protéger également la couche IP. La défense en profondeur a donc pour conséquence de ne pas faire reposer la sécurité sur un élément mais sur un ensemble cohérent. Cela signifie donc qu’il ne doit en théorie pas exister de point sur lequel tout l’édifice repose […] »

FRANCE TRAVAIL a donc manqué à son obligation de l’article 32 du RGPD en ne s’assurant pas que les comptes des conseillers de CAP EMPLOI étaient adéquatement restreints pour éviter que les personnes n’ayant pas le besoin de connaitre l’information puissent y accéder de façon illicite. Ces mesures de sécurité auraient pu prévenir l’exfiltration des données personnelles à travers un accès illicite des comptes des conseillers CAP EMPLOI lors de l’intrusion par les acteurs externes.

3.2.      Au regard des mesures de sécurité : qualité du mot de passe, mécanisme de restrictions au compte et authentification à double facteur

Dans sa Délibération n° 2022-100 du 21 juillet 2022, la CNIL a émis des recommandations sur les mesures de sécurité appropriées pour les mots de passe et les restrictions d’accès au comptes, entres autres : (i) la robustesse du mot de passe; (ii) le seuil de tentatives de connexions infructueuses établi à 10 et dans un délai recommandé; (iii) la mise en place de «captcha», un mécanisme qui permet de bloquer les tentatives hostiles de connexions automatisées et intensives et; (iv) l’utilisation de l’authentification à double facteur ou le certificat électronique. Or, la CNIL note que le choix de mot de passe (minimum de 8 caractères dont un minimum de 3 caractères spéciaux) exigé par FRANCE TRAVAIL était assez robuste.  Cependant, le mécanisme de restriction au compte prévoyait un verrouillage du compte seulement après 50 tentatives de connexions infructueuses et aucun autre mécanisme de restriction n’était mise en place. La CNIL juge donc que la vulnérabilité du mécanisme de restriction mise en place et l’absence d’autres mécanismes de restrictions pour accéder au compte de FRANCE TRAVAIL et CAP EMPLOI ne respectaient pas les mesures de sécurité recommandées par la CNIL dans la Délibération n° 2022-100 du 21 juillet 2022 citée plus haut résultant à un manquement à l’obligation de sécurité même si cette vulnérabilité n’a pas été exploité par les acteurs externes lors de l’intrusion des systèmes d’information de FRANCE TRAVAIL.

3.3.      Au regard de la mise en place d’un processus efficace de journalisation

Dans sa Délibération n° 2021-122 du 14 octobre 2021, la CNIL recommande

« […] que les opérations de création, consultation, modification et suppression des données à caractère personnel et des informations contenues dans les traitements auxquels la journalisation est appliquée fassent l’objet d’un enregistrement comprenant l’auteur individuellement identifié, l’horodatage, la nature de l’opération réalisée ainsi que la référence des données concernées par l’opération[…] » et la mise en œuvre d’« […] un système de traitement et d’analyse des données collectées et de formaliser un processus permettant de générer des alertes et de les traiter en cas de suspicion de comportement anormal […] ».

De plus, l’ANSSI dans son guide intitulé « Recommandations de sécurité pour l’architecture d’un système de journalisation réitère l’importance de continuellement analyser les journaux d’évènements afin de repérer toute activité suspicieuse ou inhabituelle et de garder un archivage des journaux pour permettre de soulever des doutes après coup. La journalisation est donc une mesure technique vitale qui permet de détecter, d’analyser et de répondre aux incidents de sécurité. La CNIL fait valoir que FRANCE TRAVAIL avait un processus de journalisation en place (avec identifiant interne technique et horodatage des actions effectuées par les conseillers) lors de la survenance de l’intrusion externe mais que l’échec de la détection des activités suspicieuses et l’absence de déclenchement des alertes démontrent que le processus de journalisation mis en place par FRANCE TRAVAIL n’était pas adéquat face au risque auquel elle faisait face. De plus, la CNIL avait déjà dans le passé (délibération n° 2022-050 du 21 avril 2022) averti FRANCE TRAVAIL de l’importance de se doter d’un bon système de journalisation afin de prévenir les incidents de sécurité.

4. La sévérité des sanctions en fonctions de la nature de la violation en vertu du RGPD

Le 13 janvier 2026, la CNIL a imposé une sanction administrative de €42 millions aux entreprises française FREE et FREE MOBILE en raison d’un manquement à une obligation de l’article 5 du RGPD relié au consentement, précisément d’avoir conservé des données d’anciens clients au-delà de la période nécessaire. Dans notre cas ici, l’imposition de la sanction administrative de €5 millions à FRANCE TRAVAIL était reliée à un manquement à l’article 32 du RGPD en lien avec le traitement sécuritaire des données personnelles. Selon l’article 83 du RGPD, la sévérité des sanctions pécuniaires imposée varie selon la nature de la violation en vertu des dispositions du RGPD, où la sanction est plus sévère pour une violation de l’article 5 du RGPD relié au consentement (83 (4) RGDP – 4 % du chiffre d’affaires) versus une sanction moins sévère pour une violation de l’article 32 du RGPD en lien avec le traitement sécuritaire des données (83 (5) RGPD – 2 % du chiffres d’affaires). Pourtant, nous avons constaté que le piratage des systèmes informatiques, qui a sans doute été facilité par les différents manquements aux dispositions de la RGPD, a résulté dans les deux cas mentionnés plus haut à l’exfiltration massive de données (incluant des données sensibles) concernant plusieurs millions de personnes physiques. La sévérité de la sanction pécuniaire imposé par le RGPD n’est pas particulièrement corrélative avec le préjudice qui peut découler des manquements aux dispositions du RGPD. Le RGPD impose donc des sanctions plus sévères en fonction de la nature du manquement plutôt qu’au préjudice résultant du manquement invoqué.  

The rapid evolution of reproductive technologies has forced courts and legislatures to confront questions that once seemed purely theoretical. In vitro fertilization (IVF) has become routine medical practice, while emerging techniques such as in vitro gametogenesis (IVG)—an experimental method of generating eggs or sperm from ordinary somatic cells such as skin cells—suggest that future reproduction may involve the large-scale creation and management of embryos.[1] Although such developments remain scientifically and politically contingent, advances in stem-cell-derived gametes and genomic sequencing could expand the scale of embryo creation and selection, thereby intensifying the regulatory and legal stakes of reproductive governance. Even at a preclinical stage, IVG suggests that reproduction may increasingly unfold within laboratory-based and institutionally structured frameworks.[2]

Since Dobbs v. Jackson Women’s Health Organization,[3] legal debate has largely focused on whether reproductive autonomy remains protected as a substantive constitutional right.[4] This blog asks a different question: when the state restructures reproductive governance—by redefining embryos, prohibiting their disposition, or altering regulatory frameworks—what procedural obligations follow?

Substantive due process asks whether certain forms of state regulation impermissibly infringe constitutionally protected liberty interests; procedural due process concerns how it must regulate when doing so.[5] Even where no fundamental right is recognized, the Constitution requires fair procedures when the state deprives individuals of recognized liberty or property interests.[6] Research on procedural justice further suggests that the legitimacy of the state depends not only on outcomes, but on the fairness and predictability of decision-making processes.[7] In the assisted reproductive technologies context, the erosion of procedural safeguards risks replacing constitutional order with government arbitrariness.

II. The Vanishing Question of Procedure

Since Dobbs v. Jackson Women’s Health Organization repudiated the constitutional right to abortion, debates over reproduction have largely been framed in binary terms: either reproductive autonomy is a fundamental right, or it is not.[8] What has followed is not a careful recalibration of how states regulate reproduction, but a proliferation of blunt legal interventions. Recent reporting by organizations such as the Guttmacher Institute indicates that multiple states have enacted or proposed legislation recognizing forms of fetal or embryonic personhood, often with significant implications for assisted reproductive technologies.[9]

Notably absent from these developments is sustained attention to procedural due process, the requirement that when the government deprives individuals of liberty or property, it must do so through fair and predictable procedures.[10] A skeptic might argue that if no substantive right exists, no procedural protection is triggered. This reasoning overlooks a central feature of due process doctrine. Procedural due process does not protect “fundamental rights” alone; it protects any “liberty” or “property” interest created by existing law, including interests shaped by state-created regulatory frameworks, contractual arrangements, or settled practices.[11] As the Supreme Court explained in Perry v. Sindermann, constitutionally protected property interests, for instance, may arise from “mutually explicit understandings,” even where no formal entitlement is guaranteed by statute.[12]

This blog does not contend that participation in assisted reproduction creates a freestanding constitutional right. Rather, it argues that once the state affirmatively structures, licenses, and regulates a reproductive medical framework, it assumes procedural obligations when altering that framework in ways that arbitrarily infringe settled reliance interests. Patients who have invested genetic material, significant financial resources, and years of medical reliance on IVF may have reliance interests analogous to the mutually explicit understandings recognized in procedural due process doctrine.

III. The Alabama Example

In most areas of law, decisions with profound personal consequences—termination of parental rights, involuntary commitment, or denial of public benefits—trigger procedural safeguards designed to ensure fairness and predictability.[13] The regulation of assisted reproductive technologies, however, increasingly operates outside this framework.

Critics may respond that general legislation does not require individualized hearings.[14] Under cases such as Bi-Metallic Investment Co. v. State Board of Equalization, 239 U.S. 441 (1915), when a rule applies to the public, due process does not mandate case-by-case adjudication.[15] That principle is well established. But the concern here is not the absence of individualized hearings; it is the absence of legitimate, transitional governance when legal rules are abruptly redefined in ways that disrupt settled reliance interests.

When a legislature or court suddenly reclassifies embryos, the deprivation of genetic material and decisional authority may follow automatically, without notice, safe-harbor periods, or prospective application. Procedural justice research suggests that individuals are more likely to accept regulatory outcomes when decision-making processes are perceived as fair, transparent, and respectful.[16] Abrupt legal shifts without transitional mechanisms undermine not only expectations but institutional legitimacy.

The Alabama Supreme Court’s 2024 decision in LePage v. Center for Reproductive Medicine exemplifies this dynamic.[17] The court’s interpretation of the state’s wrongful death statute effectively reclassified embryos in a manner that significantly altered the legal framework within which IVF patients had structured their reproductive decisions, without providing a procedural mechanism to mitigate the consequences of that shift.

No notice period or prospective application was offered.[18] Had transitional safeguards been provided—such as prospective limitation of liability, grandfathering provisions, or time to transfer embryos—the regulatory change might have avoided the appearance of judicial arbitrariness. Instead, deprivation operated immediately and categorically.

As articulated in Mathews v. Eldridge, 424 U.S. 319 (1976)—a case involving the termination of Social Security disability benefits—due process analysis evaluates the private interest at stake, the risk of erroneous deprivation, and the probable value of additional procedural safeguards.[19] Although Mathews arose in an administrative benefits context rather than general legislation, its framework highlights a broader constitutional concern: when legal change creates a significant risk of unjustified deprivation of structured private interests, the availability of procedural mitigation mechanisms becomes normatively and institutionally consequential.

IV. Why IVG Raises the Stakes

Emerging technologies such as IVG will only amplify these procedural deficits. IVG would enable the creation of large numbers of embryos from somatic cells, increasing the frequency and complexity of decisions about storage, testing, and disposition. In some cases, abrupt legal reclassification could leave patients without access to what may be their only medically feasible pathway to genetically related parenthood or could render years of reproductive planning legally precarious. As reproduction becomes more regulated through licensing regimes, statutory definitions, insurance mandates, hospital oversight, and potential embryo registries, the consequences of abrupt legal reclassification by courts or legislatures grow more significant.

As reproductive technology governance becomes more structurally regulated, its insulation from individualized procedural protection becomes more consequential. When complex regulatory systems operate without mechanisms to manage reliance or provide transitional safeguards, the risk of systemic arbitrariness increases—not because of case-specific misjudgment, but because legal change leaves no room for mitigation.

Without mechanisms to surface contradictions in policy—such as promoting childbirth while chilling the technologies that enable it—the law risks becoming not only restrictive but internally incoherent. IVG does not simply expand scientific possibilities; it intensifies the need for governance structures that account for reliance, predictability, and the cumulative effects of regulatory change.

V. Conclusion: Procedural Legitimacy and Reproductive Governance

Reframing the regulation of reproductive technologies as a procedural due process problem does not require resurrecting Roe. It requires only recognition that even where the state may regulate reproduction, the legitimacy of that regulation depends on how change is implemented. Even where regulatory authority shifts from administrative agencies to legislatures or courts, the underlying demand for procedural fairness does not disappear; it becomes more difficult to enforce and more consequential when absent. Procedural safeguards demand transparency, neutrality, and meaningful opportunities for affected parties to anticipate and respond to regulatory shifts—features that procedural justice research identifies as central to legal legitimacy. They obligate states to justify not only what they regulate, but how regulatory change is structured and whom it disrupts.

When these safeguards disappear, reproductive governance risks sliding from constitutional order toward arbitrary power. The erosion of procedural norms normalizes a vision of reproduction as an administrative permission rather than a domain structured by reasoned and predictable legal processes. Even where substantive reproductive rights are contested, the durability and legitimacy of reproductive regulation depend on procedures that respect reliance, mitigate abrupt disruption, and constrain arbitrariness.

Reproductive technology governance without procedural fairness does not simply narrow autonomy; it undermines the constitutional commitment to law as a system of reasoned and accountable decision-making.

References

[1] See Nat’l Acads. of Scis., Eng’g & Med., Heritable Human Genome Editing 89–92 (2020).

[2] See Henry T. Greely, The End of Sex and the Future of Human Reproduction 1–20, 120–50 (Harvard Univ. Press 2016).

[3] Dobbs v. Jackson Women’s Health Org., 597 U.S. 215 (2022).

[4] See, e.g., Elizabeth Price Foley, Dobbs and the Future of Substantive Liberty, 64 Santa Clara L. Rev. 159 (2024).

[5] See, e.g., Erwin Chemerinsky, Constitutional Law: Principles and Policies § 10.1 (6th ed. 2019).

[6] Board of Regents v. Roth, 408 U.S. 564, 569–70 (1972); Mathews v. Eldridge, 424 U.S. 319, 332–35 (1976).

[7] Tom R. Tyler, Why People Obey the Law (1990).

[8] Dobbs v. Jackson Women’s Health Org., 597 U.S. 215 (2022); see, e.g., Elizabeth Price Foley, Dobbs and the Future of Substantive Liberty, 64 Santa Clara L. Rev. 159, 181 (2024).

[9] See Guttmacher Inst., State Policy Trends 2025: Full-Year Analysis (Feb. 4, 2026), https://www.guttmacher.org/2025/12/state-policy-trends-2025-full-year-analysis; Guttmacher Inst., State Policy Trends: Midyear Analysis (June 16, 2025), https://www.guttmacher.org/2025/06/state-policy-trends-midyear-analysis; Guttmacher Inst., First Quarter 2024 State Policy Trends (May 8, 2024), https://www.guttmacher.org/2024/05/first-quarter-2024-state-policy-trends.

[10] U.S. Const. amend. XIV, § 1; Mathews v. Eldridge, 424 U.S. 319, 332–35 (1976).

[11] See Board of Regents v. Roth, 408 U.S. 564, 569–70 (1972); Perry v. Sindermann, 408 U.S. 593, 601–02 (1972).

[12] See Perry v. Sindermann, 408 U.S. 593, 601 (1972).

[13] Santosky v. Kramer, 455 U.S. 745, 753–54 (1982); Goldberg v. Kelly, 397 U.S. 254, 267–71 (1970).

[14] Bi-Metallic Inv. Co. v. State Bd. of Equalization, 239 U.S. 441 (1915).

[15] See Bi-Metallic Inv. Co. v. State Bd. of Equalization, 239 U.S. 441, 445 (1915).

[16] Tom R. Tyler, What Is Procedural Justice?, 22 Law & Soc’y Rev. 103 (1988).

[17] LePage v. Ctr. for Reprod. Med., P.C., 2024 WL 1161240 (Ala. Feb. 16, 2024).

[18] The Alabama Supreme Court’s 2024 decision in LePage v. Center for Reproductive Medicine, No. SC-2022-0579 (Ala. Feb. 16, 2024), exemplifies this dynamic.

[19] Mathews v. Eldridge, 424 U.S. 319, 335 (1976).

Fusus is an open-platform system that integrates existing cameras, body worn cameras, drones, and other security assets into a single common operating picture, allowing law enforcement to act on real time data and AI-driven analytics.

Lightpost draws power from streetlights to capture vehicle and license plate information, and when combined with Fusus, enables officers to track suspect vehicles, build hotlists, and resolve incidents rapidly — as demonstrated by a beta program hit-and-run case resolved in 20 minutes that recovered $80,000 in illegal drugs.

The Q&A covered topics including privacy compliance (governed by MOUs and local retention laws), the donor program that provides free Fusus cores to local businesses, the absence of facial recognition in the platform, and a subscription/hardware-based business model tailored to each agency’s existing infrastructure.

Watch Charley Moore’s presentation at the CodeX Group Meeting

Transcript

Roland Vogl: Today we have Charley Moore, who’s the product manager at Axon and is in charge of Axon Lightpost. Axon is a leading company in the real time crime center space. That’s an area and technology space we haven’t covered much before. So we’re really curious for you, Charley, to educate us a little bit about that space and tell us specifically what you’re doing there, and maybe show us a little bit of the technology.

There are no other updates other than Future is coming up in two months. So if you haven’t registered yet, please do so at codexfuturelaw.com. And with that, I will now turn it over to Charley.

Charley Moore: My name is Charley Moore. As Dr. Vogl said, I am the lead product manager for Axon Lightpost, which is a new ALPR — automatic license plate reading — solution that we launched in December. I had the privilege of meeting Dr. Vogl when I was actually representing Axon at the career fair back around September, and he invited me to present here. I could not be more grateful for the opportunity.

So I’m looking forward to talking about Axon Fusus, which is where the majority of my work at Axon has revolved around, and then giving a little background about Axon itself.

Axon was founded as TASER International. The first product that came out was the TASER, then cameras, then body worn cameras. And now the majority of my work has been in the real time crime center space. That’s what I’m going to talk about mostly for this presentation.

I started at Axon as a mid-market enterprise executive, as an SDR on that team, working in security operations centers — selling the same technology that we use for real time crime centers to enterprise customers and private businesses, allowing them to share video footage and incidents directly with law enforcement, and using a lot of the same technology that law enforcement uses for their own operations.

From there, I became a product manager on the Axon Vehicle Intelligence team, helping develop our strategy around vehicle information for crime prevention. The two cameras we came out with are Axon Outpost and Axon Lightpost. Axon Outpost is a solar powered ALPR camera, and Axon Lightpost — which you can see here — is a streetlight-powered ALPR solution. It draws power from the street light, which allows you to power the camera and provide connectivity for high-speed, high-distance, very accurate vehicle capture and license plate information for crime prevention.

What Is a Real Time Crime Center?

The first real time crime center was established by the New York City Police Department in 2005. Real time crime centers can come in many forms — they can be as large as a dedicated room with multiple different screens, or as small as a laptop. The key to a real time crime center is the opportunity to take data and analytics, merge all of your different security operation assets into a single common operating picture, use that data and those analytics as efficiently as possible, and get that information to first responders in real time — so that they have as much situational awareness as possible heading into a situation.

Axon Fusus

Fusus is one vendor in the real time crime center space. Some of the advantages of Fusus:

Fusus is an open ecosystem. It allows you to work with your existing infrastructure — whatever you have — and merge it into a common Fusus operating picture. Fusus works with multiple different camera vendors. What you can see is a Fusus core box, which sits on a network and allows you to draw video streams from multiple different vendors. So whatever cameras you already have in place — whether you’re an agency or a private business — it allows you to take your existing cameras and merge them into a common operating picture, along with all of your different security assets: body worn cameras, drones, anything you have for your security operations.

One of the other things Fusus allows you to do is use AI-driven workflows with your existing cameras. The Fusus core — which will fit in the palm of your hand — allows you to run AI analytics directly on your existing cameras, even if those cameras themselves don’t have AI analytics. So when I get into the workflows of Lightpost, one of the things to think about is: if you don’t have a license plate, you can still use Fusus to search for things like the car’s make and model, shirt colors, and so on, using those AI analytics on the cameras themselves.

To summarize, Axon Fusus provides:

• A common operating picture
• Real time emergency video access
• More efficient use of personnel, because you know what’s going on on scene
• Enhanced precision policing
• Fostered community collaboration
• Cost savings, because Fusus works with your existing infrastructure

Example Workflow: Axon Lightpost

I’m going to dive into an example workflow for Lightpost — the ALPR camera that I am the product manager of — and talk through one of the workflows and cases that came out of our beta program.

This incident was a hit and run. A vehicle was driving, crashed into another car, and immediately sped off. The victims provided officers with a picture of the suspect’s vehicle. As I noted, even if they hadn’t had a clear image of the license plate — which in this case they did — we can still use Fusus to solve that crime, because instead of searching by license plate specifically, you can search by things like “blue vehicle.”

But in this case we had the license plate. So the officers entered that license plate into Fusus to receive an automatic alert on what’s called a hotlist, which alerts when that vehicle is found on another Lightpost camera. From there, officers actually had a nearby traffic camera that was also integrated into Fusus. They were able to pull up that traffic camera, confirm that the incident took place, and then send that footage to the real time crime center in their command center — both to gather more evidence of the incident and to confirm the victim’s statement.

Officers then performed an ALPR search, going back to see all the locations that vehicle had been captured by their previous Lightpost cameras. They performed that search and were alerted to the nearest location where that vehicle had been found in the city. From there, they searched one of their other nearby traffic cameras and found that vehicle turning into a parking lot. They then dispatched officers to that location and, in real time, were able to stream that body worn camera footage directly to Fusus to see that the situation was being resolved — helping the officers on scene understand what was going on, communicating with them, and recording the incident.

If you haven’t heard of Axon — if you ever see body worn camera footage on YouTube in the future and look in the upper right-hand corner, you’ll see a yellow triangle. That’s the Axon logo. So you’ll start to notice that a lot of body worn camera footage you come across is from an Axon body worn camera.

Results of this workflow:

• Recovered $80,000 worth of illegal drugs; a large quantity of illicit substances packaged for individual sale were removed from the streets.
• Footage of the incident was used to confirm the victim’s statement, resulting in an evidence-backed arrest.

Here’s a direct quote from the officer: “Without the technology available to us, the case would have likely gone unsolved, as the driver was not the registered owner of the suspect vehicle.” In this case, had we not resolved that incident within 20 minutes — from initial call to suspect apprehension — and had footage of where that vehicle traveled, it would have been difficult to confirm that the person in the vehicle was at the location when the incident took place. But because we had all this information available, it allowed the officers to make sure they were arresting the right person, understand the situation as they were heading into the response, and proceed from there.

Video: Royal Bahamas Police Department

Before Q&A, I’ll play a short video from the Royal Bahamas Police Department, which I think is very helpful in showing the value of Fusus through a real example of them using it to capture a shooting suspect.

Royal Bahamas PD representative: You want an agency that can act fast when crime happens. You need people in a command center letting those mobile units know — “hey, this is happening, a shooting is happening, a robbery is happening.” That also helps with community trust. Once the community sees that police are responding fast enough, that is what our real time crime center was established for.

Before Axon Fusus, we had all these different platforms spread out. You would have CCTV cameras on one platform, another system on another. But now with Axon Fusus, everything is integrated into one. You have body cams, ShotSpotter, our CCTV program — everything is on one platform. It makes the response time a lot quicker.

For instance, we had a matter in 2024 with a shooting incident. ShotSpotter went off in Fusus. We clicked on the cameras, which pulls all cameras within a mile radius of where the shot happened. Of course, we don’t know exactly what happened just yet, but we see there’s a white Honda running past a red light behind the victim’s vehicle. We were able to get the license plate — and not only alert control on how the vehicle looks, we were able to broadcast a photo of the vehicle in Fusus so they could actually see the vehicle. So not only can we get a description over the radio, we’re actually seeing it on our mobile device exactly how the car looks. And within an hour or two, that suspect was arrested.

Technology revolutionizes policing, enhancing efficiency, accountability, and crime prevention through surveillance, analytics, and real time data. We balance safety and hospitality through crime prevention, rapid response, and strategic policing to protect all, while upholding the Bahamas’ welcome and reputation.

Charley Moore: While that video was from the Bahamas, I think it really shows the value of real time information in precision policing and policing operations, and the value that Lightpost and other security assets provide. With that, I’ll turn it over to questions. Thank you again for your time.

Q&A

Roland Vogl: This is super interesting. I have a couple of questions. First, how many cameras are typically deployed in a city that you provide, and how much information is coming in from other sources? The video mentioned CCTV, and sometimes you hear about feeds from Ring cameras or gas stations. How is all this information being brought together and put in front of the police officers?

Charley Moore: Great questions. For the first — how many cameras — it’s really up to the agency, what their budget is, and what they’re looking for. We’ve done pilot programs with as few as four cameras, just to capture the entrances and exits of a city, so that if something happens in their location, they can alert a nearby jurisdiction that a suspect is heading their way. Other agencies can deploy up to 50 or many more cameras. It really just depends on the size of the city and the size of the agency.

For the second part — how many assets can be in a Fusus panel — it can be every officer’s body worn camera, every drone, every single security asset. There’s no limit in Fusus; Fusus doesn’t charge by the number of assets you integrate. It’s pretty easy to integrate specific assets that have existing integrations with Fusus. So there’s really no limit. It’s up to the agency for how much information they want in Fusus, whether they want to focus on specific security assets, and what their budget is for their real time policing operations.

Roland Vogl: What about other intermediaries that try to aggregate information and then sell it to the police? For example, if a private security company has installed a camera at a gas station and a crime occurs there — is there an agreement with the police to have a feed, or does the police have to go there and subpoena or request the video?

Charley Moore: Yes, there are some companies that aggregate that data. That’s actually related to our Fusus donor program. One of the things that every agency gets when they sign up with Fusus is an allotment of those Fusus core devices that I talked about earlier. The agency can actually give those cores for free to local businesses — especially gas stations, convenience stores, stores that see a lot of crime and are very affected by it. They sign an MOU with the police department that details when and where the police can access that footage. From there, instead of having to manually share footage with law enforcement, when you call law enforcement they can immediately access that footage, see what’s going on, help with community safety, and know exactly what situation they’re getting into when they respond.

Roland Vogl: What about compliance frameworks that govern Fusus — specifically about legal document automation, chain of custody, and evidence disclosure workflows downstream from Fusus data.

Charley Moore: Right now, sharing video feeds with Fusus is governed by a memorandum of understanding signed between the business sharing their video feeds and the police department. There are also community council meetings when deploying Fusus, and it’s publicized so that the community has input. The community can see the privacy safeguards in place, and there’s a lot of documentation to make sure Fusus is very transparent about when and where the police can access video footage and how they can use it. The primary mechanism is that MOU between the business, homeowner, or private citizen who wants to share video directly with the police department.

Roland Vogl: What about privacy rules, what does compliance mean in practice? The information picked up isn’t always of a specific crime, so there’s still a lot of other information that could be otherwise sensitive. What’s done with that information — is it stored, then deleted? What practices are in place?

Charley Moore: That’s a really good question. Every state and county has its own rules and restrictions on when the video can be deleted, how it can be deleted, and how long it has to be retained. So before deploying Fusus, it’s important to understand what those agency’s rules are. Fusus was founded — I believe in 2019 — but video feeds for law enforcement have been around for a while, so most agencies are already familiar with the rules around storage and use. Fusus is able to remain compliant by automatically setting video feeds to be retained only for the duration that the county specifies, and we figure that out before deployment to make sure we’re in compliance with all local ordinances.

Roland Vogl: It sounds like body cameras have brought a lot of clarity to situations that might otherwise have been unclear, and some of those situations have led to significant public incidents. Are there rules requiring that body cam footage be shared with the public or be accessible through public records requests?

Charley Moore: Yes. There are a lot of different rules depending on the county, but in some counties all of that footage has to be stored for, let’s say, 30 days. If a member of the public makes a Freedom of Information Act request, they can request that footage directly from the agency. Within Fusus, you can go in, select the video clip from a specific camera that you want to share, and download it to your computer. If it’s a private company using Fusus, you can share that video clip directly with them — for example, if it’s a shoplifting suspect that Walmart is looking for. But if it’s a request from the general public, you download the video clip, save it, and then share it securely in response to the Freedom of Information Act request.

Roland Vogl: How does the system manage legal consent. Is the system used to perform facial recognition?

Charley Moore: Axon does not do facial recognition today in the use of Fusus. When I talk about object detection, I’m talking about things like shirt color, shoes, pants — but today Fusus does not provide facial recognition.

Roland Vogl: Benjamin is asking about the Fourth Circuit Court of Appeals saying that blanket surveillance — specifically all cameras — violates the Fourth Amendment right to privacy.

Charley Moore: I would have to look up that case specifically and do some research. To be totally transparent, I have not heard of that specific case. I’d definitely want to look into it and am happy to connect offline if you want to have a discussion, because I’d love to learn more about that. But from what I can see, we’re fully in compliance with all local laws.

Roland Vogl: Most of your customers are police departments, but you’re also selling to private organizations?

Charley Moore: Yes, exactly. Security operations centers — using the same technology that law enforcement uses, but applying it to your stores and businesses to merge all of your operations and security assets into a common operating picture.

Roland Vogl: What’s the business model — is this a subscription sold to police departments?

Charley Moore: For Lightpost, it’s part of a five-year contract and involves buying the hardware. So you’re purchasing Lightpost, you own the hardware, and then the service is billed annually. It depends on the specific products. With Fusus, it used to be that you paid per stream — for however many cameras you have, you paid for each of those cameras. Today, you’re buying packages depending on how large a real time crime center you’re building out with Fusus. But not everything is bundled together — Fusus doesn’t necessarily come with Lightpost or another camera. It’s really about building and mixing and matching the technology for each agency’s needs. A lot of that also depends on what technology the agency already has, because there’s no point in buying extra cameras if you already have high-functioning cameras in place. We come in, work with those existing cameras, and bring them into Fusus the same as any cameras you might purchase directly.

Roland Vogl: How does the software work — is it machine learning, visual recognition algorithms? What’s the tech stack used to match and track?

Charley Moore: That depends on the product. For Lightpost, it’s a third-party algorithm that runs on the device today. For Outpost, it’s a proprietary algorithm that runs on device. And then for standard video cameras, those have their own AI algorithms that run on the device and also do some processing in the cloud. They allow you to take those video feeds through what’s called RTSP — a standardized protocol for video feeds — take that video feed, run it through the core, process it with AI analytics, and then send that video to Fusus.

Roland Vogl: All right, well, thank you so much, Charley. 

Charley Moore: I’m happy to continue the conversation — I love this technology and I’m really passionate about all of it. Happy to connect with anybody and talk about Lightpost and policing.

Mello, Char, and Xu propose in JAMA a two-factor framework for deciding when healthcare organizations should notify patients about AI tools or seek their consent. The framework asks organizations to assess risk of harm and patient agency, then sort AI tools into three bins, namely consent, notification, or neither. I argue that the framework rests on three premises that undermine its own architecture. First, it treats “human in the loop” oversight as a reliable error-interception mechanism while simultaneously cataloging the reasons it is not. Second, it quantizes patient agency into a binary when agency exists on a gradient, assigning patients the role of quality-control agents while arguing elsewhere that patients cannot absorb more information. Third, it presumes that healthcare organizations possess the evaluative infrastructure to perform the risk assessments the framework demands. These are not independent weaknesses. They are surface expressions of a deeper structural problem. The Mello framework is a consent architecture, and consent architectures fail when they assume human capacities that do not exist at scale. Privacy law scholarship has already diagnosed this failure and begun developing alternatives. Healthcare AI governance should reckon with the same insight. I acknowledge that diagnosing the consent failure is easier than building the alternative, and that disclosure sometimes imposes real costs on patients. But these complications do not rescue a framework whose premises do not hold.

I. The Source and Its Stakes

Michelle Mello, Danton Char, and Sonnet Xu published “Ethical Obligations to Inform Patients About Use of AI Tools” in JAMA in September 2025. AI tools are proliferating across healthcare settings, and practitioners need guidance on what to tell patients about them. The authors propose a two-factor framework organized around (1) the risk that using the tool could cause physical harm and (2) the extent to which patients can exercise agency in response to a disclosure. Tools that score high on both factors warrant consent. Tools that score high on one warrant notification. Tools that score low on both warrant neither.

The framework contains structural weaknesses that deserve direct treatment. I will focus on three, then explain what connects them.

II. The Circular Loop

The authors build their framework on the premise that human oversight reduces residual risk to patients. They write that “most tools in use today entail a human reviewing and acting on model output,” and that “the key question therefore is whether the residual risk to patients is low once one considers the likelihood that the human in the loop will successfully detect errors.”

This is the load-bearing wall of the entire architecture. It determines which tools fall into the “neither consent nor notification” category. If human oversight reliably catches errors, residual risk is low, and disclosure becomes unnecessary.

And then the authors undermine it themselves. In the very next section, they acknowledge that “capacity constraints, automation bias, and users’ unfamiliarity with a tool’s weaknesses may undercut efficacy.” Automation bias is one of the most well-documented phenomena in human-computer interaction research. Clinicians who know an AI has flagged or not flagged a condition routinely defer to the machine’s judgment. The premise that human oversight will “successfully detect errors” is precisely the premise that automation bias scholarship has spent two decades dismantling.

The framework says, in effect, that disclosure is unnecessary when a human in the loop will catch errors, and then concedes that humans in the loop frequently do not catch errors. If the human-in-the-loop assumption holds only sometimes, the entire “neither” category is unstable. Some of the tools the authors place in that category belong there. Some do not. And the framework provides no mechanism for distinguishing which is which.

III. The Agency Paradox

The second factor in the framework asks whether patients have a “meaningful opportunity to exercise agency.” The authors identify two forms. Patients might opt out of the AI tool, or patients might alter their behavior in response to knowing AI is involved.

The second form does important work. It is the basis for recommending notification for AI-drafted patient emails and AI-generated clinical summaries. The authors argue that a patient who knows an email was AI-drafted “may be more likely to question something that seems odd,” and that a daughter who knows a nursing summary note relating to her mother was AI-generated “may be more likely to log on to the electronic health record, check the note for errors and omissions, and alert the incoming nurse.”

Earlier in the article, however, the authors argue against broad disclosure partly because “patients who want to be kept informed about their care in principle may struggle with the information overload that a hospital admission entails.”

These two positions are in tension. You cannot simultaneously argue that patients are too overwhelmed to process more information about AI tools and that informed patients will serve as effective quality-control agents for AI-generated communications. The daughter expected to check an AI-generated nursing note for errors needs the medical literacy to recognize what constitutes an error, the time and inclination to log into the electronic health record, and an understanding of what the note should contain. The framework assigns her a role it has given no reason to believe she can perform.

The deeper problem is that the framework treats agency as binary. Either patients can opt out or they cannot. Either they can alter their behavior meaningfully or they cannot. But agency exists on a gradient. A patient told about an AI tool but lacking the expertise to evaluate its output has more agency than one told nothing, but less than the framework assumes. A more honest treatment would acknowledge that the “notification” category rests on aspirational rather than demonstrated patient capacity.

IV. The Missing Institutional Competence Question

The framework instructs healthcare organizations to assess, for each AI tool, “(1) the risk that the tool poses, (2) the likelihood that errors will reach patients without being intercepted, and (3) the severity of the harm that could result.”

Who performs this assessment? With what data? And using what methodology?

Most healthcare organizations are still struggling with basic AI governance, the setting of AI governance committees notwithstanding (that’s window dressing). They lack the technical personnel to audit algorithmic performance across patient subgroups, the data pipelines to monitor error rates in production, and the institutional processes to translate risk assessments into disclosure policies applied consistently. These are not speculative deficiencies. A 2025 CHIME Foundation survey found that only 8% of healthcare organizations described themselves as ‘very confident’ in their ability to identify emerging AI risks, and a little more than half had a formal process requiring approval before AI implementation. There is a certain irony in a framework that worries about “perfunctory and legalistic” consent being implemented by institutions that may apply the framework itself in a perfunctory and legalistic manner. An organization that lacks the infrastructure to assess algorithmic risk will default to the path of least resistance. And the path of least resistance in this framework is the “neither” category, because it requires no action. The framework’s own structure creates an incentive to underassess risk.

V. The Disclosure-Harm Tradeoff

Before connecting these weaknesses, I want to engage the strongest argument the Mello framework has in its favor, because it is genuinely difficult.

The authors argue that disclosure can paradoxically harm patients. They cite evidence that patients perceive AI-drafted messages as more empathetic than physician-drafted ones, but rate those same messages significantly lower once told AI was involved. When AI-augmented care outperforms the alternative, a consent regime may result in clinicians “having to deliver suboptimal treatment.” This is a real cost. It is not hypothetical. And any critique of the framework must reckon with it.

I think the argument is correct on its own terms but insufficient as a foundation for the framework’s permissive categories. The disclosure-harm evidence establishes that some patients, told about some AI tools, will make choices that leave them worse off. That is a genuine tradeoff. But the framework uses this tradeoff to justify a “neither” category that sweeps in tools where the calculus is far less clear. The evidence that disclosure harms patients comes from specific contexts (patient messaging, mammography interpretation) and cannot be generalized to every tool the framework places in the “neither” bin without further empirical work. The framework treats a finding about particular tools as a license for institutional silence across categories.

Moreover, the cost of disclosure-induced suboptimal choices must be weighed against the cost of systematic under-disclosure by organizations that default to the category requiring no action. The former cost is visible and measurable. The latter is diffuse and delayed, which makes it harder to track but not less real.

VI. The Consent Substrate

The three weaknesses in Sections II through IV are not independent. They share a common root. The Mello framework is, essentially, a consent architecture. It assumes that if organizations assess risk and disclose appropriately, patients will exercise informed agency. The three failures are symptoms of a condition that privacy law scholarship has already diagnosed.

The notice-and-consent paradigm in privacy law fails at four sequential stages. Reading is impossible given the volume of policies. Comprehension is unattainable given their complexity. Evaluation is foreclosed because users lack the technical expertise to assess risk. And action is impossible because users face take-it-or-leave-it terms. Even ambitious regulatory frameworks like the CCPA have failed to remedy these defects, because rights are rendered worthless when they cannot be exercised. The evidence base for these claims is substantial. Carnegie Mellon researchers calculated in 2008 that reading the privacy policies an average American encounters would require roughly 30 working days per year. Literacy surveys show nearly half of American adults lack the reading level these policies demand. CCPA exercise rates remain remarkably low years after implementation.

Empirical work on patient attitudes toward AI disclosure exists and is growing, but there remains almost no empirical measurement of the structural variables the analogy requires, such as how many AI tools a typical admission implicates, whether patients in practice comprehend AI disclosure forms, or how often patients exercise opt-out rights when offered, and I want to be precise about that gap. But the structural parallels are strong enough to warrant concern. Reading is impossible when a patient’s care implicates dozens of AI tools. Comprehension is unattainable when evaluating algorithmic risk requires expertise patients do not possess. Evaluation is foreclosed because patients cannot assess whether a human in the loop is actually catching errors. And action is impossible for operational AI tools that patients cannot opt out of. Empirical work specifically measuring patient comprehension of AI disclosure forms, and patient exercise rates when opt-outs are offered, would test whether these parallels hold quantitatively. But the structural logic does not depend on the numbers being identical. It depends on the same mismatch between assumed and actual human capacity.

Privacy law scholarship has begun moving past this impasse. The emerging recognition is that when consent is structurally impossible, effective protection requires delegation to intermediaries capable of acting at scale on behalf of individuals. Whether that intermediary takes the form of an AI agent with limited legal capacity, a fiduciary with enforceable duties, or some other institutional design, the underlying insight is the same. The evaluative function that consent regimes assign to individuals must be relocated to systems designed to perform it.

I am aware that identifying the consent failure is easier than building the alternative. An intermediation model for healthcare AI governance raises questions about cost, access, liability, and regulatory design that this commentary cannot resolve. Healthcare is a domain where regulatory complexity, liability exposure, and patient vulnerability are all higher than in consumer data protection. The intermediaries that privacy law scholarship envisions do not yet exist in healthcare, and constructing them is an enormously difficult institutional project.

But the first step is recognizing that the current framework rests on assumptions that do not hold. The Mello framework asks patients to do what privacy law has demonstrated individuals cannot do, namely evaluate institutional disclosures about complex algorithmic systems and exercise meaningful agency in response. A healthcare AI governance framework that takes the consent literature seriously would ask not “what should patients be told” but “what institutional structures ensure that someone with the competence to evaluate algorithmic risk is actually doing so on the patient’s behalf.”

VII. What the Framework Does Not Say

The Mello framework assumes human oversight that automation bias research calls into question. It assumes patient agency that information overload scholarship undermines. It assumes institutional capacity that healthcare governance surveys consistently find lacking. And it inherits these assumptions from a consent paradigm whose structural failure is no longer a matter of speculation but of accumulated evidence.

A disclosure framework is only as strong as the institutions that implement it, and a consent architecture is only as strong as the humans it expects to exercise consent. Without intermediation, the elegant two-factor matrix becomes a mechanism for sorting AI tools into the category that requires the least organizational effort. That is not a governance framework. That is a permission structure for opacity.

Roger Nzouetchep Ketat est étudiant dans le cadre du cours DRT6929 (Vie privée + Numérique) (Hiver 2026)

La protection des renseignements personnels constitue un enjeu majeur pour les organisations, publiques comme privées, dans un contexte caractérisé par la numérisation accrue des services et la circulation importante de l’information. Afin de prévenir et de limiter les conséquences des incidents de confidentialité, plusieurs États, notamment parmi les pays développés, ont procédé à la modernisation de leur cadre législatif ou à l’adoption de nouvelles mesures en matière de protection des renseignements personnels. Ces évolutions ont entraîné un renforcement des obligations des organisations, tant publiques que privées, en matière de confidentialité, de sécurité de l’information et de reddition de comptes.

Dans ce contexte, les incidents de confidentialité demeurent une préoccupation constante. À titre d’exemple, un incident récent survenu au Nouveau‑Brunswick, impliquant l’envoi par erreur les numéros d’assurance maladie de plus de 350 ménages à une mauvaise adresse, a compromis la confidentialité de nombreuses personnes. Selon l’article du journal, le gouvernement du Nouveau-Brunswick explique que certaines personnes

« pourraient avoir reçu par la poste des renseignements sur l’assurance-maladie appartenant à d’autres personnes en même temps que les leurs ».

Cet événement met en lumière l’importance d’une gestion rigoureuse et conforme des renseignements personnels.

Le numéro d’assurance maladie est considéré comme un renseignement personnel sensible en raison de son lien direct avec la santé et l’intimité des personnes. Il nécessite donc une protection élevée afin éviter l’usurpation d’identité ou l’accès non autorisé à des dossiers médicaux. La communication de ce numéro à des tiers non autorisés constitue, selon la législation du Nouveau-Brunswick, une atteinte non justifiée à la vie privée au sens de l’art. 21(2) de la  .

Au Nouveau Brunswick, notamment dans le secteur public, dès qu’une atteinte à la vie privée est présumée ou découverte, l’organisme visé doit prendre un certain nombre de mesures :

Dans un premier temps, il doit limiter l’atteinte. Cela consiste, entre autres, à signaler immédiatement l’incident au responsable de la protection de la vie privée, à procéder à une première évaluation, à prendre des mesures immédiates pour mettre fin à l’atteinte et empêcher toute nouvelle communication des renseignements personnels, à sécuriser et, lorsque possible, à récupérer les renseignements personnels déjà communiqués, ainsi qu’à déterminer les personnes qui doivent être informées au sein de l’organisation.

Dans un deuxième temps, l’organisme doit procéder à une évaluation des risques. Il s’agit notamment d’identifier le type de renseignements personnels concerné par l’atteinte et d’en évaluer le niveau de sensibilité, de déterminer la cause et l’ampleur de l’atteinte, le nombre de personnes touchées ainsi que les dommages prévisibles pour les personnes concernées.

Troisièmement, l’organisme doit fournier un avis. Cette étape consiste, entre autres, à informer les personnes concernées ainsi que l’Ombud, organisme chargé notamment d’enquêter sur les plaintes du public en matière d’atteinte à la vie privée.

Quatrièmement, il doit mettre en place des mesures visant à prévenir de futures atteintes. Cela consiste à formuler les recommandations à la suite d’une enquête afin d’éviter qu’une atteinte similaire ne se reproduise, ainsi qu’à élaborer un plan de mise en œuvre des recommandations et des mesures correctives.

Finalement, le Règlement général pris en vertu de la LDIPVP, exige à son alinéa 4.2(4)b), que les organismes publics tiennent un registre de chaque atteinte véritable à la vie privée ainsi des mesures correctives prises.

Le ministre responsable de Service Nouveau-Brunswick, rapporte l’article, assure que le problème ayant causé cette erreur a été corrigé et que des démarches visant à atténuer les répercussions de l’incident sur les personnes concernées ont été entreprises. Il est également précisé que toutes les personnes touchées seront contactées par l’assurance maladie.

À des fins de comparaison, dans l’hypothèse où l’incident serait survenu au Québec, celui-ci aurait été assujetti au régime juridique établi par les articles 63.8 à 63.11 ainsi que par l’article 127.2 de la Loi sur l’accès aux documents des organismes publics et sur la protection des renseignements personnels (RLRQ, c. a-2.1) (Loi sur l’accès), de même que les orientations de la Commission d’accès à l’information (CAI).

En cas d’incident de confidentialité, l’organisme doit, premièrement, prendre des mesures raisonnables pour minimiser les risques et prévenir la survenance de nouveaux incidents. Deuxièmement, il doit évaluer si l’incident présente un risque de préjudice sérieux, notamment en appréciant la gravité du risque pour toutes les personnes concernées, et ce, en concertation avec le responsable de la protection des renseignements personnels. Troisièmement, il doit aviser la CAI ainsi que les personnes concernées, de même que toute personne ou tout organisme susceptible de prévenir ou de réduire le risque de préjudice sérieux. Enfin il doit tenir un registre des incidents de confidentialité.

Comme on peut le constater, en matière d’incident de confidentialité ou d’atteinte à la vie privée, les obligations des organismes publics au Nouveau-Brunswick et au Québec semblent essentiellement les mêmes, à l’exception des pouvoirs de l’Ombud et de la CAI.

La CAI est un organisme public indépendant doté d’un pouvoir de surveillance et d’un pouvoir juridictionnel  à titre de tribunal administratif, dont le champ d’application s’étant tant au secteur public qu’au secteur privé. Elle dispose du pouvoir de rendre des décisions exécutoires. En vertu de l’art. 127.2 de la Loi sur l’accès, lorsqu’un incident de confidentialité est porté à sa connaissance, elle peut ordonner à toute personne l’application de toute mesure visant à protéger les droits des personnes concernées que la loi leur reconnaît, pour la durée et aux conditions qu’elle détermine.

L’ombud, quant à lui, est un agent indépendant de l’Assemblée législative qui agit comme mécanisme d’enquête et de recommandation, sans pouvoir juridictionnel (art. 64.1(1) et 68(1) LDIPVP). Son champ d’application se limite principalement au secteur public.

Depuis la fin juillet 2025, le gouvernement du Nouveau-Brunswick a amorcé une révision de la Loi sur le droit à l’information et la protection de la vie privée. L’un des enjeux de cette réforme concerne précisément la possibilité de doter l’Ombud des pouvoirs d’un organisme d’examen indépendant, comme il est indiqué  à la page 9 du document de travail.

Watch video of 2.12.2026 CodeX Group Meeting with the CBC Project

Transcript

Roland Vogl:

We will now be turning it over to Alexandre Gleria, who will be updating us on his work on the CBC project, which is a project he’s been working on as a CodeX affiliate. He’s of course also a practicing corporate attorney in Brazil, and we’re excited to hear where the project stands at this point.

Alexandre Gleria:

Thank you very much, Roland, and thank you for the opportunity — for being so kind to us here from so far. I’ll try to speed up here because I have a lot of stuff to talk about today. Basically, what is the project about? We’re creating a method and a system that could indicate the human behavior behind business decisions. These could classify companies, sectors, and also jurisdictions. We’re finding that this could have a very broad application in terms of commercial and industrial perspective. 

Based on several research papers that we are writing, this could also be a game changer in several fields, especially because these sort of map human behavior. We could avoid the bias of the data that is provided by the companies, given that all the data that companies provide comes from their own sources. Of course, we have big four companies auditing these numbers, we have accounting looking into these numbers, but to this day there’s a lot of room for fraud, for loopholes, and so on.

These ratings and these classifications are also showing very promising results in terms of having anti-fraud properties, especially because we are observing that the ratings and the rankings of the company only evolve if the human behavior behind the business also evolves. This is a very cool feature that we are discovering in our research. I also have a lot of tests that I can show you of how this could be a game changer in several areas such as credit analysis, stock market analysis, and so on.

Basically, I have here just a cartoon of how this project started. In 2018 — much before the AI generation and products that we have today — we started a project in our firm to try to project, in a mathematical perspective, ways to measure financial impacts and probability of loss in contentious litigation. We also observed that we saw patterns in some of our clients — clients that were not aligned with our culture, clients that were very problematic. They also had a very peculiar pattern of litigation, a particular profile of litigation that this kind of company carried.

Based on both of these observations, from 2018 until today, we hired big four companies, interviewed C-level executives, hired mathematicians, data scientists, and economists. We have some state judges as partners of our project. One of these judges is helping a lot on the inside, and he’s also very good with equations and so on. We have economists on our team who are also here listening to us.

The first starting points of the Corporate Behavior Coding — the trigger points of the project, as we know today — is that we found that the litigation data was very rich in telling us all the stuff, all the information, that is not very clear in financial information. If a company has a bad product, this ends up in litigation. If a company is saying that it treats employees very nicely but behind the scenes does a lot of wrongdoing, this could also end up in courts and so on. If companies have corporate struggles, issues, and fights among partners — all kinds of stuff — in the end, we are confident in the judiciary branch. Basically, based on these, we classified this as a very nice source for understanding behavior, for understanding corporate behavior, or the behavior that is behind business decisions, because all business decisions are made — even though we are in the generation of machines and robots — by humans in the end.

This is a subsequent development of this project. We started to research other sources that could also indicate behavior, such as board compensation numbers and figures, in a sense that we saw that business perpetuity is very strongly connected with how the partners and shareholders of this business are being remunerated and paid over time. We also looked at the perception of qualified stakeholders.

Currently, we have a lot of data being collected from the markets — from, for example, X, or the image, or the press. But what matters here, we saw, is that we need to have quality in terms of data. So we started to realize that, for example, consumers, employees, or previous employees of the company could also provide a lot of good insights about the future, about the perpetuity of the business, and other cool features of the company.

We have also seen, as you can take a look at in the diagrams, that these traces of human behavior are very dense in the litigation and board compensation documents and in the perception of qualified stakeholders. While we have a lot of biases and a lot of limited scope in financial statements of a company, you have a lot of conflicts of interests. Of course the company wants to show the best numbers possible. It’s very unlikely that you could see a raw version of the corporate culture behind those numbers.

Based on that, we built the classification — the rationale of what kind of data we could consider as behavioral data. I will not explain this in detail because we don’t have time. The rationale of the behavioral data is essentially what I just mentioned: avoiding bias, quality of the information, the depth of this information. Most of this data is coming from external sources, and the ratings evolve if the behavior of the business people also evolves.

Basically, we are in the final stage of the paper. As you know, we are also researching a lot of mind-blowing facts from the numbers and figures that we extracted from this study. We are finding that there could exist a lot of symmetries that we also find in nature, within these numbers and these classifications. That is a very cool feature as well of CBC, but we are digging on that and also researching with other specialists.

One thing that is also cool about this project is — why are the results so good? We started to dig a lot on that. Basically, we find that the combination of two sorts of different data provides a lot of images that the numbers alone are not telling us. In an analogy that we make here: if I assume a computer system — as you know, a computer system runs on a binary data scheme — and I assume in this scheme that I am just looking at financial data, that is the zero. I’m not getting a lot of depth in the view of what I’m analyzing. When we add the behavioral data, then we can see images and assumptions that could help decision-makers understand a lot of what is going on in the business. As you can see here, zero doesn’t mean nothing, but when I add the one in the binary code, I can see vision, if you look from outside of it.

We are also seeing that these methods have a lot of advantages over other very recent and promising research. I will not talk about that in depth because of time. Let me just show some results in two or three minutes, and then we can open for questions.

Basically, we ran a lot of tests on 180 publicly traded companies in Brazil. We created some ratings from each of them. Here are some very good-rated companies from big four companies and credit agencies. As you can see, these companies all presented a very bad rating according to our methodology, like ten years before a Chapter 11 event, for example.

Roland Vogl: 

Can you give an example of what behaviors you measure? Like, do you give a rating to companies that display certain behaviors? You mentioned fraudulent behavior, but what other specific behaviors are you tracking? Do you find they have an impact and a dependent variable that goes to the performance of the company? You’re looking at measuring human behavior in a company and then seeing if it has an impact on stock performance, litigation, bankruptcy, or certain outcomes. What exactly are you measuring?

Alexandre Gleria:

We are measuring, for example, the amounts — combining pairs of information and measuring the asymmetry of this information. As an example, if I have a next level of litigation on a certain matter and the company presents very good earnings, this could be an indication that this business, in a small fraction of its DNA, is sustainable. Or, for example, if this company’s litigation level is very acceptable due to market standards — but measuring the occurrences of litigations of a particular company and comparing it to how common litigation is in that particular industry. If one company has more litigation than another, that really is the variable you’re measuring: a high amount of litigations, and then seeing if that has some predictive power over the company’s financial performance.

It does, but isolating litigation data alone is worthless. You need to combine it, and you see the asymmetry between litigation, for example, and other data and the financial robustness of the business. Or, for example, if you see the directors’ payment — the salary of the board of directors — and compare it with the earnings of the company, you can also see levels of whether this business is sustainable, or whether there’s a pattern that indicates shareholders were cashing out the company, and in two or three years this company will go through Chapter 11.

Roland Vogl: 

Got it. Could you share the main conclusions and takeaways?

Alexandre Gleria:

The main conclusions are that our results, as I said, were very robust. We created stock portfolios just based on this methodology, and the returns — not only in Brazil but also in the U.S. — are equally robust. In 2022, we also ran a real test for a hedge fund in Brazil. We analyzed a 20-stock portfolio to see what was going on with it. We realized that one particular stock had some issues when mapping these images based on these contrasts. We identified the problem, told the fund managers what was going on, and since then the stock is down almost 90%. We have a lot of tests here that will be in our paper.

Just to conclude — if possible in two minutes — I want to show you the platform that we are building based on what we are creating.

So basically, here we can compare a lot of companies, not only in Brazil but also outside Brazil. The data is not real because, as you know, we filed the patents a few days ago, but here you can compare a lot of information on litigation, the index of the company, the litigation asymmetries, and the litigation map comparison between two companies. There’s a lot of things that we can measure here regarding the methodology, but this is just a flavor. There’s a lot of theory behind it.

Roland Vogl:

How can people learn more? Is the paper available? Can you share a link?

Alexandre Gleria:

The paper will probably be available within two months. We are finishing it.

Roland Vogl:

Awesome. Thank you so much, Alexandre. I think it’s a cool idea to be able to understand legal indicators and signals — litigation, but also other things that are legally relevant — and then map that to a company’s performance. That’s what you’re trying to do in essence. So what’s your long-term goal? This started as an interest from practicing corporate law with large public companies in Brazil. You said at the beginning you saw some clients with different corporate cultures and behaviors than others. You could almost anticipate that one company would do better than another, and there is a connection between the culture and the way people conduct themselves in a business, and the outcome. That was your thesis, and that was the research project. But there’s also a patent now and a company that will presumably try to make those insights available, because it is potentially tradable information. Can you talk a little bit about how you see the future unfolding for this project?

Alexandre Gleria:

Well, basically, we realized that a company with a specific company profile and culture — they don’t leak targets, they avoid litigation, they have less litigation outstanding in comparison with peers, the quality of their products or services or intangibles are superior in a way that they don’t have a lot of claims in a general sense — not only in litigation but also in behavioral data, such as opinions issued by qualified stakeholders like a consumer, an employee, or an employer.

Based on that, we saw that, for example, if we compare and translate this into real numbers — the amount of litigation in terms of volume, the amount of litigation in U.S. dollars — and compare it with the economic capacity of the company, this gives you an idea of how sustainable the company is. The idea here is to scale that, because there are a lot of people working on unstructured data arrangements and platforms in order to use, for example, transformers to process a lot of material from companies that have a lot of information. But this approach, in our opinion, is not the best one. First, because it consumes a lot of compute resources, and the quality of the raw material and inputs used is not good either.

The idea is that with the right inputs — as in the behavioral data that I just gave examples of — we can scale this to provide, globally, more reliable corporate ratings and corporate classifications in all jurisdictions, regardless of whether that jurisdiction is common law, civil law, or whatever. Especially because we are running tests not only in Brazil but also in the U.S., the U.K., and other jurisdictions. The key point is that once this data is structured, we can scale very fast. Of course, this permits analysis of public companies more easily, but it could also be used by private health companies.

Roland Vogl:

Do you have a name for the paper already?

Alexandre Gleria:

The name — we are thinking about it. CBC was a very corporate name that we chose. The original informal name, the nickname of the project in the first months, was the Corporate Genomics Project. That’s why we named it CBC — because we built a whole theory on that and are just consolidating it to give an official name to the paper. In the paper, we have a lot of mathematicians involved because — I’m an attorney, even though I’m a specialist in tax law and corporate law — we need to have qualified people in this paper. We have a very strong background on math and financial markets in order to prove our theory.

To conclude on the tests: when we compare these ratings with rating agencies or any other kind of data available in financial markets, for some specifics these prove to be much more reliable than anything else. We also prepared slides showing that in the last 50 years we have a lot of innovation, for example in cancer diagnosis and medicines in general terms. Yet when you look through the corporate outlook over those fifty years, we still have fraud scandals, accounting mistakes, and so on. Basically, the idea here is to provide not only a new product and idea to the market, but also a culture of corporate diagnostics.

Roland Vogl:

That’s an exciting future. I think you could apply it to the corporate world, the crypto world, and other areas too. Thank you again, Alexandre. Thanks for joining us.

• Limited Partnership Agreement (LPA)
• Private Placement Memorandum (PPM)
• Subscription Document

It does this by generating a client questionnaire from a term sheet, then using the responses to auto-draft a complete document set in minutes rather than the months it traditionally takes. It also includes a native DocX editor, a co-pilot feature that cross-references documents against whitelisted legal sources like the ILPA guidelines, and visualization tools for key fund terms and entity structures. SwiftLaw operates both as a direct full-stack fund formation service and as a platform licensed to law firms, with the system running locally to ensure data privacy.

Watch 2.12.26 CodeX Group Meeting with SwiftLaw

Transcript

Roland Vogl:

We have Saketh Kesiraju, who’s the CEO and founder of SwiftLaw. So excited to learn from you. 

Just a quick announcement—we just announced the CodeX FutureLaw conference. The main event is April 16, 2026 but we have other exciting events including a hackathon,a bootcamp, and a UN AI for Good law track conference that entire week. We call it CodeX FutureLaw Week and I encourage you to check out the program and hopefully join us in April. 

codexfuturelaw.com—you can find all the information. 

 I will turn it over to Saketh. We’ve learned about SwiftLaw before and are curious to hear where the journey took you, Saketh. Over to you.

Saketh Kesiraju:

Thank you so much, everyone. It’s really an honor to be back here at CodeX. I think it was last year or maybe two years ago. A lot has happened since.

I’m founder of SwiftLaw. SwiftLaw is a vertical AI platform for fund formation. We just want to help people launch funds really quickly. That’s the premise. A little bit about me—first off, I didn’t necessarily get into funds or legal tech on purpose, I would say. 

Back in 2023, I was very much into the crypto space and I was really trying to find people to buy the various crypto real world assets that I was trying to put on chain. I went to a conference in Salt Lake City with a bunch of emerging managers. I was trying to pitch these different emerging managers to sort of buy my crypto projects, and really no one was buying it. No one really cared. But while I was there, I befriended dozens of emerging managers. A lot of them were real estate professionals or HP executives and really these Bay Area Indian uncle types. 

A lot of them really wanted to start these small venture funds, small real estate funds, and so on. While I was talking to them, you know, they were really excited about me because I was just an ambitious kid who took a gap from school to come there and work on something full time. They took me under their wing, and I basically worked with these fund managers building various tools for them. I built chatbots. I built a CRM automation tool. 

I think the biggest headache that I saw was that their lawyers were just not very responsive to them, and the entire fund formation process was just something that was super expensive and tedious for them to just execute on. I decided to sort of take a swing at that. In that process and in learning more about fund formation, specifically for an emerging manager—someone that’s trying to raise, let’s say, under $150 million—large law firms, your AMLAW firms, are not necessarily geared towards these emerging managers. 

Usually, it’s a very high-margin service for the law firm. Usually, it’s a very relationship-heavy sort of service that they provide. For an emerging manager that just wants to get their fund created and get them out the door so they can start raising capital, it’s sort of a little bit of an inefficient process for them—actually highly inefficient. Oftentimes, some managers would wait six months to get their fund created. It’s really sad if a fund manager’s asset management career ends before it really starts. And that was something that I’d seen constantly amongst these emerging managers, and so I sort of decided to do something about it.

In understanding why I chose this emerging manager sort of group to focus on, I really saw that even amongst my own peer group, there’s tons of people starting funds. In fact, Roland was someone that I was speaking to who started a fund recently. There are other people in my network—they’re starting funds. It’s really this long tail that’s emerging where even at fund launch, I met numerous fund managers that were doing really niche things or had new strategies. 

For instance, one was starting a self-storage strategy fund, another one was starting a Pokémon card investment sort of strategy fund. There are all these various strategies that a fund structure can support because the fundamental sort of insight that I had was that a fund is simply just a vehicle to pool capital and do something productive with it. If you have a strategy or if you have some sort of edge, then you could potentially go out into the market, raise capital, and then scale whatever that strategy is.

In understanding what actually fund formation is, fundamentally it’s three documents. 

It’s your LPA documents, a limited partnership agreement. It’s your PPM, which usually outlines your risk factors for your fund strategy, and your subscription document, which is how investors essentially onboard onto your fund. That’s how you get investors to come onto your fund or to invest. Usually, these documents are 160-plus pages long. They’re pretty long documents that previously, pre-LLMs, law firms would use something like Contract Express or try to use various tools, but it just wouldn’t work because these documents were far too long. 

Law firms would actually have their entire fund’s practice be manual. So you’d have associates just write these 60-page documents or use templates and essentially do the full drafting process by hand. And what I realized is that really, looking at numerous fund documents over the past years, funds really break down into a small set of recurring terms. As you can see here, I said fees, carry, jurisdiction, fund size, entity structure, and so on. Really, what the work in practice is, is customizing these terms within those templates to create a full set of fund documents. That is essentially what our platform does—it’s a vertical sort of AI platform, a workspace for fund formation in which you can create a term sheet, assign that term sheet to a client so a client can fill it out, and then use that term sheet to then generate your LPA and subscription documents, so your larger fund document set. From there, you can use that into a chatbot that’s enriched with more research so that you can have a sort of chat interactive session with your documents. And there are sort of more tools that I’ll show off right now, actually, instead of describing it.

Right here is the current fund formation workspace. Rght here, I’ll go to a new deal and I’ll create a new client. Right, also, you know, talk about what is… on one… and I’ll create an empty client workspace. So now that I’ve created a workspace for myself, I’ll go into my documents tab—all the documents I’ve been working on recently. And I’ll click into a term sheet that I want to use. This is the term sheet. One thing to note here is that we’ve built a native docs editor into this workspace. That’s something that’s usually extremely hard. DocX is the formatting sort of underpinning for Microsoft Word. Any lawyer that drafts documents or uses documents needs a DocX native editor to fundamentally do their work. We spent a lot of the last 18 months really perfecting the DocX editor and making sure that documents can get imported into it, retain formatting, get exported out, and retain the formatting. Sounds simple—quite hard to do in practice.

Right here, you can see a term sheet. It has all these placeholders or blanks in it. What I’ll do from here is actually generate questions based off of those blanks so that I can create a questionnaire and assign it to a client. Right here, I’ll say a questionnaire and generate this. Over in the questionnaire tab, this is one that I’ve worked on before, and for the sake of time, I’ll just show you what that looks like. So here, the questionnaire is created. I can hit share, and I get a link. This is the interface that the client will see—it’ll be like a form view where I can answer the questions one by one. Yeah, 80 percent, etc., and when I’m done, I just hit submit. 

Now my job as the client is over, so I can go back to the dashboard for the attorney, see that the answers from the client are all in the right places. If there’s something that I need to change, I can do that, and if everything looks good, I can hit submit and generate. Nw it will generate a complete term sheet. Well, it should… it should be a complete term sheet. I don’t know what’s going on. This is the humor with these live demos. But let me just show you what the term sheet it should generate looks like.

Okay, so this is sort of a term sheet that would get generated from it with all the insertions in the right place. From here, with this term sheet that’s been generated, I can go back to my workspace and say, you know, create now an LPA and a subscription document based off of this term sheet. So let’s say this one looks pretty good to me, and so I hit generate. Now it’s essentially extracting all those terms and creating an entire LPA and subscription document. And the way it’s doing that is we have sort of a set of LPAs that we have in the back end that we use as sort of golden reference documents. It essentially uses that as well as the terms in the term sheet to create this full document set for the LPA and the subscription document.

So we’ll give it maybe a few seconds. While that’s going, I can just show you what it should look like, which is the document that’s generated will look something like this. It’ll be a full LPA, usually 60 pages, with—this is an insertion that was made. I just ran this entire sort of flow right before the call. I didn’t really anticipate that it would start working on the call, but I guess you’ve got to anticipate that. Well yeah, this is sort of a full 60-page LPA that was generated. You can see that all these insertions are made. Everything looks pretty good. I’ll show you the subscription document as well. 

It’s essentially from term sheet to complete document set within a matter of minutes. And usually, if you’re a private funds group, the time it takes to go from a term sheet or not even having a clear idea of what the terms of a fund are to a complete document set is months. It can be. In terms of drafting time, it’s like 20-30 hours. And if you’re getting billed at a partner rate, that’s pretty expensive. And so fundamentally, we’ve abstracted and compressed the timelines to be far, far shorter than what it is today.

Now that these fund documents are created, I’ll just add them to my workspace here. And I can go to the Co-Pilot tab here, add it as context—one of these documents, let’s say the term sheet—add some sources. Let’s say the market standards, SEC, the web, some state codes. I’ll say something like, “Hey, are there glaring errors or issues in my term sheet I should watch out for?” And now what it’s doing is it’s essentially making web searches to all these various whitelisted sources that I’ve previously bookmarked, and it’s finding the relevant sources and then using our document as context to provide a sort of real informed sort of feedback for the document. And this is a back-and-forth conversation that you can continue to have. Aas you can see, there are really specific issues that it’s highlighting. It provides some citations for me as well. I can click on one of these—this is a pretty in-depth LPA sort of guide that is what a professional fund attorney would use as a reference document. These are the recommendations. I can sort of keep going, or another sort of visualization tool or tool that… well yeah, maybe, maybe folks have some questions, you know, anything specific you still want it to show, or otherwise, I’m sure…

Yeah, just… yeah, maybe this is just a visualization tab here, which is the fund that we’ve just created. These key terms are the main sort of backbone of a fund, and so you can visualize it right here. And if I wanted to change one of the key terms, let’s say from 80 to 50 million, it updates those things across the document. So this is just a really easy way to change terms in your fund without having to sort of go into the document, search for it, and make drafting-related changes. You could sort of just use this interface as a shortcut.

Another visualization we have here is an entity structure visualization. I find it pretty helpful considering that funds are complex structures, and it’s hard to understand what goes into what and how carry works or how various parts of funds work. And so this is sort of just a visualization to help fund attorneys understand exactly what those documents actually mean in practice. But that’s the sort of full workflow. It’s really an end-to-end vertical platform for funds. And that’s it.

Roland Vogl:

That’s pretty awesome. So does this help practicing fund formation attorneys with their work just to be more efficient, or is this something that the fund managers will use themselves instead of using an attorney?

Saketh Kesiraju:

So it’s really for the attorney. But how we… so we sort of operate as a full-stack sort of fund formation business as well as we sell this platform to other law firms as well. So if we have a fund manager that’s in our network and wants to come to us, then we will essentially be sort of the full-stack solution for that manager. But that’s where really our vision is. However, for now, we’ve been working with sort of pilot law firms to just make our workflows a lot stronger and to just validate them in the real world as well.

Roland vogl:

So you’re seeding the system with whatever the templates are from the law firm, and those templates, they have the different variations of specific clauses, right? Then from those templates, your system can create a questionnaire that will then help you sort of create the custom documents. Correct?

Saketh Kesiraju:

Correct, yeah. I think that’s most helpful for the term sheet, right, because that’s probably the easiest one-to-one placeholder replacement workflow. But for your larger LPA and subscription documents, it’s really, you know, there are clause-specific replacements that need to be done. It’s not just term-for-term replacements. And so that’s where it runs sort of a longer process to do this sort of document creation workflow, which I skipped over earlier just to save some time. But yeah, that is sort of a more robust workflow.

Roland Vogl:

That’s a cool feature to be able to run those documents by some external benchmarks, for example. How do you know what kind of data are you using to enable that feature?

Saketh Kesiraju:

Yeah, so market terms are really from whitelisted sources that I’ve run into or fund attorneys have told me that they use as reference points. So the LPA guideline for the Institutional Limited Partners Association—they essentially set standards. They put out templates, they put out various sort of private market data and information regarding private funds. And so that’s the number one source that we go to. And then there’s also just web search. So we also scrape the entire web, essentially, for private funds or market-related terms. And then we try to figure out if that’s a credible source and then whitelist it and see if… yeah, essentially that is in line with what we hear from our attorneys as well.

Yeah. So in terms of privacy, there’s no reinforcement learning happening to ingest the content. In fact, the entire system runs locally. So you can think of it as like Claude Code fine-tuned specifically for… or like Claude for Work, something that you might have seen, fine-tuned specifically for private funds, in that the entire system is just local, so it’s using your local files and so on.

Roland Vogl:

So Cristiana and Juan are asking, can you get capital from other jurisdictions for entities outside the US?

Saketh Kesiraju:

Yeah, you can. In fact, one firm that we were working with was actually a Canadian GPU stakes firm. And for them, their limited partnership agreement had a special feeder vehicle as well that was necessary for them to essentially raise capital from outside US investors. And so yeah, the actual structuring of these fund vehicles is obviously dependent on an actual attorney advising you on this. But yeah, that is all doable on the platform.

Roland Vogl:

Thank you so much for the updates. It’s exciting. But I know you have to hop to your next event already, so really appreciate you joining us. It’s very cool. And yeah, good luck with all the next steps.

Saketh Kesiraju:

Thank you. It was a real pleasure to be here and speak to you all. So thank you for having me, and have a great day today.

Saketh Kesiraju can be reached at saketh@tryswiftlaw.com

AI-powered chatbots present the same structure. In 2023, the National Eating Disorders Association replaced its human helpline staff with an AI chatbot called Tessa. The interface was polished. The responses arrived in fluid, reassuring prose. Within days, Tessa was dispensing weight-loss tips to people suffering from eating disorders. The organization shut her down. But the damage had already begun compounding. Tessa was not malfunctioning. She was functioning, bereft of constraint.

Absent adequate testing and oversight infrastructure, the rush to deploy AI-powered chatbots produces systems structurally incapable of meeting the obligations their interfaces promise. Some of those promises are explicit. Most are implicit, conveyed by the conversational fluency itself. This carries costs as well as benefits. Slowing deployment delays improvements and (potentially) degrades competitive edge. But the alternative is indefensible.

Rushed chatbot deployment presents two distinct problems. The first is an appearance problem. Chatbots that have not been adequately tested look indistinguishable from chatbots that have. Conversational fluency mimics competence so convincingly that users extend trust the system has not earned. This is the cardboard cockpit. Every dial, every switch, every instrument panel is in place. It looks like the real thing, but that’s all. It does nothing. The second is a scaling problem. Chatbots that do perform well become more dangerous as adoption grows, because each increment of user trust widens the blast radius of any failure the system has not been tested against. And that is the apprentice’s broom. It works and that is precisely what makes it dangerous. Both problems trace to the same root cause, which is deployment that outpaces evaluation.

The Analytical Prism

I want to examine this chatbot phenomenon through my AI Life Cycle Core Principles (AILCCP) framework. The AILCCP provides 37 principles for AI system assessment, mapped to life cycle phases, controls, and standards from NIST, ISO, and IEEE. It applies regardless of system architecture. In this context, it can help organizations build trustworthy chatbots, from informed consent to safety guardrails to demographic monitoring. Now, to be clear, this is just a sampling of what the AILCCP can be used for and I’m intentionally keeping it bounded.

The AILCCP framework defines principles such as Safety, Transparency, and Fairness with precise scope, specific controls, and designated life cycle activation points, and yes, the capitalization is intentional. These capitalized terms carry the full weight of their framework definitions. So, when this note uses the same words in lowercase, such as “safety” or “fairness,” that is also intentional and refers to the ordinary English definition. Treat capitalization as a signal that the framework’s specific machinery is being invoked.

The Fluency Trap

The AILCCP framework identifies Truth as a distinct, separate principle from Accuracy. This is because an AI system can be accurate in aggregate while generating specific outputs that are false. Chatbots present a uniquely dangerous variant of this problem. Their outputs arrive in grammatically perfect, contextually appropriate prose. The very fluency that makes them useful also makes their failures invisible. This is the appearance problem at its sharpest.

The Fidelity principle addresses precisely this kind of invisible failure. Fidelity requires that system outputs remain aligned with stated purpose and training objectives. A chatbot deployed for medical triage that generates plausible but incorrect diagnoses fails Fidelity in the most dangerous way possible. The output evades casual inspection. It looks right. It sounds right. It is wrong. In a triage context, people get sick or die.

Rushed deployment exacerbates this problem because it compresses the testing window where Fidelity failures would otherwise surface. The AILCCP framework designates a distinct life cycle phase, Evaluation & Red Teaming, in which teams probe a system’s performance, safety, robustness, and fairness before release. The phase exists because standard validation catches expected failures while adversarial testing catches unexpected ones. A red team simulating a distressed user who phrases symptoms ambiguously might discover that the chatbot defaults to cheerful reassurance rather than appropriate caution. That discovery, made before deployment, is an engineering insight. Made after deployment, it is an incident report. Truncate the phase, and those failure modes find users instead of testers.

The Consent Illusion

The AILCCP framework’s Consent principle requires something more demanding than a clicked checkbox. It requires that consent interfaces mandate active acknowledgment of operational realities material to informed choice. This includes whether the system generates outputs probabilistically rather than retrieving from fixed sources, and whether outputs may contain confidently presented false information.

My sense is that most deployed chatbots fail this standard. They present conversational interfaces that, by their very design, suggest a competence and reliability the underlying system may not possess. Users interacting with a fluent chatbot form mental models based on human conversation, where fluency generally correlates with knowledge. The AILCCP framework recognizes this gap explicitly. Consent obtained from a user whose understanding of system operation is categorically incorrect does not satisfy the principle even where disclosure was formally complete.

The Accountability Vacuum

When a rushed chatbot produces harmful output, accountability becomes diffuse in ways the AILCCP framework anticipates. The Accountability principle warns that deficiency arises where ownership is diffuse, evidence is not preserved, and redress is undefined. Rushed deployment typically means incomplete logging, absent audit trails, and unclear lines of responsibility between developer and deployer.

The FTC has demonstrated willingness to act in this space. Its enforcement action against DoNotPay targeted deceptive claims about an AI system’s legal capabilities. Evolv Technologies faced scrutiny for misleading marketing of AI security screening. These actions share a common thread. Organizations deployed AI systems whose marketed capabilities exceeded their tested performance. The gap between promise and reality was the product of insufficient evaluation, not insufficient technology.

The Dialectical Reality

Slower deployment means delayed access. AI chatbots can and do provide genuine value, particularly for populations underserved by existing systems. A well-designed medical chatbot could extend the reach of overburdened health systems. A well-designed legal chatbot could democratize access to legal information.

But the qualifier matters. “Well-designed” means tested. It means red-teamed. It means constrained by principles like Reliability, which requires that continuous validation ensures alignment between marketing claims and actual performance. It means constrained by Safety, which requires real-time monitoring and the ability to return to safe operation states. It means constrained by Transparency, which requires that marketing claims match terms and capabilities, preventing deceptive gaps between what a system promises and what it delivers.

None of these requirements are exotic. They are the ordinary discipline of building systems that work as advertised. The AILCCP framework maps them to specific life cycle phases, specific controls, and specific evidence artifacts. The infrastructure exists. The question is whether organizations will invest in it before deployment rather than after harm.

Conclusion

The cardboard cockpit fails not because it looks wrong, but because it looks right. The apprentice’s broom fails not because it stops working, but because it never stops. These are not the same problem and conflating them leads to incomplete solutions. A system that merely appears competent needs better testing. A system that genuinely performs well but scales beyond its guardrails needs better Governance. Addressing only the appearance problem leaves the scaling problem untouched, and vice versa.

Organizations deploying AI chatbots should treat the AILCCP framework’s phases not as bureaucratic obstacles but as engineering necessities. Evaluation & Red Teaming exists because it surfaces failures before users encounter them. Pre-Deployment Review exists because review gates prevent premature release. Operations & Monitoring exists because a system that passed every test at launch can still drift into harm at scale. These phases take time. That time is not wasted. It is the difference between a cockpit and its cardboard replica, between a sorcerer and his apprentice.

Ricky Bobby’s father in Talladega Nights offered advice that drove an entire career of reckless behavior. “If you’re not first, you’re last.” (He later admitted he was drunk when he said it and that it made no sense.) The AI industry’s version of this “wisdom,” that speed to market is the only competitive variable worth measuring, deserves similar correction. Being first means nothing if the product harms the people it purports to serve.

However, for a General Counsel or Chief Legal Officer (CLO) to believe this surface-level volatility is the new normal would be a strategic error. Beneath the noise, a decisive legal signal has emerged: the center of gravity for climate governance is shifting rapidly from aspirational narratives to auditable, defensible data and the legal and executive teams need to adjust accordingly.

For years, the climate disclosure portfolio was largely the domain of the Chief Sustainability Officer (CSO), and reporting often came in the form of glossy reports designed for stakeholders and consumers. That era is drawing to a close. As an increasing number of disclosure related legislative and regulatory proposals are becoming enforceable laws, the ownership of climate emissions data is migrating to the legal department. It is no longer solely a matter of corporate social responsibility; it is a matter of governance, risk, and compliance (GRC). For corporate leadership, the question is no longer what a company wants to say about its climate and sustainability ambitions, but whether it can provide data to demonstrate its actions and whether that proof can withstand the scrutiny of a regulator, a litigator, or a customs official.

Global Compliance Regimes Emerge

As the regulatory landscape hardens, three distinct categories of regulation illustrate the definitive shift toward mandatory, data-centric reporting:

1. Corporate Disclosure: Quantitative Rigor and Global Baselines

Across jurisdictions, the focus is shifting from voluntary ESG reporting to mandatory, auditable disclosure regimes that function like financial reporting.

• • California (SB 253): While climate-risk reporting under SB 261 remains subject to ongoing litigation, the quantitative reporting regime of SB 253 is proceeding apace. Large companies ($1B total revenue) doing business in California must disclose Scope 1 and 2 emissions starting in 2026, with Scope 3 following in 2027. The California Air Resources Board (CARB) is establishing a program where quantifiable facts are the only defensible basis for compliance.
  • Europe (CSRD): The Corporate Sustainability Reporting Directive requires assured, comparable sustainability data. Even before non-EU parents file their first reports for financial years starting on or after January 1, 2028, their large EU subsidiaries must report in 2026 for fiscal year 2025 and customers already in scope may already be  demanding data to meet their own compliance requirements.
  • Global Baseline (ISSB): The International Sustainability Standards Board (IFRS S1 & S2) is establishing a durable, jurisdiction-agnostic framework for disclosure. Major economies – including the UK, Australia, and Brazil – are moving toward ISSB-aligned mandatory reporting, meaning that a standardized, “financial-grade” emissions inventory is becoming a prerequisite for global market access.

2. Trade and Border Controls: Reporting plus Environmental Levies

Key imported products are now the subject of mandatory disclosures and accompanying environmental levies in the EU, with a number of other countries considering similar regimes. 

• The EU’s Carbon Border Adjustment Mechanism (CBAM): Importers of industrial goods (cement, steel, aluminum, etc.) into the EU must quantify embedded emissions and reconcile them with EU-ETS-priced certificates. This effectively shifts emissions accounting from the sustainability office to border operations and customs compliance, where data gaps can result in goods being held at the port of entry. The UK has a similar regime that will become effective in 2027.

3. Supply Chain and Circularity: The Liability of Provenance

Beyond emissions, regulators are targeting the social and physical lifecycle of products, converting voluntary “responsible sourcing” into mandatory legal liability.

• CSDDD & National Standards: Beginning in 2028, the EU’s largest companies and non-EU companies that meet net turnover thresholds will be subject to The EU’s Corporate Sustainability Due Diligence Directive (CSDDD). The CSDDD expands upon national statutes like France’s Duty of Vigilance and Germany’s Supply Chain Due Diligence Act. While not uniform these laws require a range of corporate action including transition plan adoption, due diligence processes with suppliers, and disclosure of adverse impacts and actions taken.  Some of these laws establish direct civil liability for parent companies, holding them accountable for damages resulting from a failure to prevent human rights and environmental abuses within their global value chains. 
• Extended Producer Responsibility (EPR): Circular economy laws are converting physical waste into digital data obligations. Producers must now track data on product material composition, recyclability, and waste streams to calculate fees relating to their environmental impact, and penalizing companies for failing to track and produce data.

The Fiduciary Imperative: A Defensible Legal Architecture

For the in-house counsel, a “wait and see” approach to climate disclosure is imprudent given the time required to build necessary infrastructure to respond to current and future regulatory mandates. The CLO should understand their core responsibility for establishing internal controls over climate reporting, and moving the organization away from ad-hoc processes toward regulatory-grade record keeping. The legal leadership team must drive the implementation of systems that track data provenance – who entered it, when, and why – to prepare for the inevitable arrival of “limited” and eventually “reasonable” assurance audits.

This is ultimately a matter of fiduciary duty. To navigate this, counsel must distinguish between the rigid standards required for audited quantitative data and the cautionary language required for narrative content, ensuring the former validates the latter. By treating emissions and sustainability reporting with the same consequence as financial reporting including ensuring necessary controls, assurance, and contract-ready evidence, legal departments not only ensure compliance but also build a necessary shield against future legal and political challenges.

5 Things CLOs Should Be Doing to Ensure Compliance and Manage Risk

For the CLO, the evolution of mandatory reporting requires a pivot from reactive oversight to a proactive orchestration of compliance architectures that manage exposure to long-term liabilities. Undertaking the following five strategic actions will create a strong foundation for successful compliance and risk management. 

1. Mandate the Implementation of a Defensible System of Record. The CLO must treat emissions data not as an operational metric, but as a material class of record subject to Internal Controls over Sustainability Reporting (ICSR). This requires establishing a rigorous data internal audit trail that tracks the provenance of every data point including who entered it, when it was changed, and why. By enforcing version control and audit trails similar to those used in financial reporting, the legal department can create an evidentiary record capable of defending the corporation against both regulatory, legal and reputational challenges. 
2. Transition from “Firewalling” to Strategic Alignment. Legal counsel must pivot from the traditional strategy of strictly separating historical facts from forward-looking plans to a new standard of rigorous consistency. Emerging regulations and anti-greenwashing case law now penalize the gap between a company’s “hard numbers” (audited emissions) and its “soft narratives” (e.g., Net Zero targets). Rather than insulating these streams, the CLO must ensure they are connected: validating that current capital expenditure and emissions data actively substantiate the transition story. This alignment mitigates the risk that a discrepancy between promise and performance becomes actionable evidence of misleading conduct.
3. Transform Voluntary Supply Chain Data into Binding Contractual Obligations. To address the enforcement gap created by extraterritorial regimes like the EU’s CSRD, the CLO must systematically update supplier agreements to mandate the provision of supplier data needed by the company to meet its own compliance obligations. This involves shifting from voluntary data requests to binding clauses that include audit cooperation rights and indemnification for data inaccuracies. This ensures the company possesses the legal leverage necessary to obtain the data required for its own compliance, effectively transferring regulatory risk upstream to the source of the emissions.
4. Elevate Climate Reporting to a Core Governance Priority. The corporate counsel must ensure the Board of Directors exercises its duty of oversight regarding material regulatory risks, including potential market access denials or significant penalties. Best practices should include establishing quarterly dashboards that track readiness for specific reporting deadlines and integrating climate data integrity as a standard component of M&A due diligence. By underscoring that these issues are fiduciary duties, the CLO protects the directors and officers from derivative claims related to oversight failures.

5. Orchestrate Unified Governance Across Legal, Finance, and Sustainability. The CLO must act as the cross-functional orchestrator – engaging with the CFO, CSO and even CPO – to eliminate data silos that can create exposure to liability, specifically the risk of material omission where internal risk data contradicts public sustainability narratives. By applying financial-grade internal controls to technical emissions data, the legal department ensures consistency across all public disclosures and regulatory filings. Such a unified governance structure prevents discrepancies that often serve as the factual basis for greenwashing litigation.

Catherine Atkin and Michael Schmitz are the Co-Chairs of the Stanford CodeX Climate Data Policy Initiative (CDPI).

Cette conférence explore les mutations profondes de la pédagogie juridique à l’ère digitale. Comment le numérique transforme-t-il l’apprentissage du droit ? Quelles compétences les juristes de demain doivent-ils acquérir pour naviguer dans un environnement technologique complexe ?

Venez écouter trois panélistes d’exception qui partageront leurs réflexions sur ce sujet :

• Annie Rochette : Experte en pédagogie juridique et en approche par compétences, ancienne directrice de l’École du Barreau de la Colombie-Britannique et professeure chevronnée, elle a également agi en tant que présidente de l’Association Canadienne des professeur(e)s de droit, rédactrice francophone de la Revue femmes et droit et rédactrice en chef (et fondatrice) de la Revue l’enseignement du droit au Canada. Elle s’intéresse à la formation des juristes sur tout le continuum, se penchant sur les questions concernant la pédagogie, la compétence professionnelle, l’approche par compétences, et plus précisément sur certaines compétences comme la pratique réflexive et le développement d’une identité professionnelle.

• Alicia Mâzouz : Docteure en droit, maîtresse de conférences et responsable pédagogique de la Licence Droit & Culture Juridique (Issy-les-Moulineaux), elle a exercé ses activités de recherche et d’enseignement au sein de plusieurs universités (Université Paris I Panthéon-Sorbonne, Université de Cergy-Pontoise, Université Paris-Est Marne-la-Vallée) avant d’intégrer la Faculté libre de Droit en qualité de membre permanent. Ses travaux sont marqués par l’étude du lien entre le corps humain et le droit, mais aussi par l’intérêt pour le droit civil et les sources du droit ainsi que pour les difficultés rencontrées par le droit lorsqu’il se trouve confronté aux nouvelles technologies.

• Luka Sanchez : Doctorant en droit à l’Université de Montréal, ses travaux s’intéressent à l’enseignement universitaire du droit.

The Controls table is one of 13 tables that comprise the AI Life Cycle Core Principles (AILCCP) framework. (A public-facing version of the AILCCP is available here.)

The Controls table (currently) contains 48 actionable controls —specific mechanisms, policies, and technical safeguards that translate abstract AI principles into concrete, implementable measures. Each control is classified by domain, function, and principle alignment, enabling organizations to systematically operationalize responsible AI governance across the entire system lifecycle. It transforms the AILCCP from a conceptual framework into an operational toolkit.

Note: The number of controls expands as my research advances and evolves.

Structure at a Glance

Five Practical Use Cases

An Operational Toolkit

The Controls table transforms the AILCCP from a conceptual framework into an operational toolkit.

Organizations can:

• Trace compliance from high-level principles down to specific controls and evidence artifacts

• Customize governance by selecting controls appropriate to their risk profile and regulatory environment

• Demonstrate accountability through documented control rationales and principle alignments

• Scale responsibly by applying proportionate controls as AI capabilities evolve

This structured approach ensures that responsible AI is not just aspirational—it is auditable, defensible, and actionable.

La possibilité de se procurer en quelques clics des milliers d’articles dans le confort de son foyer vaut aux achats en ligne leur popularité croissante. Une habitude cimentée pendant le confinement pandémique. Selon l’enquête NETendances 2024 sur le commerce en ligne de l’Académie de la transformation numérique de l’Université Laval, environ trois adultes québécois sur quatre (74 %) font de tels achats. C’est Amazon qui récolte la part du lion : près de la moitié des répondants ont affirmé y avoir effectué au moins 75 % de leurs dépenses en ligne. Le géant américain est talonné par des entreprises chinoises, comme Temu ou Shein, qui allient de très bas prix à un marketing « particulièrement agressif », indique le document.

Pour en savoir +

But will you or any other juror make the “right” call when faced with information they may not fully understand?

Actus Reus and Mens Rea

Before diving into brain scans, it’s important to understand two fundamental principles that typically determine criminal liability in the United States: actus reus and mens rea.

Actus reus (Latin for “guilty act”) refers to the physical act of committing a crime. This is usually the more straightforward element to prove, because it is typically based on objective action, such as pulling the trigger, taking property, or striking the victim.[1]

Mens rea (Latin for “guilty mind”) refers to the mental state or intent behind the act. This is the harder element to prove. It often looks at considerations such as whether the defendant intended to act or whether it was an unfortunate accident, and whether they knew or should have known their conduct was wrong.[2] Different crimes require different levels of intent. Murder typically requires intent to kill, while manslaughter might involve recklessness rather than specific intent.[3]

Both elements must typically be present for someone to be found guilty of a crime. The state can’t convict someone of first-degree murder simply because they caused a death; the prosecution must also adequately prove they had the requisite mental state.

Attorneys have started using neuroimaging as evidence to argue that abnormal brain scans demonstrate that the killer lacked the mental capacity to form this necessary intent, or that they couldn’t distinguish right from wrong due to mental illness. In essence, lawyers are trying to use neuroscience to prove their client lacked mens rea for a given criminal charge.[4]

How Courts Decide Whether to Admit Brain Scans as Evidence

When a party wants to introduce scientific or technical evidence like brain scans, courts don’t simply accept it at face value. After all, jurors likely don’t have the expertise needed to make credibility determinations when it comes to neuroimaging. In 1993, the U.S. Supreme Court decided the criteria for expert testimony in a case called Daubert v Merrell Dow Pharmaceuticals, Inc.. The Daubert Court noted that experts must testify to scientific knowledge that will assist a jury to better understand the facts.[5] If so, the scientific evidence must be reliable, as shown through testing, peer review and publication, potential rate of error, and general acceptance in the relevant scientific community.[6]

The Daubert court also referred to Federal Rule of Evidence 702 and 403.[7] Federal Rule of Evidence 702 requires that expert testimony be helpful to the jury, based on sufficient facts and data, properly tested with scientific methods, and appropriately applied to the case at hand.[8] Federal Rule of Evidence 403 allows evidence to be thrown out if it might mislead the jury, among other criteria such as wasting the jury’s time.[9]

In 2012, the Sixth Circuit used the Daubert criteria in a case called United States v. Semrau. In Semrau, the defendant was a doctor who was charged with healthcare fraud and attempted to introduce functional Magnetic Resonance Image (fMRI) test results showing he was “generally truthful” when claiming he tried to follow proper billing practices in good faith.[10] An fMRI is a technique for measuring changes in blood oxygenation and flow in the brain, which occurs in response to neural activity.[11] However, the signal is nonspecific, since it’s an average of millions of cells.[12] This means that it cannot easily differentiate between specific lobes of the brain, nor can it tell us exactly what a person is thinking.

Ultimately, the Semrau court decided to exclude this evidence due to reliability problems. One of the reasons was because the expert witness noted that fMRI lie detection had “a huge false positive problem,” where truth-tellers were incorrectly identified as liars 60-70% of the time.[13] A study in 2009 asked participants to commit a “mock crime” or stealing and damaging CDs, and reported that fMRI may have high sensitivity, but low specificity.[14] The study noted that this result meant an fMRI test may be helpful to ‘‘rule out’’ an innocent suspect, but not very helpful in ‘‘ruling in’’ a guilty suspect.[15]

Brain Scans to Prove Lack of Criminal Responsibility

In Commonwealth v. Chism, decided in 2025 by the Massachusetts Supreme Judicial Court, the defense of a 14-year-old brought in a structural MRI (sMRI) brain scan containing detailed images of his brain’s anatomy.[16] The scans show volumetric abnormalities (differences in the size of certain brain structures) consistent with schizophrenia. When the defendant committed first-degree murder, aggravated rape, and armed robbery, his lawyer used brain scan evidence to argue that the 14-year-old couldn’t understand right from wrong due to mental illness.[17]

The court relied on a 2014 multidisciplinary consensus report from Emory University, which concluded that “the practice of performing imaging studies on a defendant in order to shed light on brain function or state of mind at the time of a prior criminal act is problematic.”[18] The key reason is because a brain scans taken months or years after a crime occurred cannot tell us what was happening in the defendant’s brain at the moment they committed the criminal act.[19] The court also noted methodological issues because the control group (the “normal” brains the defendant’s scans were compared against) wasn’t age-matched to the 14-year-old defendant, making the comparison scientifically questionable.[20]

What This Means for the Future

Does this mean neuroimaging evidence will never be admissible in criminal cases? Not necessarily. Courts seem to have left open the possibility that as science advances and gains broader acceptance, such evidence might meet admissibility standards in the future.

However, the timing issue remains. Criminal law asks whether a defendant had a particular mental state at a specific moment in the past, while neuroimaging shows us what a brain looks like now or how it responds to stimuli in a current testing situation. Bridging that temporal gap requires scientific advances that don’t yet exist.

As neuroscience continues to advance, courts will likely continue to grapple with how and whether brain imaging should influence criminal responsibility. As legal scholar Francis Shen notes, the goal is not to wait for “magical tools,” but to adopt an entrepreneurial “What now?” mentality.[21] Perhaps the way forward is to define clear expert witness guidelines for what type of neuroimaging can be used in the courtroom, or to create better jury instructions that lead to a rightfully skeptical jury.

Understanding these issues will affect how we balance scientific advancement with legal protections, how we determine criminal responsibility, and ultimately, how we define what it means to have a “guilty mind” in an age where we can peer inside the brain itself.

References

[1] Uri Maoz & Gideon Yaffe, What Does Recent Neuroscience Tell Us About Criminal Responsibility?, 3 J.L. & Biosciences 120, 122 (2015).

[2] Id. at 122–23.

[3] Id. at 130.

[4] Neal Feigenson, Brain Imaging and Courtroom Evidence: On the Admissibility and Persuasiveness of fMRI, 2 Int’l J. L. Context 233, 234 (2006).

[5] Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 588 (1993).

[6] Id. at 593–95.

[7] Daubert, 509 U.S. 594-95.

[8] Fed. R. Evid. 702.

[9] Fed. R. Evid. 403.

[10] United States v. Semrau, 693 F.3d 510, 515 (2012).

[11] Nikos K. Logothetis, What We Can Do and What We Cannot Do With fMRI, 453 Nature 869, 869 (2008).

[12] Id. at 876.

[13] Semrau, 693 F.3d 518.

[14] F. Andrew Kozel et al., Functional MRI Detection of Deception After Committing a Mock Sabotage Crime, 54 J. Forensic Sci. 220, 228 (2009).

[15] Id.

[16] Commonwealth v. Chism, 495 Mass. 358, 360, 370–71 (2025).

[17] Id.

[18] Id. at 376–77.

[19] Id. at 376.

[20] Id.

[21] Francis X. Shen, Law and Neuroscience 2.0, 48 Ariz. St. L.J. 1043, 1085 (2016).

Derrière cette multiplicité de terminologies souvent employées, rarement expliquées, il importait de tenter de mettre de l’ordre mais aussi se questionner sur une manière de faire qui s’est imposée sans que l’on se questionne parfois sur les conséquences sociales, politiques, économiques. Le droit a changé ; et bien au-delà des usages et coutumes, expressions traditionnelles du domaine juridique, il nous importait de préciser ces concepts en émergence. Dans la tradition des dictionnaires, la conférence présentera ce travail collectif en cours d’élaboration.

AI systems access data. That access is governed. Each source is reviewed, scoped, and authorized. The governance frameworks that manage this process are mature and well understood. But AI systems do not merely access data. They combine it. And the combination produces something that no individual authorization addressed: inference. An AI that reads a calendar, an email account, and a CRM can derive a health condition that exists in none of those sources individually. The inference was never permitted. It was never prevented. It sits in a gap that current frameworks do not recognize. This essay introduces Context Stewardship, an interpretive lens within the AI Life Cycle Core Principles framework, to name that gap and propose controls for it. Context Stewardship reframes six existing governance principles around a single question: what risks emerge when authorized data sources are combined? It introduces Inference Boundary Mapping, a documented specification that defines what an AI system may and may not derive from combined context. The argument is that data access is not the right unit of governance for AI systems that synthesize across sources. Data combination is. Until frameworks reflect that distinction, organizations will continue to govern what AI can reach while ignoring what it can infer.

Between Access and Inference

An AI reads your calendar. That is authorized.

It reads your email. No problem. That is authorized too.

It queries your company’s CRM. Authorized, for certain fields.

Each access was scoped. Each was reviewed. Each was approved under data control frameworks designed to ensure that AI systems reach only the data they are permitted to reach.

But no one authorized what happens next. The AI combines what it found. A medical appointment on the calendar. Test results mentioned in an email thread. An insurance tier in the CRM. From these, it derives a health condition. No single source contained that information. No single authorization contemplated it. The inference was never permitted. It was simply never prevented.

This is the gap that current data governance frameworks do not see because they govern access. They do not govern synthesis. They ask whether the AI may reach a data source but not what becomes possible when authorized sources are combined.

This gap, between source-level authorization and synthesis-level inference, represents the most under addressed data governance risk in enterprise AI deployment today. Context Stewardship, which I explain in more detail below, is designed to effectively address it.

Context Stewardship is part of the AI Life Cycle Core Principles (AILCCP) framework. The AILCCP is a system of 37 principles organized across 10 strategic pillars, designed to guide organizations through the full life cycle of AI development, deployment, and decommissioning. Now in its third year of development, the AILCCP spans areas from oversight and accountability to reliability and robustness. (A public-facing version is available here.) It functions as both a compliance roadmap and a strategic tool for managing the legal, ethical, and reputational risks that accompany AI adoption. Context Stewardship is one of the latest additions to the framework. It is not a standalone principle, but an interpretive lens that cuts across six AILCCP principles (which appear in initial uppercase letter): Governance, Data Stewardship, Privacy, Consent, Security, and Resilience. It reframes each of these around a single question: what risks emerge from combining authorized data sources that none of those sources would present alone?

The Combinatorial Problem

The authorization pattern I described above is how most organizations operate. Each approval in that sequence was evaluated independently but no one asked what the combination would make possible.

This reflects a structural limitation in existing frameworks. They were designed for a world in which systems accessed data sources one at a time. They lack vocabulary for what happens when an AI agent moves across sources, synthesizing context as it goes. A system with access to three enterprise sources has a manageable set of possible inferences. A system with access to fifteen has a set that grows far faster than the number of sources, because every new source interacts with all the existing ones. No individual review of each source can meaningfully cover that space.

The health inference I opened with is simple by design. In practice, the inferences available from combined enterprise data are far more varied and far harder to anticipate. Consider a second example. An AI assistant authorized to access a company’s code repository notices that a senior engineer’s commit frequency has dropped. Individually, that data point means little. People take vacations. They shift to architecture work. They mentor junior staff. But the same AI also has access to the employee’s calendar, where it sees several midday blocks marked “personal.” And it has access to the internal wiki, where it can see that the engineer recently viewed pages on equity vesting schedules and the company’s non-compete policy. No single source signals anything. Combined, the AI infers departure risk. It was never asked to. No authorization contemplated this inference. But nothing prevented it either.

Of course, not every combination of data sources produces problematic inferences. Many combinations are benign or beneficial and mature organizations do attempt (at least on paper) cross-system risk analysis, tiering AI access by data sensitivity, decision impact, and safety vulnerabilities. Some security teams also treat AI agents as distinct entities with their own access credentials, applying zero-trust principles (the assumption that no entity is trusted by default, and every access request must be verified and scoped) to limit what each agent can reach. But these processes are emerging, they are not  standard, and even where they exist, they tend to operate at the access level rather than the inference level.

How Context Stewardship Reframes Existing Principles

Closing this gap requires reframing the questions those frameworks ask. Context Stewardship does this by cutting across six AILCCP principles and shifting each from source-level compliance to synthesis-level risk.

Three of these reframings carry the most weight. 

Privacy frameworks typically evaluate exposure at the source level. Context Stewardship treats aggregated context as an expanded area of vulnerability. Each additional source the AI can access opens new categories of inference that were unavailable from any individual source. The health inference from the opening illustrates this at a basic level. The departure risk example complicates it further: there, the privacy risk arises not from obviously sensitive data but from the combination of routine signals that no one would think to restrict individually. 

Consent mechanisms inform users about what each system collects, but they never make it clear that an AI drawing on multiple sources can derive things from their combination that no individual consent addressed. Consenting to an AI assistant that reads your calendar is one thing. Understanding that the same assistant cross-references your calendar with your email, your documents, and your CRM records is quite another.

Security controls enforce least-privilege access to individual resources. Context Stewardship introduces a parallel concept: least-context-necessary. An AI that has access to ten enterprise sources but only needs three for a given task should be constrained to those three. Not because the other seven are sensitive in isolation, but because their addition opens inferential possibilities that may be unnecessary and ungoverned.

The remaining three principles are also reframed. 

Data Stewardship shifts from data quality and provenance to scope authorization, asking (in relevant part) whether access was granted with awareness of how sources would combine. 

Resilience takes on new failure modes specific to synthesized environments: stale data from one source contradicting current data from another, compromised data from one source propagating through the AI’s reasoning, or temporal mismatches creating a distorted picture that the AI treats as coherent. 

And Governance at the organizational level must answer who bears responsibility for risks that arise from integration decisions, particularly when no single source owner anticipated them.

Inference Boundary Mapping

Among the controls accompanying Context Stewardship, one addresses what I believe is the most practically urgent problem: specifying what an AI system may derive from combined context.

The concept is straightforward. If an organization authorizes an AI to access both calendar data and email data, Inference Boundary Mapping asks what inferences from that combination are within scope and which are not. A scheduling optimization that combines calendar availability with email response patterns may be entirely appropriate. A health status inference derived from medical appointment entries and insurance correspondence is not.

In practice, Inference Boundary Mapping takes the form of a documented specification, developed before deployment and revisited periodically, that defines permissible inferences for each combination of data sources the AI can access. Scope is defined by use case: an AI authorized to assist with scheduling has a different inference boundary than one authorized to assist with workforce planning, even if both access the same underlying sources. The exercise requires a cross-functional team. Technical staff identify what inferences the AI is capable of drawing from combined sources. Legal and compliance staff determine which of those inferences fall within the purpose for which access was granted. The resulting specification becomes a governance artifact, auditable and enforceable, that sits between source-level authorization and system-level behavior.

But the harder cases are less obvious. Consider an AI with access to email, meeting transcripts, project management tools, and peer review records. It could construct a detailed performance profile by correlating communication patterns, meeting participation, task completion rates, and peer feedback. Each of these sources was authorized for the AI to help with workflow management. The performance profile that emerges from their combination was not. Yet unlike the health inference, the line here is blurry. Correlating task completion with meeting load to suggest workload redistribution seems helpful. Correlating email tone with peer review sentiment to predict a performance rating seems invasive and creepy. The underlying data is the same. The difference is in what the AI is permitted to derive from it, and that distinction exists nowhere in a source-level authorization framework. Inference Boundary Mapping makes that distinction explicit and assigns ownership to it.

None of this is easy. An AI system with access to fifteen data sources can draw inferences that no pre-deployment review will fully anticipate. Defining boundaries in advance requires judgment about what an AI might derive from data combinations, and that judgment will sometimes be wrong or incomplete. But the alternative, allowing AI systems to derive whatever their combined context permits, transfers risk from a design decision to an unexamined default. Organizations that make that transfer without acknowledging it may find themselves unable to explain, after the fact, why a particular inference was generated and who authorized it. That is a liability exposure, not merely a compliance gap.

Beyond Enterprise Data: A Structural Pattern

Everything I have described so far concerns enterprise data. But the same structural pattern appears wherever AI systems draw on multiple data sources to build an integrated picture of their environment.

AI research is also taking aim at systems that learn internal representations of how the world works, built from training data drawn from multiple sources. These learned representations drive predictions, and those predictions drive actions. The training data that shapes these representations raises the same questions that Context Stewardship asks of enterprise AI. Were the sources selected with awareness of how they would interact? Could the system draw inferences from combined training data that no one intended? When the system produces a flawed prediction, can responsibility be traced back to specific sources?

The pattern is consistent. Whenever multiple sources are synthesized, the whole exceeds what any individual source authorization contemplated. That is true whether the synthesis happens in an enterprise assistant combining calendar and email data or in a research system combining training datasets to build a model of its environment.

The Regulatory Gap

Current and proposed AI legislation does not adequately address combinatorial inference risk. Transparency mandates focus on disclosing what data a system accesses and how it processes that data. Impact assessments, which are increasingly required by state consumer privacy law, evaluate risks at the system level or the data source level. Neither is designed to capture risks that emerge specifically from synthesis across individually authorized sources.

This matters increasingly as AI systems become more deeply embedded in enterprise operations. The number of data sources they access will grow. The inferential possibilities will expand. Legislative frameworks that treat data access authorization as the primary safeguard will increasingly miss where the actual risk resides.

I do not think this requires entirely new legislation. In many cases, existing regulatory frameworks could accommodate combinatorial risk analysis if regulators recognized the category. What is missing is analytical vocabulary. Context Stewardship, and controls like Inference Boundary Mapping, are an attempt to provide it.

What This Means in Practice

For legal counsel advising on AI integration, the practical implication is this: reviewing AI data access authorizations one source at a time is necessary but insufficient. The review should also ask what becomes possible when authorized sources are combined, and whether the organization has mechanisms to govern those possibilities.

For policy makers, the implication is that transparency and access control mandates, while valuable, do not reach the synthesis layer. The combinatorial risks I described are present in every enterprise AI deployment that accesses more than one data source. They are not speculative.

For organizations deploying AI agents in enterprise environments, the question is concrete: when your AI system derives something from combined data that no individual authorization addressed, who is accountable? If the answer is unclear, the organization has a gap that no amount of source-level compliance will close.

Context Stewardship does not claim to solve every problem that enterprise AI creates. It identifies the specific layer where source-by-source authorization fails and proposes both an analytical framework and practical controls to address it. The AILCCP was designed to evolve as AI deployment reveals new categories of risk. Context Stewardship represents that evolution.

This is not a research prototype. StrongDM builds access management and security software. Pause on that. A team building security infrastructure has decided that human code review is an obstacle, not a safeguard. They are not alone. Dan Shapiro’s five-level taxonomy of AI-assisted programming, published weeks earlier, places this approach at “Level 5: The Dark Factory.” The term borrows from manufacturing, where robots work in unlit facilities because robots do not need to see.

I think this development is more consequential than it appears. It is not merely a story about productivity. It inverts how we assign responsibility for software behavior. Existing regulatory frameworks are not prepared for it.

The Inversion

StrongDM’s charter contains two rules: “Code must not be written by humans” and “Code must not be reviewed by humans.” Their CTO, Justin McCarthy, offers a benchmark: “If you haven’t spent at least $1,000 on tokens today per human engineer, your software factory has room for improvement.”

Why does this work at all? Consider the trajectory. In 2024, models like Claude 3.5 Sonnet and later updates substantially improved at coding tasks, especially when used in agentic workflows over long contexts. By late 2025, newer systems from Anthropic, OpenAI, and others made it routine for many engineers to rely on AI to draft and refactor large portions of production code, with human effort shifting toward architecture, safety, and integration review. By November 2025, newer models from Anthropic and OpenAI made AI-written code reliable enough that the question shifted from “can agents write code?” to “why are humans still writing code?”

This is a textbook example of what Ray Kurzweil calls the Law of Accelerating Returns (the observation that technological progress follows exponential curves, but humans consistently misjudge the pace because we instinctively extrapolate in linear fashion). The exponential curve here is not raw compute. It is model reliability on complex, multi-step tasks. Each generation of model compounds the gains of the last. The shift from human verification to machine-driven validation happened faster than almost anyone predicted, and it will keep accelerating.

But speed raises an alignment question that Stuart Russell has spent decades studying (his work on AI alignment focuses on the gap between what we tell machines to optimize and what we actually want). What are these agents trying to do? The answer is: pass the tests. Not “build good software.” Not “serve the user.” Pass the tests. That’s it. StrongDM learned this the hard way. Their agents wrote return true, which passes any test beautifully and does nothing useful.

How Do You Know It Works?

Instead of checking whether code passes or fails a fixed set of tests, StrongDM wrote detailed descriptions of how a real customer would actually use the software, step by step. They kept these descriptions hidden from the agents, so the agents could not simply memorize the answers. Then they asked a different question than traditional testing asks. Instead of “does it pass?”, they asked “if a real person used this software in all the ways a real person might, how often would it actually do what they needed?”

The AI Life Cycle Core Principles (AILCCP) framework’s Metrics principle warns against exactly this kind of substitution unless done carefully. (Note: AILCCP principles appear in initial uppercase.) The economist Charles Goodhart observed in 1975 that when a measure becomes a target, it ceases to be a good measure. Tell an agent to maximize a test score and it will maximize the test score, whether or not the underlying software actually works. StrongDM’s satisfaction metric is clever, but it uses AI-as-judge. This creates a circularity: the same class of technology that writes the code also decides whether the code works. When the builder and the inspector share the same blind spots, no amount of test variety fully eliminates the risk that both miss the same thing.

The Accuracy principle sharpens this. It requires that AI system performance match what developers and vendors claim, and that the system employ ongoing testing and self-correction. StrongDM does test continuously. But the tests are run by systems with the same limitations as the systems being tested. When a human writes a test, the human brings different assumptions, different mistakes, and different oversights than the person who wrote the code. That mismatch is what makes testing useful. When the same AI model writes the code and evaluates it, that mismatch shrinks.

This is Russell’s alignment problem. The agents are not trying to satisfy users. They are trying to score well on a test that is supposed to represent user satisfaction. Those are different things. A clever enough agent will find ways to ace the test without actually doing what users need. The “return true” episode was a crude version. Subtler versions will be harder to catch.

The Digital Twin Universe and the Economics of Impossible Things

The most creative element of StrongDM’s approach is what they call the Digital Twin Universe. They built working replicas of Okta, Jira, Slack, Google Docs, Google Drive, and Google Sheets, mimicking their interfaces, edge cases, and behaviors. Against these replicas, they run thousands of test scenarios per hour. No rate limits. No API costs. No risk of breaking real services.

McCarthy frames this as an economic inversion, and the evidence supports him. Building a faithful replica of a major SaaS product was always technically possible. It was never worth the cost. Engineers did not even propose it because they already knew the answer. Then the cost of writing software collapsed. What was unthinkable six months ago is now routine.

This is Kurzweil’s exponential logic again, applied to economics rather than capability. When a technology crosses a cost threshold, investments that were irrational yesterday become obvious today, and the unlocked capabilities cascade. The Digital Twin Universe is not just a testing technique. It is proof that the economics of software have changed in kind, not merely in degree. If you can clone Okta’s API in hours rather than months, the limit on software quality is no longer cost. It is imagination.

But the AILCCP framework’s Accountability principle asks a harder question. When software is “grown” rather than written, when replicas stand in for real services, and when quality is measured by probability rather than certainty, who is responsible for what comes out? The principle requires (among other things) that output be “traceable to an appropriate responsible party” and that there be “zero gap between AI system behavior and deployer’s liability.”

StrongDM’s architecture makes tracing difficult by design. No human reviewed the code that produced a given output. No human wrote the test that validated it. No human built the replica against which it was tested. The humans designed the system that designed the system. Existing legal frameworks assume someone, somewhere, looked at the work. Here, nobody did.

What Happens to the Engineers?

StrongDM’s team is three engineers who started in July 2025. By October, when Simon Willison visited, they already had working demos of the system that manages their coding agents, their Digital Twin Universe, and their satisfaction testing framework. Three people, three months.

That speed raises a question the AILCCP’s Workforce Compatible principle is designed to surface: does this technology augment human expertise, or does it replace it? StrongDM’s model does not augment software engineering as traditionally understood. It replaces it with something else. The humans in a Software Factory write specifications, design scenarios, and architect systems. They do not program. The skill of reading and writing code, the bedrock of software engineering for seventy years, becomes unnecessary. This is Shapiro’s Level 5, the “Dark Factory,” where the human role shifts entirely from building software to designing and monitoring the systems that build software. The lights are off because nobody needs to see.

The same principle asks a follow-on question: as the old skills fade, does meaningful oversight survive? StrongDM says oversight moves from reviewing code to designing scenarios and monitoring satisfaction. That may prove sufficient. It is also the kind of arrangement where confidence builds gradually, scrutiny fades, and the skills needed to catch a serious failure quietly disappear.

Regulatory Implications

So what happens when something goes wrong? Regulation in software has always been reactive. It responds to harm after the fact. The AILCCP framework cannot change that, but it can identify where the gaps are before they produce failures. Three stand out: nobody knows who is liable, nobody knows what to disclose, and the contracts have not caught up.

Accountability in software has historically (and to this day) worked through product liability, professional licensing, and contractual warranties. None of these contemplate software that no human has reviewed. The FTC’s enforcement actions have focused on deceptive marketing and consumer protection. But a Software Factory producing security infrastructure raises different questions entirely. If an access management system fails because an agent-written module contained a subtle error that no human ever saw, who is liable? The three engineers who designed the architecture? The AI provider whose model generated the code? The company that sold the product?

The liability question is hard enough. The disclosure question may be worse. When a customer asks “how was this software built?” the truthful answer is: “Coding agents wrote it. Other agents tested it against replicas of your services. Satisfaction scores exceeded our threshold.” Most procurement officers, auditors, and regulators have no way to evaluate that answer. But the problem runs deeper than unfamiliarity. Even if they understood, they would have no framework for deciding whether the answer is acceptable. No industry standard defines what a sufficient satisfaction score looks like. No audit methodology covers agent-built software tested against replicas. No procurement checklist asks whether the vendor’s coding agents share blind spots with the vendor’s testing agents. The disclosure is technically accurate and practically useless, not because the listener is unsophisticated, but because the tools for making sense of it do not exist yet.

And here is the quiet (or loud) absurdity that deserves attention. Open the terms of service for any AI-built product shipping today. Read the galactic warranty disclaimers, the limitation-of-liability clauses, the “AS IS” language. You will find them virtually identical to the terms that have accompanied software for decades. The same boilerplate that disclaimed liability when dozens of engineers wrote and reviewed every line now disclaims liability when no human has looked at the code at all. The contractual wrapper has not changed while the thing inside the wrapper has. A limitation-of-liability clause drafted for software built by humans, tested by humans, and reviewed by humans is now quietly absorbing the risk of software that was none of those things. Nobody updated the contract because the contract was never designed to describe how the software was made. It was designed to limit–or more accurately–extinguish what happens when the software breaks. And so the same language that once disclaimed imperfection in a human process now disclaims the absence of a human process entirely.

That gap between what the product is and what the contract says creates a credibility problem the AILCCP’s Trustworthy principle identifies directly: blanket disclaimers that contradict a vendor’s own trust claims destroy the trust they are trying to build. Try telling an enterprise customer that your software was never reviewed by a human. Then hand them the same limitation-of-liability clause their vendor used in 1996.

But perhaps this is transitional. The Software Factory represents such a thorough departure from conventional development that it might eventually produce an entirely new contractual form. A vendor confident enough to eliminate human code review might also be confident enough to offer terms that reflect what the product actually is: a warranty tied not to human inspection but to satisfaction scores, scenario coverage, or Digital Twin fidelity, with disclosures covering the agent architecture, the testing methodology, and the threshold at which the vendor considers the software fit for use. Nobody has done this yet, and the reasons are structural. Insurance underwriters price risk based on categories they understand, and “software produced without human review, tested by AI against simulated services” does not appear in any underwriting model. Investors would read novel warranty terms as voluntary assumption of liability. The legacy boilerplate persists because it limits exposure, satisfies insurers, and avoids alarming the board, not because it accurately describes the product.

The liability gap, the disclosure gap, and the contractual gap all point to the same underlying problem. Stuart Russell’s AI alignment asks a deceptively simple question: when we build systems that optimize for the objectives we give them, have we preserved the ability to step in and correct course when those objectives turn out to be wrong? For the Software Factory, the answer is not yet and probably never. No regulatory framework addresses this mode of production at all. And the exponential adoption curve means the window for getting ahead of it is narrow. If StrongDM’s approach spreads at the rate current trends suggest, Software Factories could be producing a significant share of commercial software within two years.

The Software Factory’s greatest risk is not that agent-written code will be worse than human-written code. It may very well be better. The risk is that when it fails, nobody will know why. Nobody will know how to fix it. And the institutional knowledge required to understand the failure will have atrophied, because the humans stopped reading code years ago.

Cette série de conférences portera sur des ouvrages récents et incontournables qui sont alignés avec les thématiques de l’axe Droit, cyberjustice et cybersécurité. Ainsi, pour cette deuxième édition du cercle de lecture de cette programmation, Félix Tréguer présentera son ouvrage Technopolice: La surveillance policière à l’ère de l’intelligence artificielle (Éditions Divergences) dans un premier temps puis, Benoît Dupont y réagira dans un second temps en tant que répondant.

À propos de l’ouvrage

Drones, logiciels prédictifs, vidéosurveillance algorithmique, reconnaissance faciale: le recours aux dernières technologies de contrôle se banalise au sein de la police. Loin de juguler la criminalité, toutes ces innovations contribuent en réalité à amplifier la violence d’État. Elles referment nos imaginaires politiques et placent la ville sous contrôle sécuritaire. C’est ce que montre ce livre à partir d’expériences et de savoirs forgés au cours des luttes récentes contre la surveillance policière. De l’industrie de la sécurité aux arcanes du ministère de l’intérieur, de la CNIL au véhicule de l’officier en patrouille, il retrace les liens qu’entretient l’hégémonie techno-solutionniste avec la dérive autoritaire en cours.

À propos de l’auteur

Félix Tréguer est chercheur associé au Centre Internet et Société du CNRS et membre depuis 2009 de La Quadrature du Net, une association dédiée à la défense des droits humains dans le contexte d’informatisation.

Ses recherches combinent l’histoire et la théorie politiques, le droit ainsi que les études des médias et technologies pour se pencher sur l’histoire politique d’Internet et de l’informatique, les pratiques de pouvoir comme la surveillance et la censure, la gouvernementalité algorithmique de la sphère publique et, plus largement, la transformation numérique de l’État et du domaine de la sécurité.

Il a notamment travaillé au Berkman Klein Center for Internet & Society de l’université d’Harvard, au Centre de recherches internationales de Sciences Po, à l’Institut des Sciences de la Communication du CNRS. Fin 2021, il a été chercheur invité au WZB Berlin Social Science Center. et à l’été 2024, à l’Institut Technologie et Société de Rio de Janeiro.

À propos du répondant

Benoît Dupont est, depuis 2016, titulaire de la Chaire de recherche du Canada en Cyber-résilience. De 2006 à 2016, il fut titulaire de la Chaire de recherche du Canada en sécurité et technologie. Il est professeur titulaire à l’École de criminologie de l’Université de Montréal et Directeur scientifique du Réseau intégré sur la cybersécurité (SERENE-RISC), qu’il a fondé en 2014. Il siège également comme observateur représentant le monde de la recherche sur le conseil d’administration du Canadian Cyber Threat Exchange (CCTX).

Controls frequently serve multiple AILCCP principles. Sandboxing, for instance, implements both Safety and Security, because isolating an agent’s execution environment simultaneously prevents harmful actions and resists adversarial exploitation. This cross-mapping reflects the structural reality that principles are analytically distinct but operationally entangled.

The table below demonstrates that common proposals for agent oversight, including logging, kill switches, sandboxing, rate limits, human-in-the-loop gates, and transparency requirements, already exist as named controls within the AILCCP framework. Each maps to one or more AILCCP principles that supply the normative justification for its deployment. For these mechanisms, the task is selection among existing controls based on the AILCCP principles most intensely activated by a given agent deployment. Where novel capabilities outpace current controls, the AILCCP framework accommodates additions through its versioning protocol.

* Here is the publicly-accessible version of the AILCCP.

Dans cette conférence intitulée The Coming AI Hackers, Bruce Schneier s’intéresse à l’émergence des « hackers de l’IA » — une nouvelle génération d’attaques et de vulnérabilités rendues possibles par l’intelligence artificielle. Au-delà du piratage informatique classique, il élargit la notion de hacking à des systèmes complexes tels que les cadres juridiques, les marchés financiers ou encore les institutions politiques, et interroge la capacité de nos sociétés à anticiper et encadrer ces risques.

La conférence propose ainsi une réflexion essentielle sur les enjeux de sécurité, de gouvernance et de politiques publiques liés au déploiement accéléré de l’IA, à un moment où ces technologies transforment en profondeur les équilibres sociaux, économiques et institutionnels.

I review pretty much everything related to AI governance. It’s central to my work at Stanford. It’s also central to what I call the “AI Life Cycle Core Principles” framework, which is now almost three years in the making. But governance, which, at a high level, stands for the sum of the organization’s policies, procedures, processes, and practices relative to AI, didn’t start as a central principle. It was just one of 37 principles that coalesced over time into what it takes to properly and efficiently deal with AI. But Governance (AILCCP principles appear in uppercase) rather quickly claimed its prominence. The more work I did on the framework, the more I saw that without proper alignment with the Governance principle the organization’s prospects of successful AI implementation are bleak. Now, it’s the single most important principle.

And this brings me to the present issue. Without naming names (I have no interest in stirring discomfort) I want to shed light on a common practice among consultants when it comes to talking about AI governance. (And yes, I am using the term in lowercase on purpose.) 

I recently came across a white paper published by what I’ll call the “Acme Group.” The paper is actually quite similar to many others you will find out there. So, my comments here should be helpful beyond just this example. And what you will see fairly quickly is that the advice, as well-intentioned as it may be, is a prescription for paralysis.

Acme Group seeks to advance its AI consulting expertise. It prescribes at least seven distinct committee or oversight structures:

1. Generative AI Governance Committee (cross-functional)
2. Ethics Committee or Advisory Board
3. Technology Assessment processes
4. Post-Implementation Review bodies
5. Reputation Response Team
6. Innovation Labs Oversight
7. Various “cross-functional teams” for specific controls

Each of these bodies requires membership, meeting cadences, documentation, and presumably the authority to delay or block AI implementation initiatives. The cumulative effect is decision diffusion: accountability is dispersed across so many nodes that no single entity possesses the authority or incentive to say “yes.”

Three Dimensions of the Critique

1. Velocity Degradation

Every committee represents a synchronization point. If a GenAI initiative must secure approval from governance, ethics, technology assessment, and post-implementation review bodies (each meeting monthly or quarterly), the calendar arithmetic alone suggests multi-month latency for even modest deployments.

Acme Group advises securing stakeholder approval before deployment. Sounds reasonable, or at the very least, harmless, right? It’s not. When multiplied across committees, this requirement becomes a veto chain where any single objection halts progress. Too many parties holding blocking rights inevitably results in underutilization of the resource (in this case, the desired AI capability).

2. Accountability Diffusion

The proliferation of oversight bodies paradoxically undermines the very accountability the framework purports to establish. When seven committees review a decision, who bears responsibility for its consequences? The AILCCP principle of Accountability requires, among other things, that “output is traceable to appropriate responsible party.” Yet the Acme Group structure distributes decision rights so broadly that traceability becomes an exercise in finger-pointing.

When responsibility is shared, individual ownership diminishes. Committee members assume others have conducted rigorous review, resulting in superficial approval from all quarters and genuine scrutiny from none.

3. Performative Governance

Perhaps the most pointed critique: such committee structures can become governance theater. Organizations form committees to their heart’s content, draft policies, and create gigabytes of documentation. Window dressing. The appearance of oversight; zero substance.

This misguided approach generates an organization that is optimized for demonstrating diligence rather than exercising judgment. It’s a prescription for paralysis.

Aligning with the Governance Principle

The Governance principle does not dismiss the importance of forming committees, drafting policies, and creating documentation to promote deliberation. An organization that aligns with this principle will find that it calls for properly calibrating effort to the context of its operations; it’s not one-size-fits-all. An organization operating in healthcare or financial services might very well benefit from regimes that intentionally introduce friction. But the emphasis here is that this must be the product of thoughtful intentionality. The decision to create this type of structure renders deliberation a feature, not a bug.

The next day, on November 6, an opinion paper was published in Science detailing the consensus of experts across the natural and social sciences, law, and bioethics, that “neural organoids” require governance interventions involving, at minimum, a global monitoring system for the burgeoning field and its applications.[3] The following week, a group of experts and stakeholders met at the Asilomar Conference Grounds in California to discuss such issues around these neural organoids.[4]

Notably, though, the term “neurotechnology” does not appear anywhere in the article about neural organoids, nor in news coverage of the Asilomar event. Nor does the term “organoid” appear in either the UNESCO or OECD instruments.

In fact, it is difficult to tell whether these two objects are one and the same, or different, and on what grounds. Neither the nascent law of neurotechnologies nor the scientific discourse on neural organoids provide sufficient clarify on this question. This blog explores the sources of this uncertainty and identifies how issues in legal predictability or policy mismatches may flow from the absence of clarity.

Neural Organoids

A budding scientific field has been working with new “organoid” techniques to help model and better study neural tissue in vitro—meaning in the lab, outside of the body. Organoids are “organ-like,” but are very small and much less sophisticated than the actual organs they are based on, such as the brain. Scientists can use these small clumps of brain cells to understand how parts of the brain work or develop, or even to study how infections or other diseases may affect the brain and what treatments they may respond to.[5] The field has notable promise for scientific and clinical applications, but multiple technical challenges remain before that promise can be fully realized.[6]

However, describing what these nascent technologies are and what they are for is difficult. The field is still struggling to communicate internally and to the public about neural organoids. A notable perspective paper was published in Nature in 2022 by a group of organoid researchers trying to issue terminological norms for their field.[7] The paper also strongly objects to several ways these innovations have been described in journalistic media, such as “mini-brain” or “brain-in-a-dish,” but provides no compelling alternative or easier metaphor to use. This intervention illustrates that progress is being made in institutionalizing and standardizing the field of neural organoids—but also just how recently that work has commenced and how much difficulty there is in communicating to actors outside of the scientific field about what these innovations are.

What Are Neurotechnologies? Do Neural Organoids Count?

The term neurotechnology has begun to acquire a legal definition in national and international lawmaking efforts. In the UNESCO Recommendation, for instance, “[n]eurotechnology refers currently to devices, systems and procedures–encompassing both hardware and software–that directly measure, access, monitor, analyse, predict or modulate the nervous system to understand, influence, restore or anticipate its structure, activity and function.”[1] This provides a sweeping scope for the object of regulation, but does not specifically mention organoids.

The OECD’s legal instrument has a very similar definition to UNESCO, raising similar questions about whether neural organoids fit.[2] In the US, at the federal level, legislation was recently introduced in the Senate that would call on the Federal Trade Commission (FTC) to investigate whether and how “neurotechnology” raises data protection issues. The proposed MIND Act would contain a definition of neurotechnology that also very closely mirrors the UNESCO and OECD instruments, providing no greater clarity.[8]

While the term “neurotechnology” appears to have a very broad definition in emerging legal instruments, lawmakers generally use the term to mean something much narrower. The focus of lawmaking bodies and the experts consulting with them is predominantly on physical devices with software components—primarily, and sometimes exclusively, on “brain-computer interfaces” (BCIs) or brain stimulation devices.[9]

But neural organoids could still potentially fit into these instruments. The UNESCO definition, by including the clarifying clause “encompassing both hardware and software,” does appear to suggest that the definition applies to physical and digital devices. But its interpretation would depend on whether the clause is read to be the full extent of what “devices, systems, and procedures” can mean, or merely two examples of what those larger terms can be. Of note, the UNESCO instrument goes on to provide examples of neurotechnology that do focus on devices, but is proceeded by the caveat that “[n]eurotechnology includes, but is not limited to:”.[1] These elements of the instrument illustrate the lack of complete clarity in its scope.

Within the UNESCO instrument, at least, it is not wholly unreasonable that neural organoids could be interpreted to be “systems and procedures . . . that directly . . . analyse, predict, or modulate the nervous system,” especially to “understand [or] anticipate its structure, activity and function.”[1] Whether the UNESCO instrument applies exclusively to physical and digital devices or could apply to biological objects appears unclear at present.

The Potential for Legal Unpredictability or Mismatches

Where does this leave the governance of neural organoids? It is currently not clear, legally or scientifically, whether neural organoids are a part of neurotechnology. National and global rules are being set for neurotechnology, but not specifically for organoids.

This prompts the question: Do rules for neurotechnology apply to neural organoids?

When looking at current legal definitions, it looks likely—but not certain—that those rules do not apply to organoids. Yet, lawmaking bodies, regulators, or courts appear to have at least some interpretive room that could be used to position neural organoids as neurotechnology, and therefore subject to at least some of these new rules or norm-setting processes on neurotechnology. At the same time, the scientific field of neural organoids is still in the process of organizing itself and has had trouble communicating to external actors about what they do and why.[7] This lack of clarity within the scientific field itself, while understandable given its youth, may invite or enable decision-makers to more readily engulf neural organoids within neurotechnology rules.

This lack of legal and scientific clarity could create predictability issues, since organoid scientists and product developers may not know whether any of the emerging rules on neurotechnology will apply to their activities. It could also raise concerns for funders and investors, or even insurers for organoid-based therapeutics, who may or may not want to see compliance with those rules as a condition of supporting neural organoid work.

The definitional uncertainty also raises questions about whether neurotechnology rules are fit and appropriate for neural organoids, or if applying those rules could result in policy mismatches. Most of the rules about neurotechnology that have been set have mostly or only medical and consumer devices in mind—not biological innovations like organoids.[9] Applying those rules to neural organoids without thoughtfully considering whether and how they should be adapted to biologics could result in poor match between the goals of those rules and the issues that organoids may pose.

Clarifying Definitions, For Now

Since most existing and emerging legal instruments on neurotechnologies appear to only have medical and consumer devices in mind, this blog tentatively recommends that lawmaking and regulatory bodies should consider clarifying that these rules do not apply to neural organoids—at least, for now. At minimum, clarifying that neurotechnology rules do not currently apply to neural organoids would provide short-to-medium-term predictability to scientists and other stakeholders and avoid applying rules that may not be fit-for-purpose.

Since definitions are vague, it is unlikely that this path would require full amendments of these legal instruments. Clarification could instead come in the form of formal guidance issued by governing bodies, or less formally, in the form of public statements from officials about whether the implementation of those rules would cover neural organoids.

It may, however, be prudent to not close the door entirely to these legal instruments applying to neural organoids at some point in the future. While the neural organoid field would benefit from predictability, an international and interdisciplinary group of experts has agreed it is likely that governance for these emerging technologies will be required at some point.[3] Custom rules for neural organoids may be preferable, but having governance instruments for neurotechnologies available in reserve may be valuable if no such tailored rules arise.

References

[1] UNESCO, Recommendation on the Ethics of Neurotechnology (2025).

[2] OECD, Recommendation of the Council on Responsible Innovation in Neurotechnology, OECD/LEGAL/0457 (2019).

[3] Sergiu P. Pașca, et al., The Need for a Global Effort to Attend to Human Neural Organoid and Assembloid Research, 390 Science 574 (2025).

[4] Mitch Leslie, Lab-Grown Models of Human Brains are Advancing Rapidly. Can Ethics Keep Pace?, Science (Nov. 18, 2025), https://www.science.org/content/article/lab-grown-models-human-brains-are-advancing-rapidly-can-ethics-keep-pace.

[5] H. Isaac Chen, Hongjun Song & Guo‐li Ming, Applications of Human Brain Organoids to Clinical Problems, 248 Developmental Dynamics 53 (2019).

[6] Madeline G. Andrews & Arnold R. Kriegstein, Challenges of Organoid Research, 45 Annual Review of Neuroscience 23 (2022).

[7] Sergiu P. Pașca, et al., A Nomenclature Consensus for Nervous System Organoids and Assembloids, 609 Nature 907 (2022).

[8] MIND Act of 2025, S.2925, 119th Cong. §3(5) (2025).

[9] Walter G. Johnson, It’s (Not) Just Semantics: “Neurotechnology” as a Novel Space of Transnational Law, 50 Law & Social Inquiry 865 (2025).

Pour en savoir +

Lola Gregorowius est étudiante dans le cadre du cours DRT6929 (Vie privée + Numérique) (Hiver 2026)  

Le 13 janvier 2026, la CNIL (Commission nationale de l’informatique et des libertés) a condamné les entreprises françaises FREE et FREE Mobile à hauteur de 42 millions d’euros d’amende (environ 67771794.9 CAD) pour manquement au RGPD (le règlement général sur la protection des données).

Pour rappel, le RGPD, entré en vigueur en 2018, est un règlement européen qui encadre le traitement des données personnelles au sein de l’Union Européenne. Il s’applique aux organismes publics et privés qui traitent des données personnelles.

La CNIL est une autorité administrative indépendante qui est chargée de veiller à la protection des données personnelles. Elle a un pouvoir d’investigation, de contrôle et de sanction en cas de manquement.

Pour information, l’entreprise française Bouygues Telecom avait été condamnée en 2018 par la CNIL pour manquement à la loi Informatique et Libertés. Le montant de l’amende s’élevait à  250 000 euros (environ 405324,75 CAD) seulement. La sanction prononcée à l’encontre FREE et FREE Mobile n’est donc pas anodine et montre que la protection des données personnelles a pris une place importante ces dernières années.

Nous verrons dans un premier temps le contexte de cette décision, puis nous analyserons son contenu et enfin sa portée.

1. Le contexte

A. Un piratage sans précédent

Les entreprises FREE et FREE Mobile sont des opérateurs de télécommunication français. Ils représentent une part importante du marché en la matière. Ce géant de la télécommunication est né en 1999, avant le RGPD qui date de 2018. Ils ont donc dû s’adapter à son arrivée et revoir leur système de protection des données, ce qui n’a pas été de tout repos.

En effet, ce n’est pas la première fois que cette entreprise est au cœur de polémiques concernant la protection des données de ses utilisateurs. Déjà en 2021, la CNIL l’avait condamné à une amende de 300 000 euros, puis en 2022 avec le même montant d’amende. Dans les deux cas, c’était un manquement au RGPD dont il était question. On comprend ainsi que ce n’est pas la première fois que l’entreprise FREE dépasse les limites légales en termes de protection des données personnelles de ses abonnés.

Alors pourquoi la CNIL a-t-elle décidé d’augmenter autant le montant de l’amende cette fois-ci ?

En octobre 2024, un hacker web est parvenu à s’infiltrer dans le système informatique couvrant les données personnelles des clients de FREE. 24 millions de contrats d’abonnées au service ont été touchés. C’est l’une des plus grosses fuites de données enregistrées durant l’année 2024 en France. On sait aujourd’hui que toutes ces données ont été revendues.

A la suite de cet incident, plus de 2 000 plaintes d’utilisateurs ont été déposées auprès de la CNIL.

Les renseignements d’un réseau aussi important sont très sensibles, il est question ici de données bancaires comme des IBAN par exemple. L’autorité administrative est alors de nouveau intervenue. Après plus d’une année d’enquête, la CNIL a relevé plusieurs manquements au RGPD de la part de FREE et FREE Mobile.

Ce qui nous amène à la sanction très lourde rendue par la formation restreinte de la CNIL.

B. Une sanction lourde de sens

Depuis 2024, le nombre de fuites de données en France a augmenté de 20% par rapport à l’année 2023 (selon les chiffres de la CNIL). Les chiffres de l’année 2025 ne sont pas encore sortis, en revanche on sait que la France est un des pays les plus touchés par les fuites de données au sein de l’Union Européenne.

La CNIL s’est exprimée face à cette situation :

“[…] l’analyse des différentes phases des violations révèle qu’une succession de défauts de sécurité courants ont permis à l’attaquant de passer d’une étape à la suivante.”

Un grand nombre de fuites serait donc lié à des défauts du système de sécurité de la part des entreprises.

En sachant cela, la CNIL a condamné FREE en considérant qu’elle avait largement les moyens de protéger les données de ses utilisateurs, d’autant plus au regard de la sensibilité de ces dernières. En décidant de sanctionner sévèrement l’entreprise de télécommunication, elle a fait d’elle un exemple pour les autres entreprises qui traitent de données personnelles.

Il nous faut à présent analyser la décision afin de mieux comprendre les enjeux autour de cette sanction inédite.

2. L’analyse de la sanction de la CNIL

La CNIL a repéré plusieurs manquements au RGPD qui lui ont permis de fonder son enquête et plus tard sa décision. Elle montre aux autres grandes entreprises détentrices de données personnelles, que le RGPD n’est pas une option. 

A. Un manquement à l’obligation d’assurer la sécurité des données personnelles (article 32 du RGPD)

L’organisme administratif considère que le système de sécurité mis en place par FREE était inefficace et n’assurait pas la confidentialité des données personnelles. Elle rappelle qu’on ne peut éliminer tout risque, mais qu’il faut tenter de le réduire au maximum et elle a enjoint l’entreprise à continuer ses efforts.

B. Un manquement à l’obligation de communiquer auprès des personnes concernées par la violation de données (article 34 du RGPD)

L’obligation de communiquer la violation de données auprès des personnes concernées à été partiellement réalisée par les entreprises FREE et FREE Mobile. Notamment grâce à un courriel d’information, ainsi que par un numéro vert et une section interne dédiée à la gestion des demandes.

Toutefois, la CNIL a considéré que ce n’était pas suffisant :

“ […] le courriel adressé ne comportait pas toutes les informations nécessaires […], en estimant que ces omissions ne permettaient notamment pas aux personnes concernées de comprendre directement les conséquences de la violation […].”

C. Un manquement à l’obligation de conserver les données personnelles pendant une durée limitée (article 5-1-e) du RGPD)

L’article 5-1-e du RGPD prévoit une obligation de conserver des données personnelles pendant une durée limitée. La société FREE Mobile a conservé des millions des données de ses abonnés, sans justification, et pendant une durée allant au-delà de celle autorisée. La CNIL enjoint à l’entreprise de trouver un système de tri qui permettra de stopper le stockage de données sensibles qui apparaît comme non-nécessaire.

3. Les effets de la sanction de la CNIL

Au regard de cette lourde condamnation et des dernières sanctions prononcées par la CNIL, on peut conclure que l’organisme souhaite durcir son contrôle et son impact sur les entreprises françaises. Face à des cyber-attaques toujours plus nombreuses et ingénieuses, il est nécessaire d’avoir des systèmes de protection adaptés et efficaces. Concernant cet enjeu, je vous renvoie au papier de Maître Thiébaut Devergranne qui décrypte les nouvelles priorités de la CNIL.

On peut alors se demander si cette augmentation du montant d’amende va s’appliquer dans les prochaines affaires de fuites de données personnelles ou si FREE est un cas à part.

Pour conclure, FREE et FREE Mobile ont répondu à la sanction de la CNIL en déposant un recours devant le Conseil d’Etat français. Ils estiment le montant de l’amende disproportionné aux atteintes commises, notamment au regard d’anciennes affaires similaires.

Entre une remise en cause des sanctions toujours plus lourdes de la CNIL, une augmentation des fuites de données personnelles, des systèmes de sécurité inefficaces, les choix de la CNIL seront décisifs pour les années à venir en matière de protection des renseignements personnels.

In 2025, the Project further strengthened its role as a global reference point for research at the intersection of competition law, economics, and computational methods, with a particular focus on how antitrust enforcement is adapting to data-intensive markets and artificial intelligence.

Read full report