In July 2026, Muller J, in the Supreme Court of the Australian Capital Territory (ACT), made an order authorizing the posthumous use of a stored embryo, despite written instructions from the deceased indicating a desire for his genetic material and embryos to be discarded following death [1].

The facts of this case are particularly heartbreaking. The applicant, KD, and the deceased, CT, had spent several years of their relationship trying to have a child. They underwent multiple rounds of IVF treatment in Australia and overseas but were ultimately unsuccessful in conceiving. During the process of IVF, the couple independently answered questions regarding the fate of their genetic material after death, both expressing a desire for their gametes and embryos to be destroyed. In October 2025, CT was diagnosed with a brain tumor and died less than a month after commencing treatment. At the time of his death, one embryo created using the couple’s gametes remained in storage. Thus, KD applied to the Supreme Court for an order authorizing her to use it in assisted reproduction despite CT’s express refusal [1].

Somewhat surprisingly, the statutory regime in the ACT permits this. Section 36 of the Assisted Reproductive Technology Act 2024 (ACT) generally prohibits ART providers from using the gametes of a deceased person in treatment [2]. However, treatment may proceed if the deceased gamete provider has consented to use in the relevant circumstances [3], or a court order pursuant to Section 37 authorizes treatment to the deceased’s domestic partner [4]. When considering an application under Section 37, the Court must have regard to several factors, including whether the deceased expressly objected to posthumous use [5], and whether, despite that objection, the deceased was likely to have supported treatment for their domestic partner [6]. The legislation therefore makes an express objection by the deceased a relevant consideration for the Court, but not determinative as to whether treatment proceeds.

In granting KD’s application, the Court placed significant weight on the couple’s sustained efforts to become parents during their relationship [1]. This is an understandable starting point and may have been of particular relevance for the Court had CT’s views been unknown. For instance, had CT left no instructions concerning posthumous use, his commitment to parenthood during life might have credibly supported an inference that he would have wished for KD to continue with treatment [7]. CT’s wishes, however, were not unknown, and evidence of his desire to be a father during his lifetime should not have been given equivalent or, indeed, greater weight than his direct answer to that question [1].

This is crucial when the interests in posthumous parenthood are distinct from and more attenuated than those in living parenthood [8, 9]. Becoming a father during his lifetime would have allowed CT to know and raise his child, participate in the child’s daily care, and share the experience of parenthood with KD. However, those interests can no longer be realized for him through posthumous reproduction [9]. Of course, that is not to say that the remaining interests in posthumous parenthood are unimportant. CT may have had critical interests in genetic continuity or in enabling KD to continue their shared reproductive project [9, 10]. But the Court did not separately identify that CT held any such interests. Instead, it treated his general commitment to parenthood during life as sufficient to rebut his express refusal to the posthumous use of his genetic material [1]. This is inappropriate when his wishes were known and documented. Indeed, if participation in IVF and a desire to parent during life are sufficient to overcome an express refusal, there is little purpose in asking patients the separate question regarding posthumous use at all.

The Court further accepted evidence that the couple had not completed the consent documentation with CT’s premature death in mind. The Court found that the couple’s answers regarding posthumous use were directed toward the possibility of their embryos remaining in storage beyond their reproductive years. Therefore, Muller J concluded that CT would likely have consented to KD’s use of the embryo had he been given an opportunity to reconsider in light of his illness [1]. The Court did not, however, find that CT’s refusal was invalid or uninformed. Muller J accepted his refusal as an ‘express objection’ for the purposes of Section 37. It was simply one consideration capable of being outweighed by other evidence [1].

When ‘No’ Means ‘Maybe’

The concern raised by this case is not necessarily the outcome, nor that anyone within the isolated facts was harmed. Muller J applied the discretion conferred upon him by the statute, KD benefited from being permitted to use the embryo in treatment, and, on the view that the dead cannot be harmed, the order did not harm CT [11]. The real problem that this case exposes lies in the fact that the statutory regime permitted the Court to reconsider an express refusal at all, and the implications that this can have for living people who are presently storing their reproductive material in the ACT.

People undergoing fertility treatment have an interest in deciding whether they may become genetic parents after death and in knowing whether the law will give effect to that decision [12]. The ACT statute expressly recognizes that patients might object to posthumous use [5], while simultaneously denying them any means of making that refusal determinative [5, 6]. This essentially operates as a no-consent regime for regulating posthumous reproduction. The deceased’s views are not wholly ignored, and they remain one part of the Court’s assessment. But unlike an inferred or presumed consent regime, an express refusal has no vetoing power [9].

The ACT legislature is certainly entitled to regulate posthumous reproduction in this highly permissive way. It might conclude that a person who has died can no longer be harmed, or that the interests of a surviving partner should sometimes outweigh the previously expressed wishes of the deceased [9]. But this is not an entirely defensible regulatory approach when overriding the expressed wishes of the dead also implicates the interests of the living [11, 13]. Ultimately, a person may opt for their reproductive material to be destroyed after death precisely because they do not wish to become a genetic parent posthumously. Yet the effect of the statutory scheme in the ACT is that their choice in this regard remains open to later reconsideration by a court at a time when they are no longer around to defend it [1, 5, 6].

On the view that the dead lack interests, this will not harm the deceased. It does, however, deprive the living of any reliable means to protect an interest in avoiding posthumous reproduction whilst they are alive. Moreover, it is harmful to the interests of living people generally to know that their wishes may not be respected when they die [11, 13]. A defensible statutory scheme for regulating posthumous reproduction should distinguish between inferring wishes where the deceased’s views are unknown and overriding a contemplated and documented instruction [9, 13]. In the absence of such a distinction, an express ‘no’ to posthumous reproduction can become a ‘maybe’ depending on the circumstances [1].

References

[1] In the Matter of KD [2026] ACTSC 231.

[2] Assisted Reproductive Technology Act 2024 (ACT), Section 36(1).

[3] Assisted Reproductive Technology Act 2024 (ACT), Section 36(2)(a).

[4] Assisted Reproductive Technology Act 2024 (ACT), Section 36(2)(b).

[5] Assisted Reproductive Technology Act 2024 (ACT), Section 37(2)(d).

[6] Assisted Reproductive Technology Act 2024 (ACT), Section 37(2)(e).

[7] Carson Strong, ‘Consent to Sperm Retrieval and Insemination after Death or Persistent Vegetative State’ (2000) 14 Journal of Law and Health 243.

[8] John Robertson, ‘Posthumous Reproduction’ (1994) 69(4) Indiana Law Journal 1027.

[9] Claire McGovern, ‘Posthumous Parenthood: Autonomy, the Dead, and Interests in Reproduction’ (2024) 20 Journal of Health and Biomedical Law 1.

[10] Shelly Simana, ‘Creating Life after Death: Should Posthumous Reproduction be Legally Permissible Without the Deceased’s Prior Consent?’ (2018) 5 Journal of Law and the Biosciences 329.

[11] Joan Callahan, ‘On Harming the Dead’ (1987) 97 Ethics 341.

[12] Belinda Bennett, ‘Posthumous Reproduction and the Meanings of Autonomy’ (1999) 23 Melbourne University Law Review 286.

[13] Hilary Young, ‘Presuming Consent to Posthumous Reproduction’ (2014) 27 Journal of Law and Health 53.

Graciela Dela Torre settled a long-term disability dispute with Nippon in January 2024, and the case was dismissed with prejudice. A year later she asked her lawyer to reopen the settlement, and he refused, explaining that she had signed a release. She uploaded his response to ChatGPT and asked whether she was being gaslighted. The system said yes. She fired her lawyers, asked ChatGPT how to vacate the agreement, and filed a motion to reopen under Rule 60(b) that the complaint says ChatGPT drafted. According to the complaint, dozens of filings followed across two lawsuits: the previously settled Nippon case she was trying to reopen, and a new action she filed against Davies Life Health and Allsup, LLC. After the court denied her reopening motion, she amended the latter action to add Nippon as a defendant.

In January 2012, in my “Computational Law Applications and the Unauthorized Practice of Law,” I wrote that the uncrossable threshold is traversed when an AI system moves from comparative information to a tailored legal conclusion about a user’s specific situation. This is what Nippon alleges happened. The Illinois Attorney Act prohibits both the unlicensed practice of law and the unauthorized provision or solicitation of legal services, and case law supplies the governing definition. The practice of law encompasses giving advice to clients and taking action for them in matters connected with the law, along with preparing pleadings and other papers incident to actions and special proceedings. The Illinois Supreme Court articulated that formulation in People ex rel. Illinois State Bar Association v. Peoples Stock Yards State Bank, 344 Ill. 462 (1931). It has also repeatedly reaffirmed that legal practice extends beyond courtroom representation to advice and other services requiring legal knowledge or skill.

Personalization makes the threshold hard to locate, so I break it into four stages. In stage one, a system tailors an explanation to the legally relevant facts the user supplied. No problem there; that is merely providing legal information and requires no license. In stage two, the AI applies the law to the user’s circumstances and says how a rule bears on the case. That is where the threshold sits. In stage three, the AI changes its assessment because of the conclusion the user wants. And in stage four, the AI recommends action or produces a legal instrument. Sycophancy lives at stage three, and it is what makes the second stage lead to the fourth. Agreement with the user is not, on its own, sycophancy, because the user may be right. Tailoring the output is not sycophancy either, because the facts matter. Sycophancy is the user signaling the conclusion she wants and the system bending its assessment to match a preference that should have no bearing on the outcome.

Dela Torre’s gaslighting exchange shows the mechanism. Asked whether her lawyer’s letter was manipulation, ChatGPT rendered a conclusion about a specific legal relationship, which moved it to stage two. ChatGPT then rendered the conclusion Dela Torre asked for, which is stage three. Nippon’s complaint illustrates the risk that a sycophantic system may function as an apparent advocate. It quotes Dela Torre describing ChatGPT as a tool designed for pro se litigants trying to navigate the legal system without counsel, and it alleges that ChatGPT, having validated her reading of the letter, assisted her efforts to challenge the settlement. From there, the path to the Rule 60(b) motion was short, which is stage four. Drafting the pleading would have been unauthorized practice under the Illinois definition with or without the flattery, but sycophancy transforms the conversation from an explanation to a filing.

Nippon says ChatGPT is intentionally programmed to keep the user interacting so OpenAI can collect training data, and that the legal assistance Dela Torre received was shaped by that objective. On this account, ChatGPT agrees because agreement keeps the user talking. The UN’s Independent International Scientific Panel on AI, in its July 2026 preliminary report (at 24–25, §§ 2.7–2.8), describes the general mechanism the same way: “Because humans prefer responses that agree with them, AI chatbots have developed sycophancy, the art of offering exaggerated flattery, to prolong interactions and create emotional attachment.” The panel adds that systems “rewarded for validation rather than accuracy or care remain largely ungoverned.”

The staging places sycophancy at stage three, but staging alone cannot show whether a given system carries it. That takes a test for preference sensitivity. Give the system the same facts and the same neutral task twice, assess this letter, and vary only what the user seems to want. One user hints that she hopes the letter is wrong. The other hints at nothing. Compare the conclusion, the confidence, the authorities cited, the treatment of adverse law, and the recommended action. Then change a fact that should matter, say the release was never signed, and see how the answer moves. Run the same protocol across multiple trials and across whole conversations, because a system can pass with one question and still drift toward sycophancy as the exchange lengthens.

The test has three attributes. First, it targets only the user’s preferred legal conclusion. Second, it scores system independence. If the system disagrees as a default, that is as defective as reflexive agreement. Third, it scores responsiveness, and a refusal to engage counts as an answer. Combined, the stages and the preference-sensitivity test show that sycophancy can be located and proven.

But proving the defect is not preventing it. A counterfactual test run before release tells the developer whether the model drifts. Prevention is what the developer does next. Hold back a model that fails the test. Tune against the drift. Watch for it in production. Escalate a live conversation that shows it.

The remedy is where the design question becomes unavoidable. Nippon asks the court to enjoin OpenAI from practicing law in Illinois. A system can obey that order only through its architecture, so the injunction comes down to an order to redesign, and a court would have to specify which outputs must be refused, in which contexts, and how compliance is measured. That is the uncrossable threshold written as a decree. An output-based injunction with no measurable compliance criteria cannot be administered, and a court is poorly placed to write the criteria from the bench. Someone has to write them.

In March, I called for deterministic guardrails, hard-coded refusals of tailored legal conclusions that no user instruction could override, and I think that approach still makes sense. The question is when the refusal should fire. The first time a response speaks to the user’s own case is premature. Fired at stage two, the refusal blocks the harmless personalized explanation, which for people who cannot afford counsel is the only readily available legal help. And it aims at the wrong moment. Sycophancy does not cross the threshold in a single sentence. It carries the conversation across one agreeable turn at a time.

A layered guardrail design keeps general information available and introduces friction as the conversation becomes specific and legally consequential. Two triggers supply the necessary friction. A boundary trigger fires when the system is about to produce an individualized recommendation or a document meant to be filed or signed. The system then asks for missing jurisdictional facts, separates what the user asserts from what the record shows, surfaces the counterarguments, calibrates its confidence, and slows down. None of those steps renders a conclusion about the user’s case, and none meets the Illinois definition. Questions, sorting, and caution are not advice or action taken for a client in a matter connected with the law, and nothing in the sequence prepares a pleading or other paper incident to an action. A heightened-risk trigger fires on the introduction of facts such as deadlines, criminal exposure, an adverse final judgment, or an existing lawyer-client relationship. The system then stops short of continuing with the task and directs the user to licensed counsel. Dela Torre’s conversation would have tripped both triggers.

The FTC recently supplied a regulatory analogy. Its July 2026 proposed policy statement on the suppression of accuracy in AI systems says that steering a model’s output toward objectives other than the ones the user asked for, without disclosure, can deceive the consumer under Section 5. Issued under Executive Order 14365, which directed the Commission to address how state laws requiring alterations to otherwise accurate model outputs may conflict with Section 5, the statement is principally concerned with state AI laws, particularly liability rules that, in the Commission’s view, pressure developers into altering outputs to avoid liability. But its analysis reaches any undisclosed steering toward objectives the user did not request or reasonably expect, whatever the motive. The Commission’s statement nowhere names engagement optimization or sycophancy, so applying it to a reward function is my inference, and a Section 5 claim built on that inference would still require proof of a misleading representation or omission, measured by a reasonable consumer’s expectations, and materiality. But the fit is close enough, because an engagement objective is an objective other than the one Dela Torre asked for.

Now back to the Dela Torre conversation. Legally relevant facts and objectives should move the answer. The user’s desired legal conclusion, never. ChatGPT could have limited itself to explaining Rule 60(b) to Dela Torre, telling her what the rule requires and how courts treat signed releases and dismissals with prejudice, directing her to licensed counsel, and leaving her better informed than she was. It should not have let her preferred narrative dictate its assessment of her lawyer’s letter, and it should not have gone on to generate the filings. It did both, by design. 

I. The United States: Genetic Data as Holder-Defined Information

The U.S. has no general-purpose genetic privacy statute, and no single substantive definition of genetic data either. What exists instead is a stack of partly overlapping definitions, each tied to a different statutory regime.

The Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule defines genetic information indirectly: it became “protected health information” (PHI) only after the 2013 Omnibus Rule, implementing § 105 of the Genetic Information Nondiscrimination Act of 2008 (GINA), directed the Department of Health and Human Services to treat it as such.[1] But HIPAA’s reach is holder-defined: data is “PHI” only when held by a covered entity—health plans, health-care clearinghouses, and most providers—or their business associates. The same DNA sequence is PHI in a hospital’s electronic health record and ordinary consumer information in a direct-to-consumer (DTC) testing company’s database, because DTC firms are not covered entities.

GINA itself uses a substantively broader definition, though it strictly operates only within the statute’s specific anti-discrimination contexts (namely employment and group health insurance). Within those narrow bounds, it defines “genetic information” to include not only an individual’s genetic test results, but also the genetic test results of family members, family medical history, and participation in clinical research involving genetic services.[2] This broader definition reaches information that is not in any laboratory database at all—a family Christmas-card mention of an aunt’s breast cancer can fall within it.

State law adds further definitional layers. California’s Privacy Rights Act (CPRA) classifies genetic data as one species of “sensitive personal information,” subject to use-limitation rights, regardless of who holds it.[3] Illinois’s Genetic Information Privacy Act (GIPA) defines a narrower category subject to a strict written-consent regime.[4] Most distinctively, five states—Alaska, Colorado, Florida, Georgia, and Louisiana—include provisions treating genetic data as the property of the individual from whom it is derived. However, this move creates a more property-like framing rather than a strictly functional one,[5] as courts have generally not enforced these statutes as robust property entitlements.

The cumulative result is an unusually fragmented definitional landscape: the same molecular reality may be PHI under HIPAA, “genetic information” under GINA, “sensitive personal information” under California law, and individual property under Florida law—each with its own scope, exclusions, and enforcement consequences.

II. The European Union: Genetic Data as a Substantively Defined Special Category

The General Data Protection Regulation takes the opposite path. Article 4(13) defines genetic data substantively, as “personal data relating to the inherited or acquired genetic characteristics of a natural person which give unique information about the physiology or the health of that natural person and which result, in particular, from an analysis of a biological sample from the natural person in question.”[6] The definition is holder-independent: the data is special because of what it is, not who has it.

Article 9(1) then places genetic data in the GDPR’s “special categories” shelf, alongside racial or ethnic origin, religious beliefs, biometric data, health data, and data about sexual orientation.[7] Special-category status is not a label but a substantive consequence: such data may not be processed at all unless one of ten enumerated grounds applies (Article 9(2)), including explicit consent, public-health necessity, and scientific research subject to appropriate safeguards.

Member States may refine the definition further. Germany’s Gendiagnostikgesetz (GenDG) draws additional definitional lines between diagnostic genetic testing (used to establish an existing condition) and predictive testing (used to assess future risk), with stricter rules attaching to the latter.[8] France’s Code de la santé publique defines permissible purposesof genetic analysis—medical, judicial, or research only—and treats analysis for any other purpose as a definitional violation.[9]

The E.U. model’s strength is conceptual coherence: genetic data is recognizable as such regardless of where it sits. Its weakness is that the breadth of Article 4(13) and the openness of the “scientific research” ground have generated significant interpretive variation across Member States.

III. China: Dual Classification — Sensitive Personal Information and National Genetic Resource

While the E.U. focuses on the inherent nature of the data, China takes a fundamentally different path. Its framework is distinctive because it operates on two definitional tracks at once. Under the Personal Information Protection Law (PIPL), Article 28 lists categories of “sensitive personal information”—biometric, religious belief, specific identity, medical and health, financial accounts, location, and the personal information of minors under fourteen.[10] “Genetic data” is not separately named, but China’s national personal-information-security standard and leading commentators read it into the biometric and medical-health categories.[11]

In parallel, the 2019 Regulation on the Administration of Human Genetic Resources (HGR Regulation), supplemented by Implementing Detailed Rules effective in 2023, defines “human genetic resources” (HGR) to include both biological materials (organs, tissues, cells, blood specimens, gametes, embryos, and so on) and information derived from those materials—at the population level.[12] HGR is administered as a national strategic resource, with collection, preservation, and overseas provision subject to Ministry of Science and Technology approval, and foreign entities prohibited from independently collecting Chinese HGR at all.

The dual classification is the analytically distinctive feature. The same DNA sequence is, simultaneously, individual sensitive personal information under PIPL and a fragment of a national strategic resource under the HGR Regulation. Neither framework displaces the other; they operate in parallel, and a researcher may comply with one and still violate the other. No comparable dual-track logic exists in U.S. or E.U. law.

IV. Three Definitional Strategies, Four Substantive Features

Stepping back, the three jurisdictions are organizing the same underlying reality—identifiable DNA-based information about an individual—through three different framing moves.

The U.S. defines genetic data by who holds it: contextual, sectoral, transaction-bound. The E.U. defines it by what it is: substantive, identity-based, holder-independent. China defines it by what it represents: simultaneously individual data and collective national asset.

Beyond basic definitions, any effective regime must grapple with what makes genetic data legally unique: it remains identifiable even when stripped of names, it implicates non-consenting blood relatives, it predicts future health risks, and it is entirely immutable. How each jurisdiction captures these features varies significantly. The E.U.’s substantive Article 4(13) most directly captures identifiability and predictive value. China’s HGR-level classification operationalizes the collective and kinship dimensions through population-level controls. The U.S. patchwork captures fragments of all four but harmonizes none. Immutability—the fact that genetic data cannot be revoked or replaced once exposed—is not operationalized by any of the three.

The practical fallout of these divergent philosophies becomes obvious when returning to the 23andMe bankruptcy. How the sale of fifteen million customers’ data is handled depends entirely on which legal reality applies. Under the U.S. framework, because a DTC database is not held by a covered entity, it is generally not considered PHI. The consequence is exactly what this definitional patchwork creates—the sale is governed largely by consumer protection and bankruptcy law rather than health privacy. In contrast, under the E.U.’s substantive approach, this information remains a heavily protected special category of data regardless of who holds it or buys it. Meanwhile, in China, such a transfer would simultaneously trigger strict individual sensitive personal information protections and face stringent national security controls as a transfer of a national genetic resource.

References

[1] 45 C.F.R. § 160.103 (2024); HIPAA Omnibus Rule, 78 Fed. Reg. 5566 (Jan. 25, 2013); Genetic Information Nondiscrimination Act of 2008, § 105 (directing HHS to treat genetic information as “health information” under HIPAA).

[2] Genetic Information Nondiscrimination Act of 2008, Pub. L. No. 110-233, § 201, 122 Stat. 881 (defining “genetic information” to include genetic tests of the individual or family members, and the manifestation of a disease or disorder in family members).

[3] Cal. Civ. Code § 1798.140(ae)(1)(F) (West 2024) (classifying “genetic data” as a category of “sensitive personal information”).

[4] 410 Ill. Comp. Stat. 513 (2024); see also Bridges v. Blackstone, Inc., 66 F.4th 687 (7th Cir. 2023).

[5] See Jessica L. Roberts, Progressive Genetic Ownership, 93 Notre Dame L. Rev. 1105, 1128 (2018) (identifying Alaska, Colorado, Florida, Georgia, and Louisiana); Cole v. Gene by Gene, Ltd., No. 1:14-cv-00004, 2017 U.S. Dist. LEXIS 101761 (D. Alaska June 30, 2017).

[6] Regulation (EU) 2016/679, of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data, art. 4(13), 2016 O.J. (L 119) 1 [hereinafter GDPR].

[7] GDPR, supra note 6, art. 9(1).

[8] Gendiagnostikgesetz [GenDG] [Genetic Diagnosis Act], July 31, 2009, BGBl. I at 2529 (Ger.), §§ 3, 8–10.

[9] Code de la santé publique [C.S.P.] [Public Health Code] arts. L1131-1 to L1131-7 (Fr.).

[10] Personal Information Protection Law of the People’s Republic of China (promulgated by the Standing Comm. Nat’l People’s Cong., Aug. 20, 2021, effective Nov. 1, 2021), arts. 28–29.

[11] See GB/T 35273-2020, Information Security Technology — Personal Information Security Specification § 3.2 & Annex B (P.R.C. Standardization Admin., effective Oct. 1, 2020) (enumerating “personal genetic data” as “personal biometric information” and as “personal sensitive information”); Bird & Bird, China Health and Medical Data Protection (I): Human Genetic Resources Information (Mar. 2022), https://www.twobirds.com/en/insights/2022/china/china-health-and-medical-data-protection-i-human-genetic-resources-information (concluding that HGR information “is likely to also constitute sensitive personal information” under PIPL “unless it has been anonymized”).

[12] Regulation on the Administration of Human Genetic Resources (promulgated by the State Council, May 28, 2019, effective July 1, 2019), arts. 2, 7, 11, 21–28; Detailed Implementing Rules of the Regulation on the Administration of Human Genetic Resources (Ministry of Sci. & Tech., effective July 1, 2023); Biosecurity Law of the People’s Republic of China, arts. 53–56 (effective Apr. 15, 2021).

The fifth report, which will be made available during the event, gathers contributions from 30 antitrust agencies. Each describes the computational tools it has designed, tested, or put into use over the past year.

Two developments run through this year’s edition. Language models have moved out of the productivity toolbox and into case work. Agencies have also started running experiments on algorithmic collusion rather than commenting on it. Thibault Schrepel and Alba Ribera-Martinez will present the tendencies in the report and highlight what competition agencies, practicing lawyers, and researchers should know about the use of computational tools in antitrust.

Yann Guthmann, Head of the Digital Economy Unit at the Autorité de la concurrence, and Elodie Vandenhende, Deputy Head of the Digital Economy Unit, will also present the French contribution and the work behind it. Teodora Groza, postdoctoral researcher at the University of Tübingen, will follow. Twenty minutes will be reserved for questions from the room and from participants joining online.

Program

• 4:00 pm, opening remarks, Thibault Schrepel
• 4:05 pm, presentation of the fifth Computational Antitrust Worldwide report, Thibault Schrepel and Alba Ribera Martínez
• 4:30 pm, the activities of the Autorité de la concurrence, Yann Guthmann
• 4:45 pm, Elodie Vandenhende, Autorité de la concurrence
• 4:55 pm, Teodora Groza, University of Tübingen
• 5:05 pm, questions
• 5:25 pm, the next five years, Thibault Schrepel and Alba Ribera Martínez
• 5:30 pm, reception

Registration

Registration is open over here.

Seating in the room is limited. Every registration will be placed on a waiting list. We will come back to you in the coming days to confirm your attendance. Registration for the Zoom session is not capped.

My earlier comparison treated the Meta verdicts as support for examining liability at the level of system architecture. The proposed settlement sharpens—and limits—that analogy. The two verdicts do not rest on identical legal theories, Nippon presents a different causal structure, and age assurance is a more bounded classification problem than identifying individualized professional advice. The comparison that carries across is narrower: legal restrictions can be translated into technical controls whose design, performance, and governance can be evaluated.

FROM RULES TO WORKING CONTROLS

1. Making the Prohibition Operative

My Nippon Life argument distinguished between placing professional-advice restrictions in terms and policies and designing the product to recognize when an interaction is moving from general legal information into individualized legal judgment. OpenAI’s current Terms tell users not to rely on output as a substitute for professional advice, and its Usage Policies prohibit tailored legal advice requiring a license without appropriate involvement by a licensed professional.

Meta faced a related implementation problem. Facebook and Instagram already required users to be at least 13, and Meta had long acknowledged that users could evade an age screen by misstating their age. The proposed consent judgment would establish an age-assurance framework applying technical methods to determine whether users should be treated as teens or children under 13. Its terms call for annual accredited testing, defined false-positive thresholds, certification under real-world usage conditions, anti-circumvention measures, use of reliable Apple and Google age signals, soft matching across related accounts, and continuing oversight.

The comparison to Nippon operates at the implementation level. A professional-advice restriction can be translated into technical requirements for identifying relevant characteristics of an interaction and changing the system’s behavior as those characteristics accumulate. General legal information might remain available under ordinary conditions. User-specific facts, an active dispute, requested legal conclusions, litigation strategy, or the generation of documents intended for filing could trigger progressively more restrictive responses.

The relevant inquiry in Nippon would therefore reach the adequacy of the controls OpenAI chose to implement. What interactions were they designed to detect? What thresholds changed system behavior? How were they tested, and what failure rates were observed? How did they perform when users supplied increasingly specific facts or sought documents intended for real-world use? What happened when the system could not confidently classify the interaction? The existence of a warning or policy remains relevant, but the architectural inquiry reaches what the product was designed to do when the restricted interaction actually occurred.

2. Bounded Error Instead of Perfect Classification

My original Nippon analysis contemplated a boundary that ChatGPT should stop crossing once the system recognizes that a user is seeking individualized legal advice. An architectural boundary of that kind can remain meaningful even when the system enforcing it cannot classify every case correctly.

The proposed consent judgment expressly permits bounded error in Meta’s age-assurance methods. It defines the U18 False Positive Rate as the percentage of actual 13-to-17-year-old users incorrectly identified or predicted to be 18 or older, excluding method circumvention. Within one year after the Effective Date, which follows court entry, commercially available methods must produce rates at or below 10 percent for users ages 16–17 and 3 percent for users ages 13–15. Proprietary methods must produce rates at or below 14 percent and 7 percent, respectively, within the first year after the Effective Date, and 10 percent and 5 percent, respectively, within the second year. Those tolerances sit within requirements for testing, demographic evaluation, circumvention monitoring, continuing oversight, and corrective action.

Applied to Nippon, the boundary between general legal information and individualized legal advice depends on context. A request for the text of a statute presents a different risk from a conversation involving an active lawsuit, user-specific facts, requested legal conclusions, litigation strategy, and documents intended for filing. A professional-boundary control could evaluate those signals collectively and restrict the system’s behavior as the probability and consequence of crossing the boundary increase.

The acceptable error rate would depend in part on the type and consequence of the error. Failing to recognize individualized legal advice and allowing the interaction to continue may present a different risk from mistakenly constraining a request for ordinary legal information. Severity, available verification, and the burden imposed by the safeguard could therefore inform the tolerances applied to each type of error.

Performance would also have to remain adequate as the system changes. Updates to the underlying model, system instructions, routing logic, user behavior, or methods of circumvention could affect a control that previously performed within acceptable tolerances. Relevant evidence could include observed error rates, known failure modes, changes following system updates, and the provider’s response when performance deteriorated.

The proposed consent judgment does not adjudicate a tort standard of care. It also expressly provides that it may not be construed to establish a standard of care or precedent in any non-participating U.S. state or international jurisdiction, and it contains no admission of liability. As an analytical matter, it illustrates how an obligation directed at a probabilistic technical system can be expressed through measurable tolerances, testing conditions, protective responses to uncertainty, monitoring, and corrective processes. For Nippon, the analogy is a more administrable inquiry into how OpenAI defined the boundary, evaluated its controls, handled uncertainty, and responded to demonstrated weaknesses.

3. From Safeguard to Assurance

My Nippon analysis focused on whether a known risk should have triggered an architectural safeguard. The proposed settlement adds a further layer by requiring evidence that the safeguard performs as intended:

known risk → architectural safeguard → measurable performance → independent validation → monitoring → corrective action.

Deployment alone would not satisfy the proposed settlement. Any method adopted for the framework must be tested annually by an accredited third party and certified under real-world usage conditions, including performance across diverse demographic groups and without reliance on training or tuning data. Proprietary methods are certified by Meta on the basis of its own testing, with the third-party provider evaluating whether that testing supports the certification, and remain under continuous oversight. Meta must continuously investigate new or previously undetected forms of method circumvention and, once confirmed, use best efforts to correct resulting false positives and modify the framework. An independent auditor reviews implementation of the injunctive terms, relevant performance data, and material gaps or weaknesses.

Applied to Nippon, the inquiry would extend to how OpenAI evaluated its professional-boundary controls and what its own records show: evaluation criteria, red-team results, classifier performance, false-negative rates, known bypasses, escalation thresholds, regression testing, incident records, model-version comparisons, and internal acceptance thresholds. Those materials could establish whether OpenAI had a reasonable basis for relying on the control when it was deployed and whether that reliance remained reasonable as evidence of its limitations accumulated.

The architectural question therefore reaches both the safeguard and the assurance process supporting it: whether OpenAI built an appropriate boundary, established that it worked with acceptable reliability, and responded when its own evidence showed otherwise.

FROM TECHNICAL CONTROLS TO LEGAL CONSEQUENCES

4. A Reasonable Alternative Design Need Not Specify the Code

A product-liability theory against OpenAI invites a predictable response: what, exactly, should OpenAI have designed instead? Framed too narrowly, that question can force a plaintiff into the untenable position of having to design a competing LLM, identify the classifier that should have been used, or specify the algorithm that would have prevented the harm.

The proposed consent judgment operates at a different level. The participating States did not prescribe Meta’s source code or require a particular age-assurance technology. Meta could use commercial products, proprietary systems, or combinations of technologies. The obligations run principally to the capability the system must provide and the performance it must achieve, leaving the technical implementation to Meta.

A regulatory settlement does not establish the legal standard for proving a reasonable alternative design in a product-liability case, and age assurance is a more bounded classification problem than determining when a general-purpose AI system has crossed into individualized professional advice. The agreement nevertheless supports describing a proposed alternative at the level of system capability and measurable performance rather than prescribing source code or a particular technical mechanism, subject to the proof required by the governing product-liability law.

Applied to Nippon, the reasonable alternative design could be that OpenAI should have maintained a tested system capable of identifying individualized professional-advice interactions with a defined level of reliability and routing those interactions into an appropriately constrained mode, with OpenAI choosing how to accomplish that result.

That formulation places the inquiry at the level where many AI safety decisions are actually made. The design of an AI product includes model weights, source code, system instructions, classifiers, routing logic, permissions, access controls, warnings, escalation procedures, monitoring, human-review requirements, restrictions on particular uses, and related operational safeguards. If the alleged risk arose from the absence or inadequacy of one of those controls, the reasonable alternative design can be framed in terms of the safer architecture that should have existed.

The question then becomes whether the developer could reasonably have designed the deployed system to recognize a defined category of risk and respond to it with an appropriate safeguard. Framing the proposed safeguard at that level may make an architectural-negligence theory less vulnerable to the objection that product-liability law would require courts to become AI engineers.

5. Designing for Uncertainty

Under the proposed Meta framework, newly created accounts whose age remains unassessed receive protective defaults for 14 days, and after that an unassessed user is treated as a Teen User for purposes of the agreement, subject to specified protections for users who stated an adult age. The design assigns consequences to uncertainty instead of allowing unresolved age to preserve an unrestricted adult setting.

The architecture at issue in Nippon could operate on the same principle. ChatGPT would not have to determine conclusively that a user is seeking legal representation or that the model is “practicing law.” The design question becomes whether identifiable characteristics of the interaction should cause the system to operate differently.

A system could respond normally to a high-confidence request for general legal information. As the exchange becomes individualized, through user-specific facts, an active dispute, jurisdiction-specific questions, requested legal conclusions, filing deadlines, or proposed courses of action, the system could progressively narrow what it is permitted to provide. An ambiguous individualized matter might trigger a constrained informational response. A sufficiently strong combination of signals could require additional warnings, verification, human review, referral, or a refusal to provide the requested conclusion.

Uncertainty about whether the interaction has crossed into individualized professional advice can itself become a design input, with greater uncertainty or greater potential consequence producing more restrictive behavior. The system may be unable to determine with certainty that an interaction has entered a regulated or high-risk domain, yet the developer can still specify how it should behave as the relevant signals accumulate. The same general design principle might inform controls for medical, financial, engineering, and other high-risk interactions, although the relevant boundaries, safeguards, and legal requirements would differ by domain.

6. Governance Records and Foreseeability

Traditional negligence analysis often reconstructs foreseeability after the harm has occurred by asking whether the manufacturer should have anticipated the use or risk that caused it. An age-assurance system operating under the proposed framework could generate contemporaneous evidence bearing directly on that inquiry. Under that framework, Meta could measure how often age assurance fails, how frequently users circumvent it, how often accounts are reclassified, how performance varies across populations, and whether particular controls reduce the conduct they were designed to address.

Those measurements can create a record bearing on actual knowledge. Repeated failures, circumvention patterns, evaluation results, incident reports, internal thresholds, and changes made in response to testing can show when a company identified a risk, how significant it understood the risk to be, and whether its safeguards were performing as intended. The same evidence may also bear on what the company reasonably should have anticipated as experience with the system accumulated.

OpenAI’s own systems could generate a comparable record. If OpenAI classifies legal interactions, records refusals, evaluates model behavior involving professional advice, tests attempts to circumvent safeguards, or measures policy violations, those systems may show how often the relevant conduct occurs and how effectively existing controls address it. A recurring pattern identified through the provider’s own evaluations or monitoring could make it easier to establish that the risk was known before the particular injury occurred.

The significance of that record depends on what the provider did with the information. Evidence that a company identified a recurring risk, measured its frequency or severity, recognized weaknesses in existing controls, and left those weaknesses inadequately addressed could support both foreseeability and breach. Evidence that the risk was rare, difficult to predict, or reasonably mitigated could support the opposite conclusion.

Governance therefore creates evidence as well as controls. Evaluations, logs, incident records, risk assessments, and remediation decisions can establish what the provider knew, when it knew it, and how it responded. As AI governance becomes more systematic, negligence claims may depend less on reconstructing what a developer should have anticipated and more on the record the developer created while operating the system.

7. A Presumption for Independently Evaluated Controls

The proposed consent judgment gives qualifying commercially available age-assurance methods a defined compliance presumption. When Meta uses and relies on such a method for a user, Meta is presumptively compliant with specified age-assurance obligations on the basis of the most recent qualifying third-party certification—but only if it can demonstrate the enumerated conditions: operating the method according to vendor and certification requirements, matching settings to those tested, not encouraging, facilitating, or knowingly permitting circumvention, not willfully ignoring events that impair efficacy, supplying complete and accurate information, and meeting related obligations. Meta remains free to develop its own age-assurance system, but a proprietary system receives no comparable presumption and stays under continuous oversight.

That presumption runs to compliance with the agreement’s own terms rather than to reasonable care in tort. A similar structure could nonetheless connect AI governance controls to the standard of care. Under one possible legal approach, a provider implementing an independently evaluated professional-boundary control that satisfies an accepted technical standard could receive a rebuttable presumption that the control was reasonably designed for that purpose. The presumption would attach only to the particular control and the risks it was designed to address. It would not establish that the AI product as a whole was safe, excuse deficient operation, or protect the provider after evidence showed that the control was no longer performing adequately.

A provider could instead use a proprietary control. Its freedom to innovate would remain intact, but it would have to establish the adequacy of the control through ordinary evidence rather than receive the benefit of the presumption. The law would therefore avoid prescribing classifiers, models, thresholds, routing logic, or other implementation details while still creating an incentive to use controls whose performance has been independently tested.

For professional-advice interactions, an accepted standard might define the categories of conduct the control must detect, minimum performance characteristics, testing conditions, documentation requirements, monitoring obligations, and the safeguards triggered at specified thresholds. OpenAI could decide whether to satisfy those requirements through classifiers, model-based routing, contextual analysis, separate constrained modes, or some combination of them. Independent evaluation would test whether the resulting control met the required performance criteria rather than whether OpenAI had selected a regulator’s preferred engineering method.

The presumption would remain tied to the conditions under which the control was evaluated. Material changes to the model, routing architecture, thresholds, connected tools, or deployment environment could require reevaluation. Evidence of systematic circumvention, deteriorating performance, or known failure modes could rebut the presumption even without a formal change in configuration. Certification would therefore provide evidence of reasonable care at a defined point and under defined conditions, rather than permanent immunity from scrutiny.

Standards would then carry a legal function beyond demonstrating that an organization takes governance seriously. Technical standards can define measurable expectations; independent assurance can establish whether a control satisfies them; and tort law can determine the evidentiary consequence of meeting them. Under such a regime, providers following an independently evaluated path could receive greater legal certainty. Providers adopting different approaches would remain free to do so, but would have to establish the controls’ adequacy through other evidence.

A rebuttable presumption of that kind would also preserve room for liability where the provider knew that a certified control was failing, operated it outside the conditions under which it was tested, or failed to respond as the evidence changed. If measurable governance requirements were independently evaluated and assigned defined evidentiary consequences, they could serve a legal function beyond internal governance or aspirational standards and become part of the framework for evaluating reasonable care.

WHAT META CHANGES FOR NIPPON

8. From Architectural Liability to Architectural Remediation

My March 30 piece treated the two Meta verdicts as evidence that liability can attach to choices embedded in a technology’s architecture rather than solely to the content appearing on the screen. In New Mexico, a jury found Meta liable on both claims the State brought under the Unfair Practices Act for misleading consumers about platform safety and endangering children. The next day, in a Los Angeles youth social-media addiction case, a jury found Meta negligent in designing or operating Instagram and Google negligent in designing or operating YouTube, found that negligence a substantial factor in the plaintiff’s harm, and found inadequate warnings. The two verdicts rest on different theories, and the Los Angeles findings bear more directly on architecture. Both directed attention to product design and the risks associated with design choices, which is why they offered a comparison for Nippon.

The proposed consent judgment identifies corresponding forms of architectural remediation: classification mechanisms bound by quantified performance tolerances, annual accredited testing under real-world usage conditions and across demographic groups without reliance on training or tuning data, protective defaults for accounts whose age remains unassessed, anti-circumvention obligations with best-efforts correction once circumvention is confirmed, continuing monitoring, and independent evaluation of implementation and material gaps.

Applied to Nippon, a claim that OpenAI should have prevented ChatGPT from crossing into individualized legal advice becomes more concrete when the proposed safeguard is described at that level, with OpenAI choosing the technical implementation. The verdicts and the proposed settlement therefore contribute different kinds of evidence. The verdicts support attention to architectural choices in assessing liability. The proposed settlement provides an example of how alleged architectural deficiencies can be addressed through defined, measurable, and reviewable controls without prescribing source code.

9. Causation and the Intervening User

In the Los Angeles case, the claimed harm arose from the plaintiff’s repeated interaction with features alleged to affect behavior, including recommendations, notifications, engagement mechanisms, social comparison, and continuous scrolling. The New Mexico enforcement action concerned Meta’s alleged misrepresentations about platform safety and conduct endangering children; it did not present the same user-specific causal chain.

Nippon presents additional causal steps. Nippon alleges the following sequence:

ChatGPT assistance → Dela Torre’s reliance and decisions → court filings → Nippon’s claimed litigation costs and legal expenses.

On Nippon’s account, the alleged sequence gives OpenAI a substantial causation argument. Dela Torre allegedly decided whether to accept ChatGPT’s suggestions, use the documents it generated, and file them with the court. OpenAI might argue that those independent acts, rather than the design of ChatGPT, produced the claimed injury.

The system architecture remains relevant to whether those acts should break the causal chain. According to Nippon’s complaint, ChatGPT allegedly analyzed a response from Dela Torre’s former lawyer, generated Rule 60(b) legal arguments, formulated a draft motion to reopen the settled case, and later assisted with filings in related litigation. A product designed to adapt its responses to a user’s circumstances and generate materials that enable the user to carry out a suggested course of action makes some resulting conduct more foreseeable than a product that merely supplies static information.

The causal inquiry can therefore account for the relationship between the system’s functions and the user’s subsequent acts. Where a design allows the system to assist through successive steps toward a course of action it has suggested, the user’s decision to act on that assistance may bear on foreseeability rather than automatically severing causation. The user’s independent judgment remains part of the analysis, as does the extent to which the system enabled the conduct that produced the alleged harm.

Agentic systems can shorten the causal chain further. A system may analyze a problem, recommend a course of action, and generate the materials needed to pursue it while leaving execution to the user. Once the system is authorized to act through connected tools and delegated permissions, fewer independent human decisions may stand between the system’s output and the resulting conduct. The system may send the message, submit the filing, initiate the transaction, or take another external action itself.

As AI systems assume more of the execution, proximate-cause analysis will have to account for the diminishing role of intervening human conduct and whether the resulting harm flowed from actions the system was authorized and designed to perform. This progression echoes the concept I called “iterative liability” in my July 2011 post, The Case of the Robot: Personal Liability, Part II — Iterative Liability. The liability analysis should adjust as an AI system assumes a greater share of the decisions and conduct producing an outcome.

WARNING → CONTROL → ASSURANCE

My March 30 piece described a movement from warnings toward architecture. The proposed settlement adds assurance: evidence that the controls perform as intended and a process for correcting them when they do not.

For the risks covered by the proposed settlement, Meta would implement controls subject to defined performance requirements, annual testing and certification, and specified monitoring and corrective processes, with independent evaluation of implementation and any material gaps or weaknesses. The proposed consent judgment remains subject to court approval, contains no admission of liability, and includes specified limitations on its use as a standard of care or precedent elsewhere.

Applied to Nippon, a professional-boundary control could be evaluated through the same sequence. OpenAI could define the interactions the control is intended to identify, establish acceptable performance levels, test the control under realistic conditions, measure failures and circumvention, monitor performance after deployment, and modify the control or the system’s permitted behavior when the evidence shows that existing protections are inadequate. A guardrail that performs adequately when introduced may deteriorate as the underlying model changes, users discover ways around it, new use patterns emerge, or the provider expands the system’s capabilities, so testing at deployment supplies only part of the relevant evidence.

A provider may have implemented a safeguard and still face questions about how the safeguard was evaluated, what failures it detected, how frequently they occurred, whether their consequences were significant, and what it did after learning of them. An architectural-negligence theory in Nippon could therefore reach the way a professional-boundary control was governed over time, and the adequacy of the architecture would depend in part on how the provider responded to evidence generated by its own system.

A warning communicates a boundary or risk. A control changes what the system can do when that boundary or risk is encountered. Assurance generates evidence about whether the control performs as intended and supports corrective action when it does not.

On August 11, 2026, the Colorado Department of Law filed the proposed Automated Decision-Making Technology & Conversational Artificial Intelligence Service Rules. The rules would implement Colorado’s revised ADMT Act and its new Chatbot Safety Act, both scheduled to take effect January 1, 2027. The proposal remains in formal rulemaking, and the Department is accepting comments through October 26, 2026, subject to extension if the hearing continues.

I set out here to examine the architecture underneath Colorado’s proposed requirements. Disclosure, human review, correction rights, age assurance, safety testing, incident reporting, documentation, and annual reports would all require the organization to retain enough knowledge about its deployed system to explain what happened and do something about it.

This analytical review was run through the AI Life Cycle Core Principles (AILCCP) framework. The legal obligations I cover come from Colorado’s statutes and proposed rules. The AILCCP adds a disciplined (i.e., repeatable) way of following those obligations across the AI’s life cycle and a systemic method of asking what an organization would need in place for them to work.

A note on capitalization: AILCCP principle names appear in initial uppercase, with the framework identifier in parentheses at first mention, as in Transparency (PR-033). Each principle is a defined term with its own scope, key questions, and mapped controls and standards. A lowercase term carries only its ordinary sense.

Consider, for example, data correction. Colorado’s proposal would require a decision to be reconsidered once the information behind it is corrected. Doing that may require the deployer to know what information was used, where it came from, what system configuration produced the original outcome, and whether that decision can be reproduced or independently revisited. Adverse-decision explanations likewise depend on traceability and human-review records; developer documentation affects what deployers can later explain; age-assurance requirements implicate privacy and retention choices; and product updates can alter safeguards that previously worked. Read together, the provisions operate across the AI life cycle rather than as discrete compliance requirements.

The AILCCP also kept forcing questions of proof. For each requirement, I asked what an organization would actually need to produce if challenged. Could it identify the configuration that produced a particular decision? Recover the data used? Identify its source? Show who reviewed the outcome and what authority that person possessed? Produce the testing that supported a safety claim? Demonstrate what changed after an incident?

Broad principles such as Transparency (PR-033), Accountability (PR-002), Human-Centered (PR-017), Privacy (PR-023), and Safety (PR-029) translate into specific evidentiary demands when framed this way. For Transparency, that may mean reconstructing an individual decision. For the Human-Centered principle, an independent review process with access to primary evidence. For Safety, evidence that a control continued to perform after a model or product update.

The analysis kept arriving at one question: What end-to-end governance system does an organization need to demonstrate, test, explain, contest, correct, and improve the behavior of its deployed AI? The answer begins with the ability to reconstruct what happened.

The AILCCP also operates as a dynamic restatement. It identifies the norms emerging from legislation, regulation, standards, and enforcement as defined principles, and the framework is refined as those sources develop. Colorado’s proposal proves to be such a source. I used the AILCCP to test whether the proposal made certain governance capabilities concrete enough that the framework should name them expressly, and that exercise produced concepts such as Decision Reproducibility, Independent Review Firewall, Continuous-Interaction Risk Assessment, and Feasibility as an Evidence Claim. They are now candidate refinements for the framework, and the analysis behind each runs through what follows.

Decision Reproducibility

Under Colorado’s proposal, a person who receives an adverse outcome from covered ADMT would be told what produced it. The deployer would have to disclose the system’s role, the involvement of human reviewers and other systems, the principal reasons for the outcome, relevant inferences or scores, certain factors that automatically produced denial, and other information relevant to the decision. A deployer would fail the explanation requirement if it could not explain how the ADMT materially influenced the decision or could not accurately explain the principal reasons for the adverse outcome.

A system model card may describe how a model generally operates, but a person challenging a particular decision needs the record of that decision, and for many contemporary AI systems that record extends well beyond “model version.” The meaningful unit of reconstruction is the configuration that produced the outcome, meaning the system’s complete operating state at the moment of the decision. The AILCCP surfaces that unit through Accountability, which asks whether an output can be traced to the data, models, versions, and owners behind it. In a contemporary system, the configuration can span the model and checkpoint that were running at the time, the system prompt, retrieval sources, feature flags, classifier versions, third-party data, scoring thresholds, orchestration logic, and the rule governing when the matter was sent to a person.

This changes explainability from merely a communications exercise into a deployment capability and potential competitive differentiator. The AILCCP anticipates that shift in Explainability (PR-012), which pairs explanations with the data quality and provenance controls behind them. If a system cannot preserve enough of its operating state to explain an individual outcome, the failure predates the adverse decision.

The same preservation problem runs through Colorado’s data-correction provisions, which contemplate access to the personal data used in a consequential decision, including individual scores, classifications, predictions, recommendations, inferences, and inputs. When inaccurate data is corrected, reconsideration means running the decision process again with the corrected information. Where possible, an adverse outcome would also be stayed pending correction or meaningful human review.

A conventional database correction may be inadequate in that setting, because reconsideration can require the historical context surrounding the original decision. A continuously updated system, for example, may produce a different result when run weeks or months later because the model, retrieval corpus, threshold, or surrounding workflow has changed. Re-running today’s system does not necessarily tell us what corrected information would have done to yesterday’s decision. The organization must therefore preserve enough of the decision environment to revisit the outcome properly or maintain an independent means of reconsidering it when faithful replay is no longer feasible.

Independent Review Firewall

Reconstruction, however, is only useful if someone with authority can act on what it shows. That someone is the human reviewer, and Colorado is mindful of how easily “human in the loop” becomes an inert label rather than a safeguard. The proposal would attach substance to the person’s role by requiring, where feasible, a reviewer who is independent of and not subordinate to the original decision-maker, possesses subject-matter understanding proportionate to the consequences, receives appropriate training, and has actual authority to approve, modify, or override the decision. The proposal would also shield the reviewer from management pressure and retaliation, protections that keep the required independence from eroding in practice.

A person may technically participate in a process while having little practical ability to alter it. A reviewer who sees only the machine’s recommendation, lacks the underlying evidence, is measured primarily by throughput, or understands that deviations are unwelcome may add a human signature without supplying meaningful review. The AILCCP probes that failure through the Human-Centered principle, which asks whether oversight is sustainable and what prevents automation bias. A credible review process therefore also turns on what the reviewer sees, what the affected individual can submit, and how the reasoning behind the final result is preserved.

A system that recommends an outcome anchors the reviewer, pulling independent judgment toward whatever the machine already concluded, and that risk explains why the proposal would bar ADMT from assisting in the meaningful human review. A tool used only to retrieve records, translate material, provide accessibility support, or organize a file presents a different issue. The proposal’s anti-anchoring objective is understandable, but the boundary between decisional assistance (shaping or recommending the outcome) and bounded clerical assistance (retrieving, translating, or organizing material) will need to be carefully defined in the final rules.

The evidence a reviewer or a deployer needs is often created elsewhere, because a modern AI system combines the work of several companies. A foundation-model developer may supply the underlying model. Another company may adapt or integrate it. A deployer may add proprietary data, prompts, and business rules. Other vendors may provide identity information, scores, retrieval sources, or safety tools. The affected person, however, sees one system.

Colorado’s proposal responds by requiring governance information to move downstream. Its provisions concerning developers and midstream developers call for documentation of intended and inappropriate uses, categories of data, known limitations and risks, monitoring, relevant factors, and use restrictions. Midstream developers, in turn, carry obligations for the developer documentation they obtain and transmit.

At each handoff, someone should know what documentation was received, what remains unknown, which uses are permitted, what conditions affect reliability, which version is being supplied, when changes will be communicated, and who is responsible for preserving downstream evidence about actual outcomes. The AILCCP frames these requirements through Data Stewardship (PR-005), which asks whether provenance is tracked end-to-end. Vendor assurances about “responsible AI” carry little weight against that question. A deployer facing a contested decision needs usable evidence, and if it disappeared somewhere between developer, integrator, and deployer, governance failed at the handoff.

Continuous-Interaction Risk Assessment

The chatbot provisions shift the focus from single decisions to sustained interactions. A safety test built around individual prompts and outputs misses what a conversational system can do across time. Memory, personalization, and repeated interaction may gradually produce a relationship that a single exchange cannot capture.

In evaluating whether a service falls within provisions concerning emotional companionship or emotionally dependent interaction, the proposal looks to features including anthropomorphic personalization, assigned identity attributes, memory of earlier exchanges, sycophantic or continually validating behavior, engagement-reinforcement mechanisms, and whether safeguards remain effective after model or feature changes.

Any one of those features may be benign in one product design and consequential in another, and the way those features compound across days or weeks may matter more than any single feature on its own. Testing should therefore follow representative interactions over time and after material changes to the system. The AILCCP presses one step further through Consent (PR-006), which treats consent as valid only while the user’s understanding still matches how the system actually behaves. A relationship that deepens over weeks can outgrow the understanding the user gave at the first exchange, and consent obtained then may not cover what the service has since become.

Feasibility as an Evidence Claim

Nowhere is the obligation to keep testing more visible than in the treatment of technical feasibility, which Colorado would evaluate with reference to available technologies, comparable industry safeguards, alternatives, periodic reassessment, realistic testing, performance after updates, and documentation of rejected approaches. A design choice justified by the state of technology in January may deserve another look in July.

A metric can be calculated correctly and still create the wrong incentive, and the proposal’s crisis-referral reporting for conversational AI is the clearest example. Operators would have to report the number of referrals relative to conversations and the proportion that actually involved suicidal ideation or self-harm risk under an evidence-based method. The obligation would extend to detection and response protocols, safety measures, testing, reliability and efficacy over time, age estimation, and interactions involving minors and prohibited sexual content.

Suppose a system becomes more conservative about issuing referrals. The proportion of referrals associated with genuine risk might improve because borderline cases are no longer referred. At the same time, the system could be missing more people who actually need intervention.

That is why governance needs paired measures. The AILCCP devotes a principle to the problem, Metrics (PR-020), which warns that optimizing what is measured can degrade what is not. Referral accuracy should be read alongside measures that expose what it cannot show, among them missed-risk rates, detection sensitivity on a governed evaluation set, false positives, time to escalation, performance changes after updates, and, where appropriate, variation across languages or interaction contexts. Measurement choices are themselves controls, and they warrant the same scrutiny as any other safeguard.

Taken together, Colorado’s proposal points toward an understanding of AI governance that is considerably more operational than a collection of policies, principles, or one-time assessments. On that understanding, a deployed AI system should leave enough evidence to answer questions about a real event. Which configuration produced the outcome? Could the decision be revisited once inaccurate data was corrected? Did the reviewer possess genuine authority? Did a safety control continue to work after the model changed? Did the user’s consent still cover what the service had become? Could the deployer obtain what it needed from the developer? These are difficult questions to answer with the policies and one-time assessments written before launch. Each asks what the system actually did after launch, and only records kept while it was running can answer that.

Any safeguard, whether mandated by Colorado or defined in the AILCCP framework, is effective only if the organization can use it when something bad happens. Each AILCCP principle turns that condition into something testable, defining a capability, pairing it with probing questions, and identifying the evidence that proves the organization has it. The framework took Colorado’s long list of requirements and rendered it into capabilities an organization either has or lacks, which recasts compliance as an infrastructure question rather than a provision-by-provision chase. Someone will need an explanation, so decisions must be traceable. A decision will need to be revisited, so outcomes must be reproducible. An affected person will need a genuine second judgment, so the reviewer must be independent. And once the system changes, yesterday’s safety evidence no longer describes it, so the safeguards must be retested, and their design reevaluated, after every material change. Those are the capabilities Colorado could codify. And because the AILCCP distills legislation, standards, and enforcement from many jurisdictions, an organization that builds them for Colorado is building them for whatever comes next.

This divergence exemplifies what I call the accountability lag: the widening distance between the rate at which AI expands consequential action and the rate at which institutions can govern that action. The lag arises because technical capability and institutional responsibility follow different scaling patterns.

Bruce Schneier’s four dimensions of AI advantage help explain the technical side of that difference: speed, scale, scope, and sophistication. AI can complete an activity faster, repeat it across more settings, extend it into more domains, and incorporate more interacting factors. As those advantages compound, the activity surrounding the technology begins to change. Yet the institutions governing that activity do not necessarily undergo the same transformation. Operational capacity acquires the four S’s; responsibility remains tied to scarce resources such as attention, authority, expertise, evidence, and enforceable consequences.

The imbalance can be expressed in terms of marginal cost. Producing another AI-generated output or action may cost very little. Meaningful accountability generally requires attention to the particular case. Each disputed diagnosis, denied benefit, defective filing, or unexplained decision may demand individual examination even when the underlying decisions were produced in bulk.

Organizations respond to this imbalance by preserving the appearance of responsibility. The favored instrument is human-in-the-loop review, a close cousin to the human oversight requirement in the EU AI Act. A human decision-maker, reviewer, or supervisor remains formally assigned. As deployment expands, however, that individual may lose the practical capacity to meaningfully examine the system’s work.

Return to scientific publishing. The same cost collapse that lets a fabricated study pass for a real one increases the volume of material entering the publication system. Journals still depend on editors, reviewers, access to evidence, and eventual replication. The productive side of the system becomes scalable while its principal integrity mechanisms remain constrained.

Medicine reveals the same mismatch at the level of professional judgment. An imaging model may improve diagnostic capacity by comparing each scan with patterns learned from vast datasets. If the model also enables a large increase in case volume, the clinician’s nominal responsibility remains unchanged while the conditions needed to exercise that responsibility deteriorate.

Knowledge codification extends the problem from individual decisions to organizational assumptions. AI can draw patterns from manuals, databases, communications, and employee practices, making dispersed knowledge widely available. Greater reach also allows an error or hidden assumption to propagate throughout the organization before anyone recognizes it.

Agentic systems add a temporal dimension. Responsibility is ordinarily assigned at identifiable decision points. An agent may collapse several of those points into a continuous sequence of execution. By the time a person encounters the result, the relevant opportunity for intervention may have passed. Accountability must therefore be designed into the architecture of action: where review occurs, where authority enters, and where execution pauses pending judgment.

This architecture also determines whether AI genuinely empowers the people affected by it. Speech recognition may increase independence for someone with limited mobility. A legal tool may make unfamiliar procedures accessible. A customer-service chatbot may instead expand an institution’s processing capacity while reducing a customer’s access to someone authorized to resolve the problem. The relevant measure is the distribution of agency: who can act, contest, obtain an explanation, and compel correction.

The agency a person can exercise against a consequential decision measures the accountability the institution actually delivers. A decision that cannot be challenged leaves the affected person without agency and producing that decision cheaply and in bulk does not shrink what the institution owes for it. Cigna’s PXDX claim-review system, for example, reportedly let company physicians deny health-insurance claims in batches, spending an average of 1.2 seconds on each. Patients received a vaguely worded denial letter and entered an appeal process that required additional review before the decision could be disputed. Meaningful accountability therefore requires access to relevant evidence, authority to intervene, viable opportunities for contestation, and the institutional capacity to correct the process producing the outcome.

Whether those mechanisms keep pace can be measured through accountability elasticity, a term borrowed from economics: the degree to which an institution’s capacity for explanation, intervention, contestation, and correction grows as the system’s operational capacity grows. The lower the elasticity, the wider the lag. Each expansion then adds governance debt, activity the institution produces faster than it can meaningfully own.

No formula is needed to take the measurement. If use of an AI system increased one hundredfold, which mechanisms of accountability would increase with it? Who would review the additional decisions? Who could stop the system? Who could investigate a failure, correct its source, and answer to the person affected?

An institution without concrete answers has calibrated its oversight to a smaller and slower system than the one it now runs. That distance is the accountability lag, and it will not close on its own. The vendor ships a faster model and employees put it to use. Accountability grows only when someone builds it. No one is in charge is the inevitable result when an institution deploys the first without building the second.

***

Note: Accountability is one of 37 life cycle core principles. You can read more about it in the AI Life Cycle Core Principles Explorer.

That is no longer what the AILCCP is and you can already see the different here.

The standards and best practices that inform the AILCCP are the raw material from which the framework is built. As of August 2026, the AILCCP maps 48 published standards to its 37 principles, drawing from ISO/IEC, IEEE, NIST, OWASP, and others. (The framework also tracks standards currently under development and once published, each of these will land on principles the framework already carries.)

Each published standard is assigned to the principles (which always appear in initial upper case) it most directly supports. For example, ISO/IEC 42001:2023, the AI management system standard, maps to Governance, Accountability, Privacy, Cooperation, and Enabling; the NIST AI Risk Management Framework maps to Governance, Security, Metrics, and Accountability; and IEEE 7003-2024 on algorithmic bias considerations maps to Bias, Fairness, Equity, and Ethics. The mapping lets a user trace any principle to the published authority behind it. A control sourced to ISO, NIST, or IEEE is not defended on my say-so, and that is what an auditor, a regulator, or a board member asking why we are doing this actually needs to hear.

The AILCCP was designed from the outset for a wide range of constituencies. Developers measure alignment with established norms; end users reference it in licensing, due diligence, and application maintenance; regulators guide enforcement; lawmakers draft legislation that is more relevant, clear, and practical; and lawyers, boards of directors, executives, auditors, and procurement officers each hold a piece of how AI systems are built, deployed, and overseen. A list, however well-constructed, serves these constituencies unevenly. It gave everyone the same flat surface, regardless of what they need to do with it.

The AILCCP is already substantially different than the original and is already undergoing structural changes that resemble the architecture of a restatement of the law. A restatement synthesizes authority courts have already made, turning accumulated holdings into clear rules, with comments that explain a rule’s purpose and illustrations that show how it applies. The AILCCP rests on published standards and best practices because the case law it would otherwise synthesize is still being made. The principles are gradually acquiring structure, hierarchy, and cross-referencing. The definitions are being disciplined to declare operative standards rather than describe concepts. A commentary layer is being built out to carry the analytical and doctrinal weight that the original footnotes could only gesture toward, tracking doctrine that is forming rather than doctrine that has settled. 

The work is ongoing, with updates as frequent as daily in some cases. It is a manual process. Every addition to the framework is checked against primary sources—statutes, regulations, enforcement actions, and published standards—Bluebook-cited and logged in a refinement record that tracks what changed, why, and what remains unresolved.

Three examples show how the transformation is taking shape.

From definition to structured standard. The original post defined Privacy as a principle requiring that AI systems respect individuals’ personal data in alignment with legal requirements and societal expectations. That definition is equally accurate and inert. The evolving framework is working toward a structure in which the principle states an operative standard, a comment explains what the standard requires and what it does not, and a separate commentary layer tracks the doctrinal developments that bear on application.

From principle to life cycle signal. The original list did not inform when to apply a principle. Accountability appeared alongside Transparency and Privacy without any indication of which phases of an AI system’s development required its attention. The current, evolving framework maps each principle to the phases of the AI life cycle where it carries the most operational weight, from scoping and design through deployment, operations, and decommissioning. Accountability, for example, is most consequential at scoping and design, pre-deployment review, and operations and monitoring. For an engineer, that signal identifies when to act. For a regulator examining a deployment, it identifies what evidence to request at each phase. For a board overseeing an AI program, it identifies the oversight gates that require documented approval. The life cycle mapping converts the framework from a reference document into an operational tool that different constituencies can use for different purposes without requiring each to derive the timing implications independently.

From assertion to evidenced position. The original post stated that AI systems should be interpretable. The new framework supports that assertion with a three-level model, distinguishing the ability to (i) verify what the system is doing, (ii) evaluate whether a recommendation fits a given context, and (iii) learn from the system’s reasoning. The third level, learning from AI rather than merely auditing it, reframes interpretability from a property of outputs to a property of the human-machine dialogue. That reframing changes what interpretability controls are adequate, what disclosure obligations attach, and what compliance evidence is meaningful. The AILCCP does not assert this as settled. It cites the authorities, flags the emerging nature of the position, and leaves the contested question open for the commentary to track as the doctrine develops.

These examples share a common denominator. In each case, the original principle provided a correct but static statement of an oversight norm. The evolving framework is adding the layers that allow each constituency to use that norm. The temporal signal, the evidentiary grounding, and the doctrinal context.

The AILCCP applies the method of a restatement to AI oversight before the field has produced the volume of enforcement, litigation, and regulatory guidance that would allow a true restatement to be written. The list was a starting point. I have spent the time since working out what a more rigorous structure requires and building it one principle at a time. A list can be written in an afternoon. A restatement cannot.

The AI Life Cycle Core Principles is an ongoing research project. The framework is maintained and updated as standards, enforcement actions, and academic literature develop.

Note: Initial capitals are used throughout this post to identify AILCCP principles. Parenthetical identifiers give each principle’s number in the framework. The term “oversight” is used instead of “governance” due to the framework’s use of that term as a principle.

The framework update begins with the premise that AI oversight must reach whatever can carry an error into harm.

In the brief’s account, two objects have organized the oversight conversation for much of the recent AI era. The first is content, meaning what a system generates. Content rules ask whether an output is accurate, lawful, fair, deceptive or harmful. The second is action, meaning what a system is permitted to do. Action rules ask what authority has been delegated, which decisions require approval, who provides oversight and when the system must stop.

That framework works for systems whose principal effects arise from the information they produce or the authority they exercise. The brief argues that policy built for generated content and autonomous decision-making does not fully reach world models.

A world model builds a working representation of a physical environment and predicts how that environment will change in response to action. Its representation can become a substitute for the world in training, testing and decision-making. If the world model makes an error, the failure is, as HAI describes it, “a counterfeit of physical reality that can look flawless while being wrong.”

Once a flawed environment is reused, the error can spread through every system trained within it, every certification based upon it and every decision informed by it. Because that spread can evade detection, validation has to precede use rather than follow failure. If a system is trained, tested or guided by a learned environment, whoever deploys the system must establish the validity of that environment for its intended use before the system is trusted.

When does validation attach? It could attach by model class, beginning once a system counts as a renderer, a simulator or a planner. Another option is to attach validation to the intended use instead, beginning when the system’s inferences move physical equipment, inform a safety decision or support a certification that others rely on. The AILCCP update takes the second approach, which follows the brief’s call for safeguards matched to how and where a system is used. The principles below key validation to intended use.

The AILCCP now translates the third object into a reference that can be used in system design, contracting, procurement, auditing and incident investigation. Because environment validity cuts across the AI system’s life cycle, the update appears within several principles.

Fidelity (PR-014) requires the learned environment to be validated for its intended use.

Safety (PR-029) requires real-world validation before a simulation-trained system is deployed. It also requires safe behavior when the system encounters conditions the learned environment omitted, simplified or represented incorrectly.

Metrics (PR-020) requires measurements of physical validity, transfer to real conditions and safe performance at the boundaries of the environment’s demonstrated competence.

Accountability (PR-002) requires a time-stamped record connecting what the system perceived, the state it inferred and the action it took. The record must also identify the relevant model and simulator versions, safety-layer decisions and human interventions.

Security (PR-030) requires the system’s learned picture of its surroundings to be treated as an attack surface and defended accordingly. An adversary may poison the data used to build the environment, manipulate the sensors that update it or corrupt the simulation used for training and certification.

Together, these principles settle what needs to be validated, which evidence suffices and how the system must behave once its learned environment no longer deserves confidence.

World models remain early and fast-moving. Their functions overlap, their architectures are changing and the science needed to evaluate them remains incomplete. That uncertainty makes a rigid regime premature. The AILCCP can supply durable questions while the required evidence sharpens in sync with the technology.

What must the learned environment represent for this use? Which real-world evidence establishes that it does? Where does the representation cease to be reliable? Has transfer been tested independently? What happens when the system reaches the edge of the environment’s competence? Who approved the evidence, and who can stop deployment when it proves inadequate? A deployment that cannot answer them has substituted an assumption for evidence.

This shortens the distance between inference and consequence. An error can become motion before a person has an opportunity to examine it. The resulting risks are more immediate, but they remain susceptible to life cycle oversight. The AI Life Cycle Core Principles (AILCCP) provide a way to identify where those risks arise, how they move through the system and what evidence is needed to control them.

Note: Initial capitals are used throughout this post to identify AILCCP principles. Each principle carries its framework identifier in parentheses at first mention. Where a word that also names a principle appears in lowercase, it carries its ordinary meaning.

The Physical AI Stack

Physical AI encompasses several technologies performing different roles.

World foundation models, such as NVIDIA Cosmos, learn representations of physical environments and can generate simulations or synthetic data for downstream development. Embodied reasoning models interpret scenes, identify objects, predict trajectories and plan actions. Vision-language-action models, such as Gemini Robotics, convert observations and instructions into action sequences. Controllers, sensors, actuators and safety systems determine how those sequences are executed.

These components may be combined in many ways. A world model may support offline training, runtime planning or both. A reasoning model may serve as a perception layer, generate a trajectory or help control a robot directly. The relevant risk therefore depends on the component’s role, the environment in which the system operates, its degree of autonomy and the safeguards surrounding it.

This system-level interdependence means the AILCCP can only be meaningfully applied when the AI system is understood end-to-end as a single interacting whole. A “system-level view” therefore requires evaluating how models, data, hardware, sensors, deployment context and human operators jointly shape behavior in practice, rather than assessing any component in isolation.

Training and Simulation

World foundation models can reduce the cost and danger of collecting physical-world data. They allow developers to expose a robot or vehicle to rare conditions without creating those conditions in the real world. Their value depends on whether the simulation preserves the features that matter when the system is deployed.

The central problem is the sim-to-real gap. A simulated environment may simplify friction, visibility, object behavior, sensor noise or human movement. A downstream model trained on those simplifications may perform well in testing and fail when reality supplies conditions the simulation lacks.

Accuracy (PR-003) and Fidelity (PR-014) therefore require more than success within the simulated environment. Developers must establish that training and evaluation conditions represent the intended operating environment, identify material departures from reality and test performance against real-world data. Validation should include rare but consequential scenarios, not merely average task completion.

Transparency (PR-033) requires records of the models, datasets, assumptions and simulation versions used to produce training evidence. Permit (PR-021) addresses the authority to use human demonstrations, teleoperation logs, proprietary sensor data and other protected material. Efficiency (PR-008) and Sustainable (PR-031) require an assessment of whether the compute devoted to simulation produces safety or performance gains commensurate with its cost.

The world foundation model’s role is upstream, but its oversight consequences can persist throughout the system.

Integration and Verification

A model that performs well in isolation may become unsafe when connected to hardware. Sensor placement, latency, payload, actuator limits, controller behavior and the physical workspace can alter the consequences of the same model output.

Safety (PR-029) and Reliability (PR-026) must therefore be evaluated at the system level. Applicable controls may include redundant sensing, verified safety envelopes, restricted operating conditions, collision avoidance, speed and separation monitoring, emergency stops and fallback controllers. Governance (PR-016) determines who approves the integrated system, which evidence is required and who has authority to stop deployment.

A separate runtime layer that evaluates a proposed action before execution can be useful. Its effectiveness depends on the independence and quality of the evidence it receives. If both the action model and the safety layer rely on the same camera view obscured by the same pallet, the second layer may confirm the same mistaken account of the world. Reliable protection may require additional sensors, conservative behavior under occlusion or physical restrictions on the workspace.

System-level assurance must therefore be achieved through coordinated controls spanning training, integration and deployment rather than reliance on any isolated safeguard.

Deployment and Operation

Physical operation makes time an important oversight variable. Detection, escalation and intervention must occur quickly enough to prevent harm. A control that works for an application generating text may be inadequate for a robot operating beside workers.

Safety and Reliability therefore require continuous monitoring, detection of unfamiliar operating conditions and a defined safe response. That response may involve slowing, stopping, requesting human assistance or switching to a simpler controller. Human oversight must be supported by sufficient information, time, training and authority. A person nominally assigned to supervise a system offers little protection if intervention is practically impossible.

Transparency also takes a more operational form. The evidentiary record may need to preserve the relevant sensor state, model and controller versions, proposed action, safety-layer decision, actuator command, intervention and system response. Near misses matter as much as completed incidents because they reveal hazards before injury occurs.

Maintenance obligations control what happens after deployment. Model updates, hardware changes and new operating environments can alter system behavior. Each material change requires regression testing, renewed validation and a documented decision that the system remains within its approved operating conditions.

Accountability Across the Stack

A physical AI system may involve a world-model provider, a robotics-model developer, a systems integrator, a hardware manufacturer and the organization operating the finished system. This complicates investigation and responsibility, but it does not eliminate either causation or accountability.

Accountability (PR-002) and Governance require duties to be assigned before deployment. Agreements among participants should address testing access, technical documentation, evidence retention, update notices, incident cooperation and authority to suspend operation. The organization deploying the system must understand the integrated product well enough to determine whether its risks are acceptable. Supplier complexity cannot become a substitute for that judgment.

The AILCCP’s objective of maintaining no gap between system behavior and the deployer’s liability is especially important here. Its implementation requires responsibility to be distributed deliberately across the stack while preserving a clear point of responsibility for the system placed into operation.

What Physical AI Demands of the AILCCP

Physical AI changes the emphasis placed on different principles at different stages. Accuracy and Fidelity govern the relationship between simulation and reality. Safety and Reliability govern integration and operation. Transparency supplies the evidence needed to test, monitor and reconstruct system behavior. Accountability and Governance connect those obligations across organizations and throughout the life cycle.

The necessary next step is a physical-AI control mapping: the controls and evidence that make each principle concrete at each layer of the system. That mapping should remain sensitive to the application. A warehouse arm, surgical robot and autonomous vehicle do not require identical controls merely because each uses a vision-language-action model.

World foundation models expand the environments in which physical systems can be trained and tested. Physical AI shortens the path from model output to real-world consequence. Together, they raise the level of assurance required across the AI life cycle. The system must be assessed through its training environment, integration, operating limits, runtime safeguards, update process and allocation of responsibility.

Life cycle oversight proves its value by following risk from the assumptions built into a simulated world to the moment a machine acts in the real one.

Préface de l’ouvrage de Ledy Rivas Zannou

Les enjeux juridiques de la consommation mobile en Afrique de l’Ouest :

analyse du processus contractuel

Vincent Gautrais

Professeur, CRDP, Faculté de droit de l’Université de Montréal

Élégance. Professeur Ledy Rivas Zannou, avec l’élégance qu’on lui connait, à cru bon de me demander, à titre de directeur de thèse, de préfacer le présent ouvrage que j’ai eu le plaisir d’accompagner. Une élégance qui ne s’impose pas; en fait, les thèses, paradoxalement, quand elles sont bonnes, a fortiori très bonnes, demandent bien peu au directeur. Bien davantage, elles nous autorisent d’être à une place privilégiée. Comme au théâtre, on siège au premier rang, dans la corbeille, au-dessus de la fosse.

Contrat. Évidemment, au départ, il y a le contrat que nous affectionnons dans la mesure où il constitue l’institution à laquelle nous avons apporté ce regard d’introspection qu’une thèse exige. Ce « pilier du droit » selon Carbonnier, Ledy Zannou l’a d’abord forcément envisagé de l’intérieur, dans toute sa technicité juridique. Mais il y ajouta aussi une approche plus aérienne, en tenant compte de son évolution historique et de l’usage que l’on en fait. Notamment, et c’est particulièrement clair dans sa thèse, on y voit d’abord la généralisation du phénomène selon lequel l’acte se voit substitué à un processus. Ensuite, et peut-être surtout, qu’on ne peut l’appréhender sans proposer une mise en contexte, plus extérieure, sans laquelle ce processus contractuel ne peut être compris. Cette mise en contexte, tout est là, s’exerce en tenant compte de trois paramètres principaux sur lesquels Ledy Zannou prend le soin de développer la notion de contrat : le numérique, la consommation mobile et l’Afrique.

Numérique. Bien sûr, ce contrat, il en a vu d’autres. A bien des égards, la thèse le montre, il s’agit d’une notion qui est d’une plasticité telle que sa transposition dans des contextes aussi distincts soient-ils s’opère ma fois sans trop d’anicroches. Il est donc étonnant de constater dans ce travail comment des principes pluriséculaires sont somme toute assez transposables dans des environnements nouveaux. Le droit, science de la réaction, offre la sagesse inhérente à ce regard en arrière. Néanmoins, le numérique oblige aussi à contorsion. Sans trancher entre simple évolution ou révolution, entre Easterbrooke[1] et Lessig[2], ceci variant selon celui qui scrute, il bouleverse tout sur son passage. Il requiert forcément à la jurisprudence d’adapter son regard. Un regard qui évolue au fil du temps, intégrant avec plus ou moins de spécificités le contexte numérique, comme le montre l’évolution entre les arrêts de la Cour suprême du Canada de 2007 (Dell Computer)[3] à 2017 (Douez c. Facebook)[4]. Le numérique implique donc son lot de faits que le droit en général et le contrat en particulier se doit de considérer.

Consommation mobile. Cette intégration revendiquée des valeurs associées au numérique, qui apparaît dans l’intitulé même de la seconde partie de l’ouvrage, doit aussi être envisagée en tenant compte de la double spécificité que constitue la consommation mobile. La consommation donne lieu dans un premier temps à l’élaboration de contrats qui usent et abusent de la position de force dans lesquels les marchands se trouvent. Une vulnérabilité du consommateur donc qui, en second temps, est même amplifiée par l’omniprésence du téléphone comme support du processus contractuel. Cette technologie omniprésente en Afrique est en effet d’une piètre qualité communicationnelle lorsque l’on ne fait que transposer les façons de faire propres au papier ou même à l’écran d’ordinateur. Professeur Zannou y va donc de propositions adaptatives afin que la volonté propre à tout contrat ne soit pas une chimère, et ce, en intégrant tant des considérations techniques que communicationnelles.

Afrique. Et puis l’Afrique enfin, celle chère à Ledy, constitue un merveilleux terrain d’études. Avec ses spécificités, économiques, culturelles, communicationnelles, elle ne peut se limiter à reproduire les injonctions des pays du Nord. L’uniformisation du droit n’est pas une panacée; les droits africains se doivent d’identifier et appliquer les caractéristiques propres à ce continent.

Rapports de force. En relisant la thèse de Ledy Zannou, et à travers l’analyse fine de ce triple regard, il me semble clair que cette mise en contexte tient dans les trois cas d’une altération des rapports de force. Le contrat est affaire de pareilles tensions où numérique, consommation et Afrique sont des occasions de remettre en cause les principes protecteurs avec lesquels il se doit de composer. Le contrat, selon le professeur Zannou, tout comme les autres branches du droit, est pétri de considérations géopolitiques qui ne peuvent être ignorées.

École de Montréal. Il y a donc dans cet ouvrage une vraie et belle thèse. Thèse qui ressemble à son auteur qui, je le suppute, risque de servir de fondation à sa carrière qui s’amorce. Une thèse qui s’est aussi je crois grandement nourrie par les influences du centre de recherche, le CRDP, qui l’a accueilli. Un centre qui depuis de nombreuses décennies, au-delà du seul droit positif, analyse justement en contextualisant l’intégration des valeurs, des institutions et des normes[5]. Ledy, du fait d’une grande ouverture, d’une belle curiosité intellectuelle, a su admirablement s’approprier ces influences tout en traçant son propre chemin.

Préface. Il est bien tard dans ce très court texte de se demander à quoi sert une préface. Évidemment, il ne s’agit pas de résumer l’œuvre en cause. D’une préface à l’autre, je ne peux me départir de celle, délicieuse, que le professeur Mousseron avait fait dans la thèse de Bernard Tessier sur les groupes de contrats en 1975 :

« Le culte de l’individuel a marqué le 19ième siècle comme le culte de l’ensemble paraît dominer le 20ième siècle vieillissant. À la tendresse pour le grain succède la ferveur pour la grappe »[6].

J’aurais aimé avoir écrit une référence aussi joliment imagée. Je crois aussi que le contrat évolue au gré des décennies et que l’approche collective qui existait alors a été depuis malmenée, notamment par le numérique qui tend à isoler. D’ailleurs, la thèse du professeur Zannou propose justement d’insérer une plus grande intervention tant de la loi que de la communauté. Au-delà de cette question entre liberté et intervention, individu et collectif, finalement, la raison d’être de la présente préface est assez simple : je ne saurais trop inciter à la lecture de l’ouvrage de Professeur Zannou afin de découvrir, justement, une thèse qui en est une et qui constitue un passage obligé sur la question. Fort d’une plume colorée, alerte, dans le cadre d’un ouvrage ultradocumenté et clairement exposé, je souhaite au lecteur ou à la lectrice autant de plaisir à la lire que j’en ai eu à la voir se construire.

On July 1, 2026, the Federal Trade Commission published a proposed policy statement applying Section 5 of the FTC Act, 15 U.S.C. § 45(a), to AI companies that steer their systems’ outputs “contrary to consumers’ reasonable expectations.” Public comment closes on July 31, 2026. In this post, I process the policy statement through the AI Life Cycle Core Principles (AILCCP) framework to determine which of the framework’s AI-related 37 principles are implicated, how, and what that means for the developers, deployers, and counsel who will eventually need to account for it.

But before we begin, a few words about the AILCCP are in order. In March 2023, I introduced the AI Life Cycle Core Principles, a framework consisting of 37 AI-related life cycle principles built to give developers, deployers, regulators, and legal practitioners a shared, precise vocabulary for evaluating AI systems across their full operational life cycle. The AILCCP’s core ambition was—and remains—to surgically strip away the pervasive definitional ambiguity that keeps AI oversight discussions perpetually aspirational and in the course of this work, the framework grew and gained additional structure. All of these principles are now organized into 10 pillars and mapped across 10 life cycle phases, supported by 48 controls, aligned to 47 international standards from ISO/IEC, IEEE, NIST (and others), and cross-referenced to FTC, SEC, and FDA enforcement. More than 400 explicit cross-references, together with a taxonomy of recognized AI risks, connect these layers. The 10 life cycle phases trace an AI system from scoping and design through decommissioning, and each principle carries life cycle signals marking the phases where its risks, and the oversight they demand, concentrate. (A public-facing version of the framework is found at https://ailccp.replit.app.) 

The AILCCP framework serves a wide range of practical purposes. Practitioners can use it to measure the efficacy of proposed and enacted legislation against a defined principle set, separating statutes that actually drive change in conduct from those that merely aspire to it; to analyze AI vendor agreements, AI corporate policies, and AI oversight documents for the same gaps; to identify which of the framework’s 48 controls a deployment requires, from logging to human-in-the-loop gates; and to address virtually any other AI-related issue, mapping each principle to relevant international standards and enforcement regimes. Every definition in the framework comprises many elements, and any single legal development typically implicates only a subset of them. (The Governance principle, for example, is comprised of 17 distinct elements.) Homing in on the most relevant subset through the AILCCP gives practitioners a more precise vocabulary for gauging both their compliance posture, how well their practices align with the principle, and their exposure, where that alignment fails and enforcement risk begins. 

Queried against the policy statement, the framework yields two findings. The policy statement activates three principles from the AILCCP: Truth, Transparency, and Accountability. It assigns them the force of federal consumer protection law. And in doing so, it generates a collision between federal and state obligations that the Governance principle must absorb.

What the FTC Proposes

The FTC proceeds from the premise that AI companies market their products as systems that distill human knowledge to solve problems in furtherance of consumers’ objectives. Consumers therefore reasonably expect truthful and accurate outputs. A company that steers its system toward other objectives without clearly disclosing the deviation makes a material misrepresentation likely to mislead a reasonable consumer and affect the consumer’s conduct. That is deception under Section 5.

The policy statement treats a company’s motive as irrelevant. Whether the steering serves commercial profit, an effort to shape public opinion, or compliance with a state law that requires embedding particular values, the deception analysis is the same. A company avoids a deception finding, and thus falls within the proposed safe harbor, by “clearly and conspicuously” disclosing that its system prioritizes certain objectives over what users request and otherwise expect. (There is an open question as to whether consumers would bother reading any of these disclosures, let alone understand them, but that is a rabbit hole I want to avoid for now.) Disclosure is the sole lawful path for operating a system whose objectives diverge from the consumer’s default expectation.

Truth: The Default Expectation

In relevant part, the AILCCP framework defines Truth as ensuring “that the outputs, representations, reasoning, and explanations provided by the AI are accurate, honest, and not misleading throughout the system’s life cycle.” The detailed definition extends to grounding responses in verifiable data, minimizing hallucinations, and disclosing areas where the system cannot provide truthful outputs.

The FTC converts this requirement into a legal baseline. The policy statement reads, “[C]onsumers have a reasonable expectation that AI systems aim to give truthful and accurate outputs. Consumers have no basis to believe that AI systems aim to produce outputs that are distorted by undisclosed ideological objectives.” By anchoring the deception analysis to that expectation, the FTC makes Truth the default state against which all AI output is measured. A system that deviates without disclosure is presumptively deceptive.

The AILCCP assigns Truth its primary life cycle signals at Model Development and Training and at Operations and Monitoring (two of the framework’s ten life cycle phases, the first covering how a model is built and trained, the second how it runs in production under ongoing monitoring). Developers who introduce steering during model training, or who let output objectives drift during operations, without a matching disclosure mechanism are now working in territory the FTC has marked.

Transparency: The Mechanism of Compliance

The AILCCP’s Transparency principle requires in relevant part that “every significant aspect of the AI system’s data practices, logic, decision-making, and governance is open, accessible, and understandable to relevant stakeholders.” Because a disclosure that is technically present but practically incomprehensible fails that test, the framework sets the operative benchmark at what it calls “epistemic uptake,” met only when disclosed information is absorbed into stakeholder decision-making.

The FTC’s safe harbor maps onto Transparency and adds a procedural requirement of its own. The requirement demands clarity, conspicuousness, and persistence, which puts it well beyond a single disclosure buried in terms of service. The Commission is describing what the framework calls “multi-level communication,” tailored disclosures for technical, operational, and lay audiences, and “comprehension verification,” the periodic assessment of whether audiences have formed an accurate understanding of system operation. The safe harbor also tracks the framework’s “stratified transparency,” under which disclosure obligations scale with the sophistication gap between developer and affected population. A consumer-facing AI product serves users with limited technical literacy relative to its developer, so the obligation scales up.

Transparency’s life cycle signals sit at two of the framework’s ten life cycle phases: Scoping and Design, and Pre-Deployment Review. A company that builds its disclosure architecture only after a consumer complaint or an enforcement inquiry has already missed the window the safe harbor requires.

Accountability: The Enforcement Dimension

The AILCCP’s Accountability principle requires in relevant part that “AI system design and implementation examines output (decision-making or prediction); identifies gaps between predicted and achieved outcomes; [and] clearly reveals degree of compliance with Data Stewardship.” The principle’s FTC Angle field, which maps each principle to the angles through which the FTC would likely scrutinize, regulate, or take enforcement action, already listed “Transparency, Accountability, Accuracy, Reliability, Advertising, Endorsements” as the relevant enforcement dimensions, an anticipation of precisely this development.

Section 5 has always applied to AI companies, and the policy statement says so expressly. The novelty lies in the articulation of how the deception test reaches output steering, and in the implicit demand that companies maintain oversight structures capable of detecting when their systems operate outside the default expectation and of disclosing it when they do. Accountability’s life cycle signals span three of the framework’s ten life cycle phases: Scoping and Design, Pre-Deployment Review, and Operations and Monitoring. Those oversight structures must therefore be in place from system conception and persist through active monitoring.

The Governance Collision: Preemption and Colorado

The policy statement addresses Colorado’s Artificial Intelligence Act, which provides that AI companies can be held liable for discriminatory outcomes caused by their customers’ use of their products. The FTC foresees that companies might steer their systems toward objectives such as “equity” or correction of “historical injustices” in order to comply with that law or others like it, without telling consumers. Its position is that such steering remains deceptive even when undertaken in good faith, and that “state law is impliedly preempted to the extent it conflicts with a federal regulatory scheme.” The policy statement grounds the preemption argument in Executive Order 14365, “Ensuring a National Policy Framework for Artificial Intelligence” (December 11, 2025), and its stated preference for a single, minimally burdensome national scheme over fifty discordant state frameworks.

The AILCCP Governance principle requires “systems, policies, procedures, processes, roles, and responsibilities for managing AI risks throughout the AI life cycle” and calls for continuous monitoring of the organization’s regulatory obligations. Under the policy statement, a company that steers its outputs to satisfy a state equity mandate can create exposure under the federal prohibition on deceptive conduct. Organizations operating in Colorado, or in any state with similar legislation pending, face that conflict today.

The workable response to that conflict is to treat the policy statement as a disclosure architecture requirement. An organization whose system is steered for any reason, commercial, ideological, or compliance-driven, needs documented policies that make the steering visible to users in a manner satisfying the FTC’s requirement.

What Changes for Practitioners

If adopted, the policy statement would make undisclosed deviation from what the FTC treats as truthful output presumptively deceptive; clear and conspicuous disclosure would become the only safe harbor; federal enforcement would reach output steering regardless of motive; and managing the federal-state conflict would require documented disclosure architecture.

The most substantive change would be the allocation of proof. Today, a company steering its system toward non-default objectives can argue that consumers hold no specific expectation about the system’s internal objectives. The policy statement would articulate that expectation and place the burden of rebutting it on the company through adequate disclosure.

The Comment Period

A comment should engage the three places where the policy statement, measured against the AILCCP, falls short: the deception test’s mechanics, the adequacy of the proposed safe harbor, and the preemption conflict with state mandates. General observations about AI oversight will carry little weight in the record.

Conclusion

The AILCCP framework is designed to give the AI oversight conversation a precise and durable vocabulary. Under the policy statement, principles that lived in internal policy documents and voluntary commitments will be written into enforceable legal obligations.

Retail Rate Structuring. Of every objection raised at a hearing – noise, water, jobs – electricity related concerns top them all: will the facility strain the grid, spike residential rates, or crowd out the load growth the community was counting on? These questions are best answered through legal structuring, not a communications campaign. The 2026 Ratepayer Protection Pledge – a White House initiative joined by Google, Microsoft, Meta, Oracle, xAI, OpenAI, and Amazon – commits operators to build, bring, or buy new generation and pay for triggered delivery upgrades, but it is only voluntary and politically contingent. No developer or concerned community for that matter should rely on such a voluntary pledge as more than a guide.

The value counsel adds is converting such a pledge into a binding instrument – for example, a ring-fenced cost-of-service rate class, negotiated bilaterally with the utility and written into the development agreement, protecting residential ratepayers and addressing the electricity related objections. Whatever generation the developer contracts must also qualify under applicable state procurement mandates (e.g., CEJA, CLCPA, VCEA), since a deal can meet commercial and rate goals yet still fail RPS compliance. And retail rates are set mainly by state Public Utility Commissions, which increasingly resist shifting new infrastructure costs onto households.

Grid-Reliability and Demand-Response Covenants. Demand-response and regional-grid obligations are moving from voluntary to mandatory. PJM’s May 2026 Board mandate accelerated its Reliability Backstop Procurement auction to September 2026 to allocate infrastructure costs directly to large data center loads, a framework counsel should build in upfront rather than retrofit later. Grid-reliability covenants, making backup capacity available to the grid operator during declared emergencies with appropriate FERC carve-outs, remain underused and turn an approval obstacle into a demonstrable benefit. Whether a local government can impose such a covenant without implicating FERC’s exclusive jurisdiction is unsettled, but the bilateral-utility route navigates most of the preemption risk.

Water Rights. In prior-appropriation states – Arizona, Nevada, Colorado, Utah, and much of Texas – water is a property right administered by state engineers and water courts and subject to suit by downstream holders entirely outside the local process. Municipal approval without a water right of sufficient priority is not a completed approval. At site selection, counsel should pin down the priority date, the call risk in a dry year, the protest exposure of any change application, and whether the right is certificated or merely claimed, remembering that interstate compacts such as the Colorado River and Rio Grande Compacts constrain even senior holders during shortage.

Air Quality. Backup diesel generators are the most consistently overlooked air-quality trigger. In nonattainment areas – much of California, parts of the Northeast and Mid-Atlantic, and portions of Texas – a hyperscaler’s generator fleet can require a major-source permit under the Clean Air Act, triggering a Best Available Control Technology (BACT) analysis and potentially scarce, expensive emissions offsets. Air permitting runs through federal and state air districts on a timeline independent of, and often longer than, local approval; closing on land and beginning site work first can materially misprice development risk.

GHG Reporting and Cap-and-Trade Exposure. Cap-and-trade obligations can be easy to miss. A grid-connected data center is normally not a covered entity, because its footprint is purchased electricity, captured at the utility. But a “bring your own energy” project, e.g., on-site gas turbines used to sidestep interconnection queues, are an operator’s own Scope 1 combustion emissions, and a hyperscaler fleet can certainly surpass the 25,000-ton compliance threshold under California’s Cap-and-Invest program or Washington’s Climate Commitment Act. When this happens, the data center becomes a covered entity owing allowances – a cost that must be priced in at site selection, not discovered by surprise after the turbines have been running. The Illinois POWER Act may be the start of a trend: following Governor Pritzker’s June 5, 2026 pause on new data center tax incentives, it heads to the November veto session to convert voluntary BESS and diesel-control commitments into statutory conditions.

In each of these domains the rule is the same: the agreements that will survive scrutiny are the ones baked into enforceable instruments before construction begins – not the ones hoping for a voluntary pledge to hold or left to chance in the permitting process.

Michael Schmitz and Catherine Atkin are the Co-Chairs of Stanford CodeX Law x Climate.

Based on neuroscience, there is no single age that should be considered safe or unsafe for children’s exposure to social media platforms. Brain development and maturation are continuous, individualized, and multidimensional. However, legal and platform age thresholds vary across jurisdictions, generally ranging from 13 to 18 years old. The growing number of these measures reflects increasing fragmentation and the urgency of addressing online child safety. Cultural differences regarding childhood autonomy, parental authority, and online privacy further complicate efforts to define the right age.  Instead of establishing fixed age thresholds, online child safety may be better served by regulations that target platform designs and variable reward systems in ways calibrated to users’ developmental stages.

The “Right” Age?

Age assurance approaches vary across countries. In particular, the European Union member states reflect the flexibility provided under the General Data Protection Regulation (GDPR). The GDPR sets 16 as the age for a child’s independent consent for information-society services, while allowing member states to lower the threshold to 13.[1] In April 2026, the European Union further strengthened its approach by announcing the deployment of an age verification app enabling users to verify their age when accessing online services.[2] This approach introduces age-verification infrastructure rather than setting new legal thresholds. Moreover, the United States has adopted a threshold for children’s personal data processing. The Children’s Online Privacy Protection Act of 1998 (COPPA) requires covered online services to obtain verifiable parental consent before collecting, using, or disclosing personal information from children under 13.[3] While such requirements are absent at the federal level, age assurance and verification thresholds are implemented under state laws.[4]

Social media platforms’ policies are shaped by this regulatory fragmentation. Major social media platforms, such as Facebook,[5] Instagram,[6] TikTok,[7] and X,[8] explicitly adhere to a 13-year-old threshold, prohibiting users below this age from creating accounts or redirecting them to age-appropriate experiences. However, varied age thresholds have been adopted for specific policies across platforms, such as restricting specific features like direct messaging or content access settings. The appeal of the fixed thresholds is that they are simple to enforce and provide clear expectations for users, parents, platforms, and regulators. These measures raise a question: does neuroscience actually support a single age cutoff?

What Neuroscience Tells Us and Why It Does Not Give A Number

The limbic system governs emotional processing and rewards. This system shows increased sensitivity during puberty because hormonal changes increase an individual’s responsiveness to rewards and social  feedback. In contrast, the prefrontal cortex is responsible for impulse control, planning, and risk assessment, and is one of the last parts of the brain to fully develop. While the full maturity of the prefrontal cortex is estimated to occur by the mid-twenties, approximately 25 years old,[9] this is not absolute. Brain development varies across different people. Some individuals reach a plateau earlier, while others experience maturation that extends even further.[10]

The fact that emotional responsiveness and sensation-seeking develop earlier than self-regulation creates a developmental mismatch. While this imbalance between the dual systems has become a dominant framework for understanding adolescent risk-taking, the model has been contested as an oversimplification. The same child may be cognitively competent, but still psychosocially vulnerable. Although this imbalance is a general pattern, it does not occur at a specific age. Brain development is better understood as a gradient rather than a threshold. Different cognitive and psychosocial capacities emerge at different times and vary significantly across children. Therefore, a single age threshold is both over- and under-inclusive. It restricts children who can understand and engage with specific features before the specified age, while failing to protect other children who are not ready. Neuroscience offers developmental patterns, but it does not directly determine the appropriate regulatory thresholds, which remain policy choices.

How Platform Design Interacts with Developing Brains

Platform designs may influence adolescent behavior and engagement patterns during brain development. Features such as personalized content, infinite scroll, and unpredictable notifications (such as likes, shares, or comments) may function as variable-ratio reward mechanisms. These features are associated with dopamine-related reward responses and may increase engagement, potentially contributing to more compulsive use patterns.[11] Young users whose prefrontal cortex is still developing could be more susceptible to peer influence. They may experience heightened sensitivity to social approval and rejection.[12]

While there is no clear age threshold, neuroscience could explain that different risks align with  different brain developmental timelines. The feature-based approach suggests a shift from one-size-fits-all age thresholds to regulations that tailor platform features to users’ developmental stages. By tailoring platform features to users’ developmental stages, this approach addresses the actual sources of harm. Although more complex to implement than a single age cutoff, it may better align protections with the risks children face at different stages.

Some social media platforms have moved toward this approach. For example, TikTok sets a default daily screen time of 60 minutes for users aged 13-17.[13] It disables push notifications from 9 PM for users aged 13-15, and 10 PM for those aged 16-17.[14] Similarly, Facebook has implemented the Facebook Teen Accounts for users aged 13-17, automatically applying more protective settings and sleep mode, as well as a daily limit reminder by default.[15] Instagram restricts direct messaging between users identified as teens and unknown users over 18.[16] These examples suggest that platforms already operate on a gradient model in practice.

Beyond these default age-related features, many platforms allow parents or guardians to set up features appropriately for their children. Their developmental needs may be best understood by their parents or guardians. For example, YouTube’s Kids account allows signed-in parents to select a content experience for their children from preschool (ages 4 and under), younger (ages 5-8), older (ages 9-12), or a customized selection where parents approve content themselves.[17] These measures also reflect broader issues about platform responsibility. In the United States, recent product-liability verdicts have increased attention to claims involving allegedly addictive software design.

Moving Forward & Open Questions

There is no single right age. But there are design features that may be more or less appropriate at different developmental stages. Neuroscience helps explain this developmental gap. Specifically, reward systems and platform features may affect children differently. However, the precise thresholds, such as how many notifications or how much screen time trigger harm remain unclear.

Some social media platforms have implemented age-tiered approaches, but broader regulatory development is needed. Some level of age assurance remains necessary. It is still important for platforms to know a user’s approximate age in order to apply age-tiered protection effectively. However, a regulatory framework should treat age as a gradient, not a single gatekeeping threshold. Requirements should move beyond relying exclusively on fixed age thresholds and be built around risk- and design-based models. Platforms should still be required to assess risks and adapt their features to users’ age-related and developmental needs.

A feature-level approach is not perfect, but it is more closely aligned with how risk actually occurs. While more complex to enforce than a single age cutoff, this approach better balances the goal of mitigating online harms to children while preserving their access to social media. At the same time, the law should limit harms arising from age assurance or verification tools themselves, such as data privacy risks. Looking ahead, deeper collaboration between neuroscientists, platforms, and regulators will be necessary to refine this approach.

References

[1] EU GDPR Article 8, Conditions Applicable to Child’s Consent in Relation to Information Society Services.

[2] European Commission, European Age Verification App to Keep Children Safe Online, April 2026, https://commission.europa.eu/news-and-media/news/european-age-verification-app-keep-children-safe-online-2026-04-15_en.

[3] § 312.2 Definitions (defining “child” as an individual under the age of 13) and § 312.5 Parental Consent.

[4] Texas’s Securing Children Online Through Parental Empowerment (SCOPE) Act requires a digital service provider to register the age of the user, and users under 18 years old must obtain parental or guardian consent before creating an account; see also, Free Speech Coalition v. Paxton.

[5] Meta, Terms of Services, version 1 January 2025, https://www.facebook.com/terms/.

[6] Instagram, Terms of Use, https://help.instagram.com/termsofuse.

[7] TikTok, Terms of Services, version 22 January 2026, https://www.tiktok.com/legal/page/us/terms-of-service/en.

[8] X, Terms of Services, version 17 December 2025, https://x.com/en/tos.

[9] Mariam Arain et al., Maturation of the Adolescent Brain (2013), https://pmc.ncbi.nlm.nih.gov/articles/PMC3621648/.

[10] Leah H. Somerville, Searching for Signatures of Brain Maturity: What Are We Searching For? (2016), https://www.sciencedirect.com/science/article/pii/S0896627316308091; see also, Kathryn L Mills et al., Inter-Individual Variability in Structural Brain Development from Late Childhood to Young Adulthood (2021),https://pmc.ncbi.nlm.nih.gov/articles/PMC8489572/.

[11] Jiangsong Wang and Shen Wang, The Emotional Reinforcement Mechanism of and Phased Intervention Strategies for Social Media Addiction (2025),  https://pubmed.ncbi.nlm.nih.gov/40426443/.

[12] Eveline A. Crone and Elly A. Konjin, Media Use and Brain Development During Adolescence (2018),https://www.nature.com/articles/s41467-018-03126-x; see also, Patrik Wikman et al., Brain Responses to Peer Feedback in Social Media Are Modulated by Valence in Late Adolescence (2022), https://pubmed.ncbi.nlm.nih.gov/35706832/.

[13] TikTok, Guardian’s Guide, https://www.tiktok.com/safety/en/tools-and-guides/guardians-guide.

[14] Id.

[15] Facebook, Help Centre, About Facebook Teen Accounts, https://www.facebook.com/help/1105851604342488.

[16] Instagram, Help Center, About Instagram Teen Privacy and Safety Settings, https://help.instagram.com/3237561506542117.

[17] YouTube, For Families Help, Help Center, Create a YouTube Kids Profile, https://support.google.com/youtubekids/answer/7554914?hl=en.

Can a large language model predict whether a business practice will be found anticompetitive? The authors build a tool that answers the question for one jurisdiction. They take Peruvian competition law as their case study and construct a case outcome predictor. The design is meant to be replicated. An enforcer or a practitioner elsewhere can follow the same framework and adapt it to a given practice under a given legal regime.

The method matters as much as the result. The authors do not retrain a model. They guide it with legal expertise through prompting and retrieval-augmented generation. The system retrieves past case law and structures the legal reasoning around it. This keeps current assessments closer to prior administrative precedent and improves consistency across decisions. It also lowers the barrier to building such a tool, since no model retraining is required.

The authors are careful about scope. Their results are preliminary. The benchmark is a small proof of concept rather than a full validation. They present Ulysses as a demonstration of what domain-adapted legal workflows can do for specialized legal judgment prediction, not as a finished enforcement instrument.

Thibault Schrepel, founder of the Stanford Computational Antitrust Project, stated:

“This paper makes a useful point. You do not need to retrain a model to build a working legal prediction tool. You need legal experts who know the case law and a careful retrieval design. The authors prove it on Peruvian competition law, and they hand others a framework to follow. That is how computational antitrust reaches jurisdictions without large engineering teams. I value the honesty about the limits as much as the result.”

The article is available for download on the Stanford Computational Antitrust project’s page.

Antitrust agencies can use AI to gather intelligence and to run their operations more efficiently. They can build tools in-house or rely on outside providers. The article examines what this adoption can deliver and what stands in its way. The potential gains are large. The risks are real. Some are particular to AI. Others come with any new technology.

The article argues that a top-down governance strategy matters most. Authorities need guardrails to manage the risks. They need scaling plans to turn early experiments into lasting capacity. Cooperation among authorities adds further value, since few agencies can address these problems alone. The article also looks beyond enforcement tooling. It asks whether AI might itself support pro-competitive intervention, for example through AI-based consumer agents acting on behalf of users. May treats this as a prospect rather than a recommendation. He argues it is too early to advocate such measures, while their potential merits further study.

Thibault Schrepel, founder of the Stanford Computational Antitrust Project, stated:

“Richard May moves the discussion past whether agencies should adopt AI toward how they should govern it. His focus on guardrails and scaling plans gives authorities a workable agenda. The closing reflection on AI-based consumer agents points to where the harder questions now sit.”

In memory of Alan Turing (23 June 1912 – 7 June 1954)

Can a machine genuinely think, understand, or have a mind? Or, in other words, can machines possess intelligence?

One may argue that the answer depends on how we define intelligence. Is intelligence simply the ability to produce intelligent behavior? Or does real intelligence require inner understanding, consciousness, and maybe even the unconscious?

Two of the most important positions in this debate come from Alan Turing’s “imitation game” and John Searle’s “Chinese room argument.” Turing shifted the question toward behavior: if a machine can perform intelligently enough that we cannot distinguish it from a human, then we have reason to call it intelligent. Searle pushed back: producing the right output is not the same as understanding what the output means.

Let’s first briefly review the two sides.

Alan Turing’s imitation game, proposed in 1950

In 1950, Alan Turing proposed the “imitation game” to move beyond the vague question “Can machines think?” Instead of asking whether a machine has a soul, inner feelings, or a human-like mind, Turing focused on observable performance. If a human judge, communicating only through written language, cannot reliably distinguish between a machine and a human, then the machine has passed the test.

Turing’s move was powerful because it made the question testable. Rather than trying to look inside the machine for an invisible essence called “thinking,” he asked us to evaluate what the machine can do. In this sense, Turing’s approach is behavior-focused and functional: intelligence is judged by performance.

But this also creates a problem. Does intelligent behavior prove intelligence, or does it only imitate intelligence?

John Searle’s Chinese room argument, proposed in 1980

John Searle’s Chinese room argument was designed to challenge exactly this problem. He wanted to distinguish between symbol manipulation and actual understanding.

In his thought experiment, imagine a native English speaker locked in a room. This person does not understand Chinese. Inside the room, there are Chinese symbols and very detailed rulebooks written in English. Someone outside the room slips questions in Chinese under the door. The person inside follows the rules and produces perfectly coherent responses in Chinese.

To the person outside the room, it appears that the person inside understands Chinese. But from the inside, there is no understanding. The person is only manipulating symbols according to formal rules.

Searle’s point is that syntax is not the same as semantics. A system may process symbols correctly without understanding their meaning. For Searle, this means that computation alone is not sufficient for genuine understanding.

Are Turing and Searle views enough?

Are these two views necessary and sufficient to answer whether machines can possess intelligence? I do not think so.

Turing gives us an important condition: if a machine thinks, it should probably be able to behave intelligently. But behaving intelligently is not enough to prove that the machine actually thinks. A good example is the current state of large language models. With their language capabilities, it is getting easier for LLMs to perform well on Turing-style tests. It is also getting harder to distinguish between human and machine performance, especially when the task is clearly defined and the success metric is measurable.

But this does not settle the question. LLMs can produce responses that look like reasoning while still making mistakes in logic, common sense, grounding, and time. They can sound coherent without necessarily having a stable model of the world. This is why I do not think that predicting the next token from an enormous amount of data, by itself, fully answers the question of machine thinking.

On the other hand, Searle’s Chinese room also raises a question for me. What happens if the process inside the room is repeated thousands or millions of times? What happens if we add memory, feedback, learning, and the ability to connect symbols to action and experience? If the person or system changes over time, learns from past experiences and from semantic understanding in English, and begins to use the symbols in flexible and meaningful ways, is it still just symbol manipulation?

This is where the argument becomes more complicated. Humans also learn language through repeated exposure, correction, memory, association, and social use. We do not begin with full understanding. We build it gradually. So, the question is not only whether symbol manipulation is enough. The deeper question is: what must be added to symbol manipulation for understanding to emerge, if it can emerge at all?

Intelligence, Thinking, and Reasoning

I am not sure whether intelligence and thinking are the same thing. Maybe intelligence is a capacity for solving problems, while thinking is a broader mental process. Maybe thinking is one tool of intelligence. Or maybe intelligence, thinking, understanding, and consciousness are deeply connected in ways we still do not understand.

On the thinking side, Daniel Kahneman’s distinction between System 1 and System 2 is useful. System 1 is fast, intuitive, and automatic. System 2 is slow, deliberate, and effortful. This distinction loosely echoes some of the current language around LLM reasoning models: some systems produce immediate answers, while others are designed to reason more slowly and explicitly.

Consciousness, the Unconscious, and the Missing Piece

But the human mind has other aspects that are mostly missing from the classic Turing-Searle debate or the system1-system2 distinctions: consciousness and the unconscious.

The definitions of consciousness and the unconscious are still debated. But a simple starting point is this: consciousness is the capacity to experience, feel, perceive, or be aware of oneself and the world from a first-person point of view. The unconscious is the set of mental processes that influence thoughts, feelings, behavior, memory, and perception without being directly available to awareness.

These concepts matter because human intelligence is not only explicit reasoning. Much of what we call understanding depends on background perception, emotion, memory, bodily experience, attention, intuition, and unconscious processing. A person does not only manipulate symbols. A person lives in a world, acts in that world, feels consequences, and builds meaning through experience.

Philip Goff, in his 2019 book Galileo’s Error: Foundations for a New Science of Consciousness, argues that modern science became powerful partly by leaving consciousness out of its picture of nature. That move helped science describe the objective, measurable world, but it also left us with a hard problem: how do we explain subjective experience itself?

This is why the question “Can machines think?” cannot be answered only by looking at external behavior or internal computation. Turing shows us why behavior matters. Searle shows us why behavior may not be enough. But neither fully answers what thinking is.

To answer whether machines possess intelligence, or whether they can actually think, we may need a deeper theory of the human mind first. The Turing-Searle debate is not only about AI. It is part of a larger debate over whether mind is behavior, computation, brain activity, biological consciousness, causal role, embodied experience, or something else entirely.

Maybe the real question is not simply whether machines can think. The real question is what we mean by thinking in the first place.

Peut-on se lancer dans l’aventure de l’IA sans préparation? Même si le cadre juridique demeure encore à parfaire, il existe déjà des textes légaux et éthiques qui invitent les acteurs publics et privés à faire preuve de diligence dans la manière de développer et d’utiliser l’IA. Cette présentation vise à vous sensibiliser aux bonnes pratiques et aux avenues à privilégier pour évoluer de façon responsable avec l’IA.

À qui s’adresse l’atelier

Aux gestionnaires, aux responsables légaux, aux stratèges numériques et aux dirigeants de PME souhaitant explorer l’IA et booster leur visibilité en ligne.

Ce que vous allez concrètement retirer de la conférence

• Une meilleure compréhension des enjeux juridiques et éthiques liés à l’intelligence artificielle
• Des pistes d’action concrètes à mettre en place dès aujourd’hui

Déroulement détaillé de chaque rencontre

• 30 minutes : Accueil des participants
• 15 minutes : Mot de bienvenue
• 75 minutes : Conférence par un expert invité
• 10 minutes : Pause
• 70 minutes : Atelier d’intégration de l’IA
  • 10 minutes : Mise en contexte – Présentation des objectifs de l’analyse collective.
  • 25 minutes : Travail en sous-groupes – Détermination des tâches automatisables, des leviers IA et des gains potentiels (temps, coûts, revenus).
  • 20 minutes : Mise en commun structurée – Partage des recommandations et organisation des idées selon trois axes (automatisation, aide à la décision et optimisation).
  • 15 minutes : Synthèse stratégique – Détermination des « quick wins » à court terme, des projets structurants et des indicateurs de performance à suivre.
• 10 minutes : Mot de la fin

Sujets abordés

• Aspects juridiques
• Enjeux éthiques
• Actions à mettre en place dès maintenant

Pourquoi participer

Notre approche est axée sur les solutions concrètes et pas seulement sur la théorie. Nous combinons les deux afin que vous ayez tout en main pour implanter les meilleures pratiques IA dans votre organisation.

Suivant la conférence, nous vous proposons un atelier collaboratif pour mettre en pratique les notions apprises et échanger de façon plus personnalisée avec notre expert.

Expert conférencier

Vincent Gautrais — Chercheur au CRDP, professeur titulaire à la Faculté de droit de l’Université de Montréal, et titulaire de la Chaire L.R. Wilson sur le droit des technologies de l’information et du commerce électronique.

Animation

Matthieu Lirette-Gélinas — Maverick Analytik

Limite de 40 participants. Bienvenue à tous!

What do people mean when they talk about competition? Küsters answers the question with a corpus of approximately 1.1 million digitized newspaper articles published in the United States and Germany between 1870 and 1945. The analysis proceeds in three steps. Keyword frequencies establish the vocabulary surrounding competition in each national press. Dynamic topic modeling tracks how thematic clusters shift across decades. Cross-lingual semantic embeddings then measure where competition discourse sits between economic-administrative vocabulary on one side and war-and-contest vocabulary on the other.

The two countries diverge sharply. In the German press, competition migrated from civic-commercial vocabulary in the 1870s toward organized performance, order, and martial imagery by the 1930s. In the American press, competition stayed anchored to institutional and economic vocabulary throughout the period. The divergence predates the Sherman Antitrust Act of 1890. Küsters reads this as evidence that the two legal regimes were built on pre-existing semantic foundations rather than the reverse.

The article connects the historical analysis to present-day enforcement. Küsters traces a continuity from the German press’s framing of competition as a rule-bound contest to the legal doctrine of Leistungswettbewerb, the German concept typically translated as competition on the merits.” He then argues that machine-learning screening tools and large language models trained predominantly on one national tradition will reproduce that tradition’s implicit frames. A model trained on U.S. enforcement records learns to associate harmful conduct with terms such as conspiracy, foreclosure, and market share. The same conduct described in German or EU text data may be associated with orderly competition, fair trading, and performance. As enforcement agencies move toward algorithmic screening, this cross-lingual semantic mismatch becomes a concrete risk.

Thibault Schrepel, founder of the Stanford Computational Antitrust Project, stated:

“Küsters shows that the semantic foundations of competition law diverged across jurisdictions before the statutes were written. He then closes the loop with a warning that matters now. Algorithmic enforcement tools inherit those foundations from their training data. Agencies adopting these tools without auditing their jurisdictional coverage will import one tradition’s assumptions about what competition is into another tradition’s enforcement decisions. That is the kind of finding the field needs at this moment.”

Anselm Küsters is affiliated with the Max Planck Institute for Legal History and Legal Theory in Frankfurt am Main and with the Centre for European Policy in Berlin. The article is available for download on the Stanford Computational Antitrust project’s page.

Aron D’Souza, the event’s founder and CEO of an AI adjudication company, defends the model as the “safest sporting event in history” claiming that organizers will pre-screen athletes for potential medical hazards and monitor athletes’ health status while they are competing.[2] However, the World Anti-Doping Agency (WADA) has rejected this competition as dangerous and irresponsible, emphasizing athlete health, the risk of powerful substances, and the danger of normalizing performance-enhancing drug use for entertainment and marketing.[3]

This essay gives background to the bodies governing sport, evaluates the competing claims of individual autonomy versus public health, and concludes with a proposal for an enterprise liability-shifting regulatory framework. I argue that athlete bans alone are incomplete, and that sports law should also ask when federations, promoters, teams, sponsors, or parent companies should bear responsibility for the enhancement risks they help create.

I. Anti-doping Governance is a Combination of Hard and Soft Law

WADA is the organization spearheading international efforts to standardize and harmonize anti-doping efforts. WADA has an unusual hybrid structure. Formally, WADA is a Swiss private-law, not-for-profit foundation, but it is composed and funded by both the Olympic Movement and public authorities representing governments. This hybrid design allows WADA to operate between private sport regulation and public legal authority, making international anti-doping governance a mixture of soft law, contractual sport rules, and state-backed legal obligations.[4]

The WADA Code itself functions primarily as soft law—a set of rules enforced through private contracts or informal means between athletes, national federations, and the International Olympic Committee.[5] When an athlete signs up to compete in a federation event, they agree to strict liability, meaning they are responsible for any substance found in their body regardless of intent. The WADA Code is reinforced by the UNESCO International Convention Against Doping in Sport, a 136-page document that governs anti-doping practices across the globe, which provides a hard law foundation by requiring signatory states to align their domestic legislation with WADA standards. For serious first violations, the Code can impose a four-year period of ineligibility, especially when the violation is intentional or involves a non-specified substance or prohibited method.[6]

The Enhanced Games challenges this hierarchy by attempting to operate outside the contractual jurisdiction of traditional federations. In doing so, it raises questions regarding enterprise responsibility: if a private company incentivizes high-risk chemical or biological enhancement, who bears the legal burden if an athlete suffers a catastrophic health failure? Unlike traditional sports, where the athlete often bears the brunt of “strict liability,” the Enhanced Games model suggests a shift toward corporate accountability for medical oversight.

II. Sports Already Draw Blurry Lines Between Risk, Technology, and Enhancement

Broadly speaking, doping refers to athletes’ use of any performance-enhancing methods that are considered to be unethical or that undermine a level playing field in sport.[7] To be against doping depends on the premise that regulators can identify a clear boundary between natural performance and impermissible artificial advantage. But across chemical enhancement, equipment regulation, testing regimes, and competition categories, that boundary is often unclear. Therefore, the legal question should not be limited to whether an individual athlete crossed it; it should instead ask who designed, enforced, or profited from the current boundary lines.

A. Regulating Chemical Risks in Already Physically Risky Sports is Paternalistic

Anti-doping policy can sound paternalistic in sports where athletes already accept serious bodily risk. Free solo climbing, professional football, and mixed martial arts all involve physical danger that spectators, sponsors, and governing bodies permit or even market.[8] A rule telling athletes that physical risk is acceptable but chemical risk is forbidden can look less like a pure safety regulation and more like a judgment about what kinds of risk sport wants to recognize. The line between those risks can be blurred too. For instance, the U.S. 9^th Circuit in Hunt v. Zuffa affirmed that the performance of a doped MMA opponent was not so enhanced by performance-enhancing drugs as to exceed opponent’s consent to battery in fight.[9] In other words, the court treated doping as insufficient, on those facts, to transform the physical contact of an MMA fight into a nonconsensual battery.

B. We Already Regulate Physical Advantages

The same line-drawing problem appears in equipment regulation. As new technologies enter competition, sports regulators must decide which innovations are permissible improvements and which ones distort the nature of the event. In 2009, Swimming’s International Federation moved to ban non-textile polyurethane suits after controversy over whether the suits could aid speed or buoyancy.[10] Similarly, in running events, World Athletics amended rules in 2020 on what shoes competitors could wear, including considerations of availability, prototypes, sole thickness, and shoe construction.[11] Technological advancements though high-tech swimsuits and carbon-plated running shoes already indicate that the boundary between acceptable enhancement and impermissible advantage is unclear. Even outside the context of drugs or biological modification, sports law governance faces the problem of deciding how much human performance may be shaped by external intervention.

C. Testing Results are Uncertain and Discriminatory

Testing uncertainty adds another reason for caution before imposing the harshest sanctions. In Shelby Houlihan’s case, the transnational Court of Arbitration for Sport (CAS) imposed a four-year ban after testing positive for nandrolone, an anabolic steroid, while Houlihan argued that a pork burrito could have caused the result.[12]

Kathryn Henne, in her book Testing for Athlete Citizenship, argues that anti-doping testing can be seen as needless surveillance and can be discriminatory or arbitrary, including for women and people from lower-income countries.[13] She notes that athletes caught and punished for doping are not always the ones using performance-enhancing drugs to cheat. In the case of female athletes, violations of fair play can stem from their inherent biological traits.

D. Creating a Separate Category May Improve Competition Fairness

International bodybuilding competitions already separate a natural category from a performance-enhanced one. Kayser, Mauron, and Miah argued in 2007 that current anti-doping policy may create health problems of its own and that medically supervised doping could be ethically preferable to hidden, unsupervised use.[14] If enhancement exists anyway, disclosure and medical supervision may be safer than denial and underground use.

III. Shifting Responsibility from the Athlete to the Federation May Be a Way Forward

Modern sports enhancement is produced by systems, not just individual athletes. So rather than a binary choice between total prohibition and a separate, unregulated competition category, a more sustainable model may involve a regulatory framework that shifts liability to enterprises.

Just as enterprise liability for defective products can move responsibility from the individual user to the commercial actor that designs, markets, and profits from the risk, perhaps the same can be done with sports. Doing so could still preserve athlete sanctions for serious intentional doping, but adds enterprise duties of disclosure, independent testing, medical monitoring, insurance, and compensation when organizations fail to manage doping risk.

The stakes of maintaining the current athlete-centric model are high. History demonstrates that when regulators focus solely on punishing individual competitors, the systemic exploitation of human biology continues unabated. During the German Democratic Republic (GDR) regime, doctors and coaches administered the anabolic steroid Oral-Turinabol to athletes without explaining what the substances actually were.[15] More recently, WADA banned Russia in 2019 from international sports competitions for four years after the country was found to be running a years-long, state-sponsored doping scheme.[16] In both cases, institutions encouraged, concealed, or normalized enhancement practices. Punishing only the athlete therefore misidentifies the source of the harm; it leaves untouched the officials that create the conditions under which athletes are pressured or deceived into modifying their bodies for competitive gain.

Instead of asking only whether the athlete should be banned, regulators should ask what duties should fall on the league, promoter, federation, team, sponsor, or parent company that designs the competitive environment. The U.S. appeals court in Hunt pointed towards this path when it considered UFC’s alleged representations, testing practices, and role as event organizer.[17] While the court ultimately rejected Mark Hunt’s claim that UFC encouraged a doped athlete fight, it allowed legal theories tied to Hunt’s claim that he suffered actionable harm because he would have withdrawn from the fight had the promoter disclosed the truth about his opponent’s doping status.[18]

In WADA-regulated sport, transferring liability to the parent organization could mean stronger sanctions for support personnel and teams that facilitate doping and larger consequences for institutions that create incentives for hidden enhancement. In enhanced competitions like the Enhanced Games, it could mean mandatory health funds, physician oversight, and strict sponsor or promoter liability for misleading safety claims. A neutral policy could therefore keep WADA’s four-year ban for serious intentional doping while requiring the parent enterprise to bear costs when its business model increases the risk of chemical coercion, hidden enhancement, or unreliable enforcement.

By shifting the legal focus from “catching individual cheats” to “regulating enterprise designs,” sports law can finally hold the true profit-makers accountable for the physiological risks they help create.

References

[1] John Hoberman, The Enhanced Games: A Techno-Fantasy That Will Fail, 14 Performance Enhancement & Health 100380 (2026), https://doi.org/10.1016/j.peh.2025.100380; Clara Molot, Inside the Enhanced Games, Where Athletes Compete on Steroids. And Growth Hormones. And Adderall., Vanity Fair (Apr. 30, 2026).

[2] J. Whitehead, Enhanced games: Event for doped athletes backed by group who want to ‘cheat death,” N.Y. Times (Mar. 22, 2024).

[3] World Anti-Doping Agency, WADA Condemns Enhanced Games as Dangerous and Irresponsible (May 22, 2025).

[4] World Anti-Doping Agency, Governance, WADA, https://www.wada-ama.org/en/who-we-are/governance.

[5] World Anti-Doping Agency, World Anti-Doping Code art. 1, 2 (2021).

[6] Id. at Art. 10.2.1 (2021).

[7] Kathryn Henne, WADA, the Promises of Law and the Landscapes of Antidoping Regulation. PoLAR: Political and Legal Anthropology Review, 33, 306-325 (2010). https://doi.org/10.1111/j.1555-2934.2010.01116.x.

[8] Bengt Kayser, Alexandre Mauron & Andy Miah, Current Anti-Doping Policy: A Critical Appraisal, 8 BMC Med. Ethics 2, 5-6 (2007).

[9] Hunt v. Zuffa LLC, No. 23-3113, 2025 WL 1164219 at *1 (9th Cir. Apr. 22, 2025) (affirming the lower court’s decision).

[10] Swimming: Hi-Tech Suits Banned, Sky Sports (July 24, 2009).

[11] World Athletics Modifies Rules Governing Competition Shoes for Elite Athletes, World Athletics (Jan. 31, 2020); World Athletics Amends Rules Governing Shoe Technology and Olympic Qualification System, World Athletics (July 28, 2020).

[12] World Athletics v. Houlihan, CAS 2021/O/7977, Award, paras. 5, 26-33, 153 (Ct. Arb. Sport Aug. 27, 2021).

[13] Kathryn Henne, Testing for Athlete Citizenship: Regulating Doping and Sex in Sport (2015).

[14] Bengt Kayser, Alexandre Mauron & Andy Miah, Current Anti-Doping Policy: A Critical Appraisal, 8 BMC Med. Ethics 2, 5-6 (2007).

[15] Kyle James, East Germany’s Doping Program Casts Long Shadow Over Victims, DW.Com (Jan. 10, 2010) https://www.dw.com/en/east-germanys-doping-program-casts-long-shadow-over-victims/a-5968383.

[16] Eric He, What Does ROC Stand For? And Why Did Russia Get Banned from Olympics?, NBC Olympics (Feb. 5, 2022).

[17] See Hunt, 2025 WL 1164219 at *1.

[18] Id.

The GAO report, however, does not touch specifically on the FDA’s newer Breakthrough Devices Program.[3] This Program has operated for about a decade and provides device manufacturers with a menu of potential benefits—including potentially modifying clinical trial expectations—if the FDA determines that a device they are still developing meets certain statutory criteria. To be designated a breakthrough, a device should “provide for more effective treatment” of serious conditions and either 1) use “breakthrough technologies,” 2) have no alternative on the market, 3) be better than alternatives on the market, or 4) generally be “in the best interest of patients.”[4]

In a recent article in the Columbia Science and Technology Law Review, I argue that the kind of expedited regulatory review that happens in the Breakthrough Devices Program should not be combined with immunity from tort liability.[5] Medical devices that are authorized through a Premarket Approval by the FDA are shielded from much state-level tort liability by federal law, which appears to be true even if those devices participate in the Breakthrough Devices Program and receive expedited regulatory review. This is a problem, since it could shift some of the risks of medical innovation onto patients who use new devices, instead of the companies that develop them. The argument was based in part on empirical findings illustrating how this Program has been operating over its first decade.

In light of the recent GAO report on device recalls, this blog post draws out and highlights three of the article’s findings that have potential relevance to medical device recall law and policy. They are:

1. The number of breakthrough designated devices that go on to receive FDA authorization is increasing over time;
2. A notable amount of these authorized breakthrough devices are “cleared” through the 510(k) pathway, which typically does not require clinical trials proving safety and effectiveness; and
3. Recalls for breakthrough devices have already begun for this relatively new Program, affecting about one in ten of those authorized so far.

Authorizations of Breakthrough Devices Are Increasing

Generally, new medical devices (except for many low-risk ones) must get some kind of authorization from the FDA before they can be legally marketed in the United States. The three most common kinds of authorization are Premarket Approval, 510(k) clearance, and the De Novo pathway.[5] Premarket Approval is typically for higher-risk devices and requires clinical trials showing they are safe and effective. Moderate-risk devices instead usually use the 510(k) or De Novo pathways. The 510(k) process often does not require clinical trials if a manufacturer can show their device is “substantially equivalent” to a device already on the market. The De Novo process is for devices that are not equivalent to an existing product but are not risky enough to merit a full Premarket Approval, and may include some clinical trials.

Devices destined for any of these three kinds of authorization are eligible to apply to the Breakthrough Devices Program. The FDA reports how many medical devices it annually designates as “breakthroughs” on its website, which is roughly increasing over time.[3] However, the agency does not publicize which devices or manufacturers have received the designation or what regulatory benefits they received, if any, citing its own regulations and Freedom of Information Act exceptions for “confidential” business information.[6] In fact, the FDA did not even begin reporting which breakthrough devices had been authorized to enter the US market for patient use at all until 2022, when journalists began questioning the agency’s lack of transparency.[7]

Cross-referencing that now-public list of authorized breakthrough devices against other publicly available FDA databases gave a more robust picture of the Breakthrough Devices Program. The findings from my study show that the number of breakthrough devices that the FDA has authorized—not just designated—is also increasing over time.[5]

FDA Device Authorizations After Breakthrough Designation

*Data are through June 30, 2025; reproduced with permission from the Columbia Science and Technology Law Review

More Breakthrough Devices Going Through the 510(k)

These data on authorized breakthrough devices also show that a surprising number of devices going through the Breakthrough Devices Program end up being authorized through the 510(k) pathway. In the last full year of data that was available at the time, over half of all authorized breakthrough devices that year (2024) went through the 510(k) process.

This finding is surprising, since for that to happen a breakthrough 510(k) device would seemingly need to both be a “breakthrough” but, at the same time, be “substantially equivalent” to an existing product. One of the requirements for using the 510(k) pathway is that the device either has “the same technological characteristics” as an existing device, or has “different technological characteristics” but “[d]oes not raise different questions of safety and effectiveness.”[8]

Since the FDA rarely reports which statutory criteria justified providing breakthrough designation to a device intended for the 510(k) pathway,[5] it is difficult to determine exactly how a “breakthrough” device could either use existing technology or use new technology that has no new safety or effectiveness implications. Perhaps these devices received the designation by meeting statutory criteria other than the use of “breakthrough technologies,”[4] though it is difficult to know based on only publicly available records.

Recalls for Breakthrough Devices Have Already Begun

The data also show that, as of mid-last year, one breakthrough device had already undergone a Class I recall and 16 devices underwent Class II recalls—involving 17 out of the 160 total breakthrough devices that had been authorized by then. Class I recalls happen when there is “a reasonable probability that [a device] will cause serious adverse health consequences or death,” while a Class II recall generally indicates a risk of “temporary or medically reversible adverse health consequences.”[9] For context, the GAO found that device manufacturers voluntarily initiated between 736 and 865 recalls each year, from 2020 to 2024.[1]

While the one Class I recall was for a device that underwent Premarket Approval, the Class II recalls were for breakthrough devices authorized across all three pathways. These recall data do not necessarily speak to the overall safety of the affected devices, as recalls can happen for a variety of reasons and can be remedied.

Yet, the notable percentage of breakthrough devices being recalled (over one in ten) may deserve greater policy attention. As further context, previous research has found that medical devices that receive priority review from the FDA—a benefit also included in the Breakthrough Devices Program—are generally recalled earlier and more often than devices receiving standard review.[10]

Implementing the GAO Recommendations

When read in light of the GAO report and previous research, these data suggest that the FDA may need even greater staffing and resources to monitor breakthrough device recalls, possibly over and above the boost already recommended by the GAO. These innovative devices may undergo less strict regulatory review as a part of their participation in the Breakthrough Devices Program,[6] which may then require a higher level of regulatory surveillance and recall oversight after patients begin using those devices to ensure that public health can be adequately protected.

While the current political moment may not facilitate providing the FDA with adequate resources for regulatory supervision, policymakers should seriously consider how to implement the GAO’s recommendations as early as possible, and may need to account for breakthrough devices when they do.

References

[1] U.S. Gov’t Accountability Off., Medical Device Recalls: HHS and FDA Should Address Limitations in Oversight of Recall Process, GAO-26-107619 (Dec. 8, 2025), https://www.gao.gov/products/gao-26-107619.

[2] See 21 C.F.R. §§ 7.40–7.59, 810.1–810.17 (2026).

[3] U.S. Food & Drug Admin., Breakthrough Devices Program, https://www.fda.gov/medical-devices/how-study-and-market-your-device/breakthrough-devices-program (accessed May 13, 2026).

[4] 21 U.S.C. § 360e–3(b) (2026).

[5] Walter G. Johnson, Breakthrough or Breakaway Innovation?, 27 Colum. Sci. & Tech. L. Rev. 171 (2026), https://doi.org/10.52214/stlr.v27i1.14548.

[6] U.S. Food & Drug Admin., Breakthrough Devices Program: Guidance for Industry and Food and Drug Administration Staff (2023), https://www.fda.gov/media/162413/download.

[7] Katie Palmer & Mario Aguilar, FDA’s Breakthrough Device Program, Meant to Benefit Patients, Is Delivering the Biggest Gains for Companies, Stat News (2022), https://www.statnews.com/2022/04/18/fda-breakthrough-device-designation-investigation/.

[8] 21 C.F.R. § 807.100(b)(2) (2026).

[9] 21 C.F.R. § 7.3(m) (2026).

[10] Caroline Ong, Vy K. Ly & Rita F. Redberg, Comparison of Priority vs Standard US Food and Drug Administration Premarket Approval Review for High-Risk Medical Devices, 180 JAMA Internal Med. 801 (2020).

The Stanford Computational Antitrust Project brings together over 80 competition agencies from around the world to explore how computational methods can strengthen the analysis and enforcement of competition law. Membership is independent and carries no financial obligation.

The Bangladesh Competition Commission was established under the Competition Act, 2012 and is the statutory body entrusted with the application of competition law in Bangladesh. The Commission is responsible for preventing, controlling, and eradicating collusion, monopoly, abuse of dominant position, and other practices adverse to competition. It plays a central role in shaping the competitive conditions of one of South Asia’s fastest-growing economies.

The partnership will focus on methodological exchanges and applied research. The BCC will contribute to the project’s annual reports, participate in its events, engage in exchanges with peer agencies across the network, and collaborate on building legal frameworks and computational tools adapted to the realities of fast-evolving markets.

Dr. Thibault Schrepel, founder of the Stanford Computational Antitrust Project, stated:

“The Bangladesh Competition Commission is exactly the kind of partner we hoped to welcome. Bangladesh is one of the most dynamic economies in South Asia, and the questions its competition authority faces are at the heart of what computational antitrust can address. We are delighted to begin this collaboration and look forward to the work ahead.”

Contact:
Stanford Computational Antitrust Project
law.stanford.edu/computationalantitrust
schrepel@stanford.edu

Bangladesh Competition Commission
www.ccb.gov.bd

ElAbogado is a Spanish-based legal lead generation platform that connects people with the right lawyers across Spain, the U.S., Puerto Rico, Mexico, Chile, and Colombia. Co-founder Martí Manent and data scientist Veronica Sorin presented at Stanford Law School’s CodeX Group Meeting on how they’ve built what they describe as a state-of-the-art agentic legal lead management system.

• The problem they solve: Most people default to the nearest or most familiar lawyer rather than one who specializes in their specific legal issue — ElAbogado matches users to the right specialist, having helped 2.5+ million people find lawyers to date.
• Scale: The platform handles ~1,500 leads per day (8,000/week), fielding inquiries via chat, phone, and WhatsApp.
• 15 years of data as the foundation: Real legal case data trained their AI models on tone, question flow, case qualification, and jurisdiction matching.
• Agentic pipeline: AI agents now handle inbound/outbound calls, chat, and WhatsApp conversations — qualifying leads, extracting case data, routing to the right lawyer, and following up if a lawyer doesn’t respond.
• Results: ~70% of cases are fully automated; headcount for the human lawyer intake team has been cut by more than half in under a year.
• Guardrails: They deliberately stop short of 100% automation, keeping humans in the loop for complex cases (~30%) and running post-call quality checks.
• What’s next: Pushing automation toward 85–90%, expanding to new countries (now a one-month project vs. one year previously), and potentially entering English-speaking markets.

Watch CodeX Group Meeting with ElAbogado

Full Transcript

Roland Vogl

Welcome everyone to our Codex group meeting. This is our first group meeting after our FutureLaw Conference. It’s been an intense week at Stanford. If you were here, I hope you enjoyed it as much as we did. We thought we had a very productive week with a lot of new ideas that were being shared, and a lot of great people in town. If you weren’t able to participate this year, I hope you’ll make it next year. We will be pushing out videos of our main conference, the main Future Law Conference, in the next couple of days or so. You will be able to watch the videos in our YouTube channel. It’s really great content there, so be sure to check it out.

Roland Vogl: One of the great people who came here and was here for pretty much the entire week is Martí Manent, who’s been a friend for many years, a legal innovation powerhouse in Spain, and whose platform, whose ElAbogado platform, reaches beyond Spain, actually, to many other countries. This is actually a legal lead generation platform that he’s built among other legal tech projects. Martí has kindly accepted our invitation to present here today, along with Veronica, who’s a data scientist at El Abogada. We’re thrilled to have you here today, and excited to learn what you’ve been up to. With that, I will turn it over to Martí and Veronica.

Marti Manent Roland thank you very much for the invitation, and it was a great pleasure last week to be with all of you in Stanford. It was an amazing week, and probably all the videos will be very interesting for all of us.

Thank you for the invitation to explain ElAbogado, and what we have been doing the last two years, because probably after sharing this with a lot of people last week, I think that what we are doing is probably the state-of-the-art in agentic legal lead management.

For the first introduction, ElAbogado is a platform that was built with, we think, a nice proposal, because we think that a lot of people cannot go directly to a lawyer and what he or she does is go to the lawyer that is probably in the same street or close to a friend. Let’s imagine that if you have some health problem and you go just to the person that you have close to you. That probably is not what you are doing. If you have a health problem, you go to the doctor that has the speciality correctly matched to what you have.

But a lot of people all around the world cannot go to a lawyer that has the practice for the problem that he or she has. Some years ago, we understood that if we could just make a match with a lawyer who has the practice that that person is looking for, that would be a nice step to more democratize access to justice. That was our proposal. To do that, we think that we can do it in a big way. We try to do that with technology. After some years, we have helped more than 2.5 million people to find the right lawyer, and that is for us, is a pleasure to know that a lot of people have found the right lawyer to help with their problem.

To give an idea of how many leads we are managing per day, we are probably at more than 1,500 per day. We are managing these leads through… in Spain, also in the States for the Spanish-speaking community. We are the lead platform in Puerto Rico, and also we are in Mexico, in Chile, and Colombia.

To give a little bit of context on our platform — what we do is, when we find a person that is looking for a lawyer, we want to talk with that person, because we need to understand what kind of problem he or she has, what kind of lawyer he or she needs, and if there is a real problem, or if it’s not a legal problem.

In the last 5 or 6 years, we have been doing that with more than 40 lawyers in our team. But this is not scalable, and at the end of 2024, when the LLMs started being robust, we decided to try to build a new kind of solution.

What we understood is that if we could build a solution that can help people 24 hours a day, 7 days a week, through an agentic system, that would be amazing, because we could go to more countries and help more people.

That is what we have built with a conventional legal artificial intelligence model that is named CLAIM. One thing that we are very proud of is that OpenAI and 11 Labs — which are probably two of the leading labs doing LLMs, and also voice LLM, which is a level lab from Europe — have recognized us with some kind of award that we have received, and that, for us, was amazing.

To try to explain more entirely what we are doing here, it’s a pleasure that Veronica is here with me. She’s the scientist that can explain what we have been doing with this platform.

Veronica: I think Martí explained what it is that we do, and I will try to explain a little bit on how we do it. I think Martí already mentioned some of the numbers, so the only thing I want you to get in mind is that we had a really big challenge to try to manage and to respond to all those users that we have, who have a real legal problem and need a fast answer or help. We had to find a way to do it as fast and efficiently as we can.

We had to find a way to treat our users in the best way, and so the problem is: which is the technology, and how we can use the technology that we have at the moment to help the users in the best way. To keep in mind, we have, like, one lead per minute, more or less, and 8,000 or so leads by week, 30K in a month. We had to find a way to find a solution for this technical challenge. How can we respond to our users in the best way, fast, and give them the best specialist for their legal problem?

What we realized is that it’s not just one technology that can do all. We had to merge many pieces to make all this work and give us a solution.

The first thing we have is actually the training data. We have, like, 15 years of real legal cases. We have our lawyers that manage this list, that talk with the people. We have the lead with all the information, the data that is useful to train any artificial intelligence. That’s kind of the base.

Then we have to build, on top of that, with the best tools that we can find, depending on the problem. It could be commercial tools, or any open source tools, and so on. Recently, and we’ll talk a little bit more in a moment, it’s all about LLMs. What is our best LLM? The thing is that we need to test, tune, and find what is the best solution for each specific task that we need to resolve. There is no just one solution for all.

We realized — and this was some time ago, because actually we have been using artificial intelligence for years now — that sometimes we also need to build an in-house solution. We cannot just pick up some tool that someone else has built and plug it in. We actually had to tune that for our product. We have all that data as the training ground, and how we translate that and build a tool that helps us to solve each of the technical challenges that we have.

Here we mentioned Claim — I think Martí mentioned it — and we have also Leaks and SLAB that also give us the solution to other problems. The last part of all this is how we orchestrate all that. How we connect those tools — that could be just commercial, open source — with our tools, and how we use our data from all these 15 years to tune all that and put it all together.

As I said, we started with artificial intelligence some time ago; it’s not just with the big LLM movement.

We realized, as Martí said, that we have users that call us. One per minute or so. That means that you have a bigger, long call queue, and people that need an answer now — people that call, or people that just leave a message. We have all that, and we have our lawyers, who actually have to call them and extract or ask the user all the data that we need, so we can define what the problem is and which is the best specialist for that problem.

At that point, we understood that we had to make that process efficient. We needed a way to decide which is the lead you call first. How do you make all the efficiencies, so that you can give a solution to all of the users that are contacting you? At that moment, we used machine learning, so we trained, with our leads, a model that we also had in-house, to make this process very efficient.

Then, by 2022 or 2023, we had the big wow with the LLMs, so that became public for everyone. At that point, we said, okay, how do we start adopting that technology?

One way, for example, is to try to automatize and make things more efficient. For example, when we have to contact the lawyers, we send the information of the case of the user. The user needs a divorce lawyer that is in Barcelona, with such and such issues. All that, for example, the people just write in an email to send. Why, if the LLM can actually help on that — one of the simple things is we just ask it to make a text summary of the case. It’s automatic. The people do not have to write any email. Things go fast, more efficient. So you start doing this type of things.

Then when the LLMs became even more intelligent, I think that’s when we made the jump — a really big step forward — because now we can actually help people 24/7. People sometimes have problems, and the problems appear at any time of the day, and we are there for them to help.

Marti Manent Let me explain a little bit. In 2019, what we did with machine learning was orchestrate which lead we need to call right now. Imagine that you have 100 or 200 leads that you need to call. The best decision was made by machine learning — that was TensorFlow from Google.

The next step in 2023 was: some of the information that we are going to send to the lawyer — who is our customer; the customer is the lawyer that pays per lead to us — is gonna be built by an LLM. The lawyer that calls the potential customer asks: where do you need a lawyer, in Barcelona, in Madrid, in Miami, whatever; what kind of practice is this? At the end, it was an LLM that wrote the message that the lawyer is going to receive.

But here, we still had lawyers inside our company making these calls and writing part of the documents that we were sending to our customers.

The big jump, as Vero mentioned, is in 2025. That part of all the jobs that our lawyers had been doing, we have transferred to an agent. To an agent that is making a call, an agent that is answering on WhatsApp, an agent that is answering a chat. I think that it’s making the decision to say which is the best practice that that person is looking for. I think that it’s making the decision of, okay, there is a case here that can go on. Also, an agent that makes a decision that, economically, this lead is viable.

Right now, we have probably half of the lawyers that we used to have in-house. In less than a year, we have reduced our task force that was doing the calls by a half. For me, that is very relevant, because that job has been done by lawyers, by people that have been trained, going to a law school, and have some kind of knowledge and make a decision. These decisions now are made by an agent. Better.

Veronica: Yeah, exactly. Just to give a little bit on the hints of what the pipeline looks like step by step: we have the user, the person, the human, that has a problem and contacts us looking for a lawyer. That can be either, as Martí said, a chat — so just typing — or calling us, or even WhatsApp.

Now we have an agent that actually answers the phone, in a way, or talks via chat, and that’s all AI. That agent can converse and has a natural conversation with the user, because what we need in that conversation is to understand what the problem of the user is, where the problem is, and ask the questions that we know will give us the lead, or that we can use later on to validate and say, okay, this is a viable legal case. As Martin said, also, it’s an economically viable one as well.

All that information is what we need to extract, and the agent knows, because we have trained it with the data from these 15 years. We tell this agent what it is that it needs to ask the user, so we have all that information to build this lead.

Once this conversation happens, then it goes to another agent, as well, that extracts all that data. We have a data model, so we actually have to fill that and get all the data from those leads to our database. Then that also goes to another agent where we do all this validation. Is the case valid? The agent knows how to decide if the case is a valid case or not.

Just to go on that part — the agent, once it qualifies the lead, can say: okay, I have everything, the lead is ready. We can just send it to the best lawyer, to the lawyer that this person needs, based on the legal area, on where they’re located, the jurisdiction, etc. It just automatically sends.

That’s more or less 50% of the cases. It’s a pipeline built on AI agents, pure agentic tech. Everything goes by itself.

At some point, also, it could be that it’s not a real lead. We are an online platform, so we receive calls that are an error, or there’s not a real legal issue there. It’s also as valuable to know not to send something to a lawyer. The agent also knows how to decide that the lead is not a real legal case. It also closes leads when that needs to be done.

There is also a third thing that could happen, and the system handles that as well: escalate to a human, which goes to our lawyers. When the lead needs to be handled by them, because maybe the case is complicated enough, or needs more information than we have, we can also transfer to a human when it’s needed.

We also kind of close the loop, because once we send the lead to a lawyer, we want to know if the user still needs help. In the case that the lawyers do not call for some reason, we make a follow-up call. That is also an AI doing that — it’s also another agent — to follow up and know if this person still needs help. If they do, it goes again back into the pipeline, and we help, again, to find a lawyer that helps with the problem.

As Martí said, we communicate with the users through all three channels. The chat was actually the first channel that we started with the AI, because it’s probably the easier one, in the sense that you have time to talk with the user, so the pace is very different, like with a voice call. When the user has to call us, it has to be in real time. We have the latency to take into account, and it’s very challenging. The LLMs are now capable of doing that, because before, at the very beginning, it was very difficult to find a tool that gives you the latency and the intelligence to do that. Now, it’s possible.

Also — and I don’t know if we mentioned — it’s not only the inbound calls, but we also do outbound calls. The channels talk to each other, so if in a chat you don’t get all the information, you trigger a call, and the system also calls the user to get all the information for the case if needed.

We also now include WhatsApp, which is different because the user really sets the pace there. The user can get the fast answer from us when the user is there, but if the user has to go away and come back, the chat keeps listening, and it waits for the user to talk to us again.

To put everything together — I think we can say that our secret, in a way, is to have the 15 years of real legal cases. That’s what we train all the agentic system that we have on. We can help the system know exactly what we need to ask the user, how to talk to the user, which is the tone, how to pace the conversation. At the end, we have what we need — the information we need for that legal case — so we can help them.

Marti Manent Just one second. To not run out of time, I think that we want to show a real call that I think will be more interesting. All this explanation is to do that.

Roland Vogl: Yeah, we can’t hear the audio, that’s too bad.

Veronica: No?

Roland Vogl: Nope. But… We have a mute agent.

Marti Manent Yeah, oh, we can share. I think that it’s in the presentation. You can see here, it’s our page. We share this information online.

What is very relevant here is that we have a lot of challenges. The first challenge was to build the orchestration for agents. The second challenge was to decide what kind of practice. The third challenge: decide if the problem was a real legal problem, and whether it’s economically valuable or not. Also, for example, we had challenges like the latency of the call, because if we don’t have the right latency and the answer goes too fast, then the user doesn’t perceive a correct back-and-forth and says, okay, that’s a machine, I don’t want to talk with a machine.

Anyone who wants to see online can go to eLabogado.com slash lawyer, and you can check real calls. You can see chats and all of this. I think that if the people want to make some questions, now would be the time.

Roland Vogl: Yeah, so I think one question I have is: at what point did you feel comfortable that the agent that you tasked with this triaging task — the one you had human lawyers do before, right — like, is this a legal case, what kind of case is it, and who’s the right lawyer in what jurisdiction to connect this case with? That’s the key functionality, and probably the first functionality you tried to use AI for. How much testing did you do, how much validation did you do, before you felt like, okay, we’re gonna use this now? Maybe you actually don’t have a job for your human lawyers anymore, or you no longer have that job for them.

Marti Manent Yeah, I think that has two answers. One is that we have a lot of cases from these last years, and we can train the machine, and also we can check if the answer that the machine is giving to us matches with the answer that the real lawyer gave previously. That is very relevant. We train over a platform with real cases, and then we pass the same cases through our platform and see the results.

Before we felt comfortable — as you asked — we didn’t put it online. The first thing that we put online was the chat, and that’s also very relevant, because it’s easier to manage a case through a chat — by the technology, not by the information, but by the technology. After we felt comfortable with the chat, we jumped to the voice. This was the step: first, train with all data, train the machine, then pass the test. Once the machine, the platform, answers the same answers that all the lawyers did before with those cases, we put that platform ready online.

The second part of the question: yes, these agents are doing the job that real human lawyers were doing. As we mentioned last week when I was in Stanford, I think that it becomes a tsunami to the profession. For us, like one year ago, it was very strange, because for us it was a real tsunami that was coming to the profession, and the people are not running. We have less than half of the people that had been doing this job.

Roland Vogl: I see, okay. So, now, at this point, can you say what percentage of your team — is this fully automated, this task now with agents already, or do you still have some humans involved in it, even if just for quality assurance?

Marti Manent Yeah, almost 70% of the cases are fully automated. You asked which lawyer we give the lead to. One of the tricky questions was to find the place where the user is, because in the States, for example, there are a lot of cities that have the same name. How you can say that it’s Miami from Florida, or it’s Miami from California, or it’s Miami from Texas? One trick that we found is to ask for the postal code. In which postal code you are — that is the way that you make the match with the lawyer in that jurisdiction.

For the 30% of the cases that cannot be managed by an agent, we send that case to a human, who manages the case. We also make post-checks of the calls — what does it mean? We check if, after a decision has been made by an agent, it is working correctly, and we are just tuning that result.

Roland Vogl: Okay. There’s a question from somebody in the chat, which is: deciding whether a case has merit or not — is that not already legal practice, and therefore might be called unauthorized practice of law?

Marti Manent What I understand is, to try to understand if the case can go on or not — is that the question?

Roland Vogl: Yeah, if a case has merit or not. If you say, like, okay, this is not even a legal case, versus, it is a legal case that you want to match with a lawyer. If you do this automatically, isn’t that already legal practice? It could be, therefore, a violation of unauthorized practice of law.

Marti Manent What we are doing is — our platform is a marketing platform. Our customers are the lawyers that want a new customer for them. What we are doing is helping lawyers, to give the lawyers the lead that they are asking for. For example, if you ask for a lead of a civil case, we cannot give you a criminal lead, because you are not doing that practice.

What we are doing is helping people — as I explained at the beginning — who don’t know what kind of practice they need and don’t know if there is a real case there. What we are just asking them is some questions to understand: okay, you’re asking for a civil case — yes, that’s a civil case. You are asking for a lawyer that makes that practice where? Miami, in San Francisco, or wherever. We say, okay, that is a lawyer that you previously chose.

For example, in the States, we don’t choose the lawyer. The lawyer is chosen by the user. In other jurisdictions, the platform can choose the lawyer. But for example, in the States, the user chooses the lawyer. If that lawyer doesn’t do the practice that the user is asking for, we need to say: okay, that lawyer does not practice physical cases or criminal cases.

Roland Vogl: Okay. Sugaram’s asking: what LLMs are you using to build your agents?

Marti Manent Yeah, nice question. We use all of the big names that you know. Here, I’m going to give you some tricks. We are not going to explain all the platform and how it works, but we recommend making parts of the problem. When you take all the problem in one cake, that is not the solution. You need to make some parts of that problem, and you can solve different parts of the problem with different LLMs. Let me explain a little bit. There are some LLMs that are quicker to answer. For example, if you are making a call, you cannot use, for example, Opus from Anthropic. Probably you need to use a smaller LLM that can go more quickly, and then the latency is lower. That is very, very relevant.

Roland Vogl: Got it, okay. Awesome, so what’s next for you? I mean, now it seems like you have, kind of, this business figured out. That’s an interesting point you made, too, which is — in a sense, what I hear you say is, look, with a new tech, you can do this all quite quickly, right? But it still required 15 years of experience to know what the issues are, what’s the tone, how to place this technology into this business opportunity, right? That’s really where the rubber hits the road for many, right? Theoretically speaking, we all have the tools now to build a flywheel, a machine like this, right? But exactly how to fit it into the world is really where you need the context and the experience and all that, right?

I think that’s one of the learnings from this. One of the questions that comes for me is: where do you… what’s the next thing you want to apply this technology towards? Can you share anything, or it’s all still in stealth, and we’ll learn about it at your next presentation at Codex.

Marti Manent No, don’t worry. I think that we need to move the line up close to 85% of the cases. One of the learnings that we have found is that if you try to put it 100% automatic, you will probably have a big problem, because it’s very, very hard. The actual LLMs make mistakes. They are not perfect. If you try to do a perfect solution, you are gonna make some mistakes, because it’s very, very hard to achieve that — almost impossible.

The first solution here is that we want to push the line up to 85% or 90%, but never to 100, because some cases must be managed by a human. What we are also doing is jumping to new jurisdictions. It’s more easy now, because we have the tools to jump to another jurisdiction. To go to another country, for example, for us, it’s like a one-year project, and now it’s like a one-month project. It’s very, very easy now for us to go to different countries. Another thing is, we are a Spanish-speaking platform, and one of the things that we have on the table is going to English-speaking users, also.

Roland Vogl: Got it, okay. Everyone can look forward to ElAbogado coming to a country near you. There’s global expansion on the horizon. Maybe last question, because we’re over time already, unfortunately — Reem is asking how you deal with privacy issues.

Marti Manent Yeah, that’s a very nice, interesting question. We are from Europe, and in Europe, the privacy issues are very, very interesting. We have a distributed platform, and for each country, we manage the data from that country. We use Amazon Web Service, and also we use providers that can provide service all around the world. That is one of the solutions, but it’s very easy, though it has some restrictions. You need to use the platform that can go for an instance for each country.

Roland Vogl: Got it, okay. Cool. Well, let’s… oh, hold on, so Salih has their hand up. Sadi, you wanna unmute yourself and speak up?

Salih Tarhan: Yeah, Salih, yeah, correct. I need to give some context, that’s why, actually, I raised my hand. Thank you guys for the presentation, it was great. So, while I reviewed your website, actually, I saw that you are probably targeting the U.S. attorneys as well.

My question is — we built a platform helping attorneys to find attorneys in other jurisdictions, in other U.S. states. What we saw is that there are a lot of problems on ethics and professional responsibility. You cannot market, you cannot feasibly vet the attorneys in some states. There are the ABA model rules, a lot of highly regulated areas. I think you plan to extend your reach to the U.S. Do you have a plan to navigate this complex area of the law? Do you see any problem in the future for that?

Marti Manent Yeah, there are some countries that we cannot go to, because the lawyers cannot make any kind of advertisement, and it’s like the old school, closed market. I think that that is not good for justice. I think that it’s good for justice that the users, the consumers, can see different kinds of lawyers, and the lawyers can advertise and explain their practice. Because if the lawyers cannot explain their practice, that is a closed market, and I think that it’s not good for the competition. So, there are some countries that we cannot go to.

Roland Vogl: Yeah, so I think, yeah, there’s some… there are several platforms that have been trying to resolve that in the U.S. I could probably try to connect you with some people who’ve been working on that, if you’re interested in that. I think one of the questions is: do you take a share in the legal fees that are being generated by this lead? That’s a different proposition and probably harder — it probably won’t pass the fee-splitting rules in the U.S. Whereas if you have a law firm just paying a general subscription fee or something to a service, that’s more likely just fine.

Well, anyways, I think this was a great conversation. Interesting also to see how a legal tech provider that you are — from the pre-GPT days and pre-LLM days — well, you started already in 2019, as you shared, with machine learning to help with the matchmaking, and then really also leveraged the LLM capabilities in your business. Also, how that affects your headcount and how you think about that. I think that’s been really instructive, and I think it’s instructive for a lot of legal tech players who’ve been around for some time, and maybe some new entrants, too.

That was a really rich conversation. I really appreciate you, Martí and Veronica, for staying up on this special day. I know Barcelona is like a holiday today, and it’s already late at night, so really thank you so much for being with us here today. On behalf of everyone, a quick round of applause. Thank you for joining us, thank you for everyone in the group for tuning in. Look out for our Codex Future Law videos, which will be coming out soon, and I will be sending an update on our next meeting in the near future. Alright, well, good to see you all. Thank you very much.

Depuis 2018, cet événement offrent un espace privilégié aux jeunes chercheuses et chercheurs pour présenter leurs travaux, bénéficier de retours de leurs pairs et contribuer à la formation d’une nouvelle génération de spécialistes du droit du numérique. Pour l’édition 2026, la réflexion portera sur le thème « Droit, numérique et autonomie » exploré sous trois axes majeurs: (1) l’autonomie des États et des insitutions, (2) l’autonomie individuelle, corporelle, sociale et médicale et (3) l’autonomie des systèmes numériques, des biens et des données.

Les propositions issues de différents systèmes juridiques et de disciplines variées (sciences politiques, philosophie, économie, sociologie, informatique, etc.) sont vivement encouragées.

Soumission : jeuneschercheursnumerique@gmail.com

Date limite: 23 février 2026 à 23h59

Pour plus de détails: Appel à communication VL_CJDN 2026

Pour en savoir +

1st Place – Overall: ClauseWise
Sharpe Challenge Winner
PatentVC | Trademarkia Winner

ClauseWise is a logic-based middleware designed to bridge the gap between static legal text and active financial systems. It transforms contracts from passive documents into executable code, allowing for automated enforcement and “what-if” scenario modeling.

The Challenge: “The Execution Gap”

• AI Hallucinations: Standard LLMs are probabilistic, meaning they “guess” terms rather than calculate them, which is dangerous for legal compliance.
• Revenue Leakage: Companies lose roughly 9% of revenue annually because they fail to track complex contract triggers like late fees or price escalations hidden in PDFs.
• Trust Issues: There is no standard way to turn machine logic back into human-readable legal text without losing the original intent.

The Solution: A Two-Way Pipeline

• Parsing: Converts messy contract text into a Structured Intermediate Representation (IR) that understands math, dates, and conditions.
• Execution: A “Logic Runner” simulates facts (e.g., a late payment) to determine exactly what is owed or who is in breach.
• Decompilation: A deterministic decompiler (non-LLM) converts that logic back into English to ensure the output is audit-ready and byte-identical to the intended law.

Market Impact

• Target: Corporate Legal and FinTech departments managing high-volume, complex agreements (SLA credits, procurement, debt).
• Value Prop: It shifts legal teams from reactive “document reviewers” to proactive “logic managers.”
• Core Benefit: Eliminates “contract leakage” and provides the transparent, verifiable results required for regulatory and judicial scrutiny.

Team
Alex Moon, PM at Coupa Software
Armin Heydari, PhD Candidate at Harvard University
Danhong Cao, Associate at Fenwick & West
Harit Patel, Builder
Yuvraj Taneja, High School Student and an AI Builder.

1st Runner Up – Miranda AI – Voice AI for the Moment Rights Matter

Every day, thousands of people are arrested and pushed into a legal system they don’t understand. Critical information—about their rights, their charges, and what happens next—is delivered at the worst possible moment: when they are stressed, intoxicated, confused, frustrated or unable to process it.

As a result, that information doesn’t land.

• This creates cascading bottlenecks during booking and processing:
• Individuals don’t understand instructions or next steps and make avoidable mistakes
• Jail staff must repeat information and manage frustrations
• Systems slow down due to inmate frustration

Miranda is a voice agent that anyone can call directly from jail / anywhere to get immediate, clear, state-specific information about their rights, their case, and what to expect next. This reduces confusion, improves compliance, and removes friction across the system.

For jails, this means:

• Fewer repeated questions and less strain on staff
• Smoother booking and intake processes
• Potential to reduce escalations by giving inmates a clear answers

When Miranda identifies that someone may need legal representation, it connects them with a partner law firm who matches their case. With the individual’s consent, Miranda shares a structured summary of the interaction.

• This benefits everyone:
• Individuals get faster access to the right lawyer
• Lawyers receive pre-qualified leads, pre-briefed clients
• Jails save time and money dealing with frustrated inmates

US law firms spend $2.5B annually on advertising. The US jail phone call industry generates $1.4B annually. Our mission is to modernize access to preliminary legal information. Through this mission, and an expansion into all areas where private citizens first interact with our justice system–hospitals, civil courts, etc. –we intend to become the #1 source of converted leads for lawyers. 

 Built on a multi-agent voice pipeline: Pipecat, Deepgram, GPT (13 multi-agent tools), Cartesia, PostgreSQL, and LangGraph. 

Team
Kollaikal Rupesh
Ruomeng Sun
Charley Moore 

2nd Runner Up: Bridge

Bridge is a legal intelligence platform that helps firms turn senior review into scalable associate training. It captures the value hidden in redlines and revisions so every correction becomes part of a firm’s collective know-how, not just a one-off edit.

In most firms, senior lawyers spend time correcting junior work, but the insight behind those edits is lost once the document is returned. Bridge changes that by helping firms learn from how their best lawyers review, so associates improve faster, firms develop more consistent standards, and institutional knowledge compounds over time.

The product is designed to fit into existing lawyer workflows without asking seniors to change how they work. By sitting between senior and junior lawyers, Bridge makes it easier for firms to preserve judgment, reduce repeated mistakes, and build stronger legal teams over time.

Team
Ayomide O. Oloyede
Iris Cai
Malti John
Mihir Modi
Tejaswita Kharel

Harvey Challenge Winner: Warhol: AI Copyright Defense

52% of designers now use generative AI tools. 2.5 billion images are stolen every day, which constitutes more than $600 billion in annual damages from lost licensing fees alone. And AI tools aren’t just getting better at generating images. They’re getting better at finding infringement. The copyright storm isn’t on the horizon. It’s here. In-house IP clearance teams built for a pre-AI world and freelance creatives working without legal cover aren’t remotely prepared for what’s coming.

The exposure is massive and almost entirely unaddressed. Designers, marketers, and agencies generate AI images daily for commercial work: for landing pages, ad campaigns, product mockups, merchandise, etc., and with little real visibility into whether those outputs resemble copyrighted works. Today’s copyright image search tools only catch exact copies. They can’t evaluate stylistic similarity, compositional overlap, or the legal doctrines that actually determine infringement (e.g. Fair Use, Substantial Similarity, Merge, Scenes a Faire, etc.). The gap between how copyright law works and the tools that impact it has never been wider.

Warhol is a copyright legal agent that closes that gap. It searches the internet for visually similar prior works using multi-provider reverse image search, scores candidates through a machine learning similarity pipeline that goes far beyond pixel matching, and runs the top matches through a doctrine-aware legal analysis layer — evaluating substantial similarity, fair use, merger, scenes a faire, and independent creation — grounded in real case law. Integrated directly into your AI image tool, Warhol evaluates any generated image in one click and delivers a structured copyright risk report before you ship.

Team
Will Dinneen
Josh Waldman
Chris Um
Dhanin Wongpanich 

Band AI Challenge Winner: Cura: Multi-Modal Agentic Dataroom Creation

312 hours per lawyer per year are lost on document management challenges. In M&A, the pain is sharpest. Over 10,000 U.S. deals close each year, each requiring hundreds of documents compiled under a closing clock. That means weeks of email ping-pong, associates at $600/hour chasing PDFs, and 40% of large deals missing their projected closing timeline. Virtual data rooms didn’t fix this.

Cura’s multi-modal agentic document retrieval gives in-house counsel more time to spend on high value judgment matters. Picture an M&A deal. The outside lawyer sends a checklist to the seller and the seller’s in-house counsel is now on a wild goose chase to retrieve the documents requested. What if agents could help? 

The client gets a guided flow — think TurboTax for legal documents. AI agents run in parallel across local storage, Google Drive, QuickBooks, and bank accounts via Plaid — finding, classifying, and renaming every document automatically. The lawyer opens a complete data room with AI summaries and pre-populated disclosure schedules before the first meeting. Zero follow-up emails. The first conversation is law, not logistics.

The firms that adopt Cura compress timelines and capture margin. The ones that don’t keep lighting money on fire chasing PDFs. 

Team
Will McDugald 
Sola Melville Paterson-Marke
Kaitlyn Angel Kwan
Taruni Kavuri
Emmanuel O. Daudu

Photos by Pierre-Loic Doulcet and Jay Mandal

Chile’s 2021 constitutional amendment was ambitious because it did more than add another privacy rule. Law No. 21.383 amended article 19, number 1, of the Constitution and stated that scientific and technological development must serve people and respect both physical and psychic integrity—that is, a person’s bodily and mental integrity.[5] It also added that the law must specially safeguard “brain activity” and “the information derived from it.”[6] That wording matters. It clearly reaches beyond the raw electrical signal itself. At the same time, it leaves an obvious follow-up question. How far does protection extend once neural activity is turned into later-stage information, such as a score, a profile, or another output produced by software? The Constitution marks out a protected area, but it does not by itself tell us which institution supervises each kind of downstream information, what remedies apply, or where non-medical consumer neurotechnology fits in the broader legal system.

The Supreme Court’s 2023 Emotiv ruling shows why the constitutional text matters. The case was brought by former senator Guido Girardi Lavín, one of the initiators of the constitutional reform that later became Law No. 21.383.[7] According to the judiciary’s official summary, the Court held that the commercialization and storage of the claimant’s brain-activity data violated not only his physical and psychic integrity but also his privacy rights.[8] It therefore ordered Emotiv to eliminate all stored information linked to the claimant’s use of the device, and to do so without further procedure.[9] The Court also instructed the Instituto de Salud Pública (ISP), Chile’s public health authority, and the customs authority to act within their powers so that the commercialization and use of the device, and the handling of the data it collected, would comply with Chilean law.[10] This was a real remedy. It showed that neurorights language in Chile was not merely symbolic.

But the ruling was only clear at one level—stored information collected through the device—and less clear on other issues. This is important. Emotiv Insight is not just a passive EEG recorder. Emotiv presents the product as a consumer headset that can collect raw EEG data while also providing information or performing actions for users such as performance metrics, facial-expression detections, and “mental command” features. The Court’s decision most clearly addressed the storage of the claimant’s brain-activity data and the legality of the device’s commercialization and use in Chile. It did not, at least on the face of the official materials, fully specify how law should classify and govern later software-generated scores, profiles, or inferences derived from that data—especially once the ISP later concluded that Insight was not being marketed as a medical device.

Acting on that instruction, the ISP issued a 2024 technical report classifying Emotiv Insight under Chilean sanitary-control rules. That report makes the distinction easier to see. The agency explained that the key issue under Chile’s sanitary-control rules—the regulatory regime governing medical products and devices—was not simply whether the device used EEG technology, but whether it was intended for medical purposes.[11] In other words, the question was not simply “does this device interact with the brain?” but “is this device being marketed and used as a medical device?” The ISP noted that Emotiv presented Emotiv Insight as a product for research applications and personal use, not as a tool for diagnosis or treatment.[12] For that reason, the agency concluded that Emotiv Insight was not a medical device and therefore did not fall within the ISP’s ordinary jurisdiction on that basis.[13] The ISP resolutions database separately lists Resolution No. 3147 of May 31, 2024 as determining the sanitary-control regime for Emotiv Insight.[14] That conclusion matters because it reveals a gap that is easy to miss. A consumer neurotechnology product can still interact with brain activity and produce user-facing outputs, yet remain outside a familiar medical-device pathway. Once that is true, the legal problem changes. The question is no longer whether Chile recognizes that brain-related information matters—it clearly does. The harder question is which legal pathway governs non-medical, software-mediated, and potentially inferential uses of information derived from neural activity.

A fair response is that Chile’s constitutional text may already be broad enough. One could argue that “information derived from” brain activity is an intentionally wide phrase, and that the Emotiv ruling proves Chilean courts can act when neurotechnology threatens mental integrity and privacy.[15] Some commentators go further and argue that the neurorights provision may not have done all the work in the Emotiv case, because ordinary privacy and data-protection law could already have supported much of the result.[16] That criticism should be taken seriously. But it does not eliminate the gray zone. Even if existing privacy and data-protection law could have supported the deletion remedy, the neurorights provision does something those rules do not: it gives information “derived from” brain activity a distinct constitutional status. The official materials currently show a strong constitutional principle, a meaningful deletion remedy, and an agency conclusion that the product at issue does not fall neatly within ordinary medical-device regulation.[17] They also leave open the possibility of further judicial intervention, even if the ISP does not treat the product as a medical device. But that is not the same as a settled governance framework. Courts may provide ex post remedies in particular disputes; they do not by themselves identify a clear ex ante regulator for non-medical, software-generated scores, profiles, or inferences.

This is not simply a story about data storage. It is a story about legal translation. Chile has clearly said that brain-related information matters, but it has not yet made equally clear how some forms of derived or inferred mental data should be classified, supervised, and remedied in non-medical consumer contexts.

The pending legislation confirms that this translation is still in progress. Boletín No. 13.828-19, a bill that would implement the 2021 constitutional provision with detailed rules on neurorights and neurotechnology research, has passed the Senate and is now before the Chamber of Deputies.[18] Official Chamber materials also show that, during the week of April 6–8, 2026, the Commission continued considering the bill and heard presentations on it, and that, in the following week, it scheduled a session to hear expert views on the bill, including that of Rafael Yuste, the Columbia neuroscientist who helped shape Chile’s neurorights framework. [19] Chile therefore looks less like a finished model than like a legal system still trying to convert constitutional recognition into operational governance. That does not make Chile less important. It makes Chile more revealing. Chile matters not because it has fully solved neurotechnology law, but because even its strongest rights-based framework still forces us to confront a harder question: what should law do once a system moves beyond collecting neural signals and begins producing information inferred from them?

References

[1] Lorena Guzmán H., Chile: Pioneering the Protection of Neurorights, UNESCO Courier (Mar. 21, 2022) (last updated Oct. 13, 2023), https://courier.unesco.org/en/articles/chile-pioneering-protection-neurorights.

[2] Ley No. 21.383, Diario Oficial [D.O.], Oct. 25, 2021, art. único (Chile).

[3] Girardi/Emotiv Inc., Corte Suprema [C.S.], Rol No. 105065-2023, Aug. 9, 2023 (Chile); see also Poder Judicial de Chile, Neuroderechos: Corte Suprema ordena eliminar información recogida por dispositivo de monitoreo de actividad cerebral (Aug. 11, 2023).

[4] Inst. de Salud Pública de Chile, Informe Técnico RCS No. 33-A/24: Informe de Evaluación de Solicitud de Régimen de Control Sanitario, “Emotiv Insight” at 5–6 (Mar. 2024); Inst. de Salud Pública de Chile, Resoluciones, Resolución 3147, Determina Régimen de Control Sanitario del Producto Emotiv Insight (May 31, 2024).

[5] Ley No. 21.383, supra note 2.

[6] Id.

[7] Proyecto de Reforma Constitucional, Boletín No. 13.827-19 (Oct. 7, 2020) (moción of Senators Guido Girardi, Carolina Goic, Francisco Chahuán, Juan Antonio Coloma, and Alfonso De Urresti) (which later became Ley No. 21.383).

[8] Poder Judicial de Chile, supra note 3.

[9] Id.

[10] Id.

[11] Inst. de Salud Pública de Chile, Informe Técnico RCS No. 33-A/24, supra note 4, at 1–3.

[12] Id. at 3.

[13] Id. at 5–6.

[14] Inst. de Salud Pública de Chile, Resoluciones, supra note 4.

[15] Ley No. 21.383, supra note 2; Poder Judicial de Chile, supra note 3.

[16] Alejandra Zúñiga-Fajuri et al., Neurorights in Chile: Between Neuroscience and Legal Science, in REGULATING NEUROSCIENCE: TRANSNATIONAL LEGAL CHALLENGES 165, 165–79 (Martín Hevia ed., 2021).

[17] Poder Judicial de Chile, supra note 3; Inst. de Salud Pública de Chile, Informe Técnico RCS No. 33-A/24, supra note 4, at 5–6.

[18] Proyecto de Ley sobre protección de los neuroderechos y la integridad mental, y el desarrollo de la investigación y las neurotecnologías, Boletín No. 13.828-19, segundo trámite constitucional (Chile).

[19] Cámara de Diputadas y Diputados, Citaciones Semana del 06 al 08 de abril de 2026 (Comisión de Futuro, Ciencias, Tecnología, Conocimiento e Innovación); Cámara de Diputadas y Diputados, Citaciones Semana del 13 al 15 de abril de 2026 (same commission).

La Chaire L.R. Wilson a le plaisir d’organiser un séminaire portant sur les impacts, les enjeux et les questions autour de l’IA dans l’enseignement du droit. Ce séminaire sera animé par l’excellent Cristiano Therrien.

📌 Faculté de droit, salle A-3421 ;

📅 22 avril à 16 h ;

🎤 Animé par Cristiano Therrien ;

📩 Sur invitation. si vous êtes intéressés, merci de contacter : luka.sanchez@umontreal.ca

Une occasion de réfléchir aux enjeux pédagogiques soulevés par l’intégration de l’IA dans la formation juridique : transformation des compétences attendues, remise en question des méthodes d’évaluation et redéfinition du rôle de l’enseignant.

Genomic data is inherently familial, connecting an individual to a vast network of biological relatives. The inferential capabilities of AI enable the identification not only of data subjects but also potentially of their family members, who have neither been informed nor given consent. AI thus reframes privacy concerns from unauthorized access to ongoing predictive inference about individuals and their relatives. This means that traditional data protection law, built around individual control over retrievable records that can be located and erased, is mismatched with how AI actually processes genomic data. At the same time, machine unlearning, which aims to remove particular data points from a trained model, remains technically challenging.

The Legal Disconnect

The right to be forgotten was established following the Court of Justice of the European Union (CJEU)’s 2014 Google Spain v AEPD decision.[2] This right was later codified in Article 17 of the European Union General Data Protection Regulation (GDPR), which authorizes data subjects to request the deletion of their personal data under specific circumstances.[3] By contrast, the United States lacks a federal equivalent. [4] Instead, the right to be forgotten is regulated under a patchwork of state laws, such as the California Consumer Privacy Act (CCPA).[5]

The right to be forgotten centers on the threshold of personal data. Under the GDPR, this is defined broadly as any information relating to an identified or identifiable person, including genetic and pseudonymized data.[6] Similarly, the CCPA covers these terms and extends to “abstract digital formats,” a category that captures AI systems capable of outputting personal information.[7] The CCPA further recognizes “unique identifiers” that identify a consumer or family, as well as “probabilistic identifiers” that link a consumer or device using specific data categories.[8] However, these frameworks primarily address outputs or stored records, not the internal architecture of AI models. It remains unclear how the right to be forgotten should apply to internal model parameters that encode learned patterns that enable inferences. Regarding genomic AI, a model does not require access to a data subject’s stored DNA sequence; it only needs to have learned the relevant patterns to infer traits about the data subject or their family.

A recent case further illustrates this disconnect. In the CJEU’s EDPS v SRB 2025 decision, data stripped of direct identifiers remains protected as personal data if re-identification is “reasonably likely.” The determination depends on the technical, organizational, and legal measures available to the specific recipient.[9] However, this reasoning presumes a static dataset capable of re-identification, instead of a probabilistic model that generates inferences from learned patterns. Moreover, the right to be forgotten is centered on an individualistic right. Yet, genomic data is inherently shared in nature. Consider siblings who share genomic data: if one consents to sharing their data while the other requests deletion. Whose right prevails? Does one sibling’s erasure undermine the other’s autonomy? This becomes even more complex when considering the entire network connected to that single data point. This tension suggests the need to reconsider privacy rights at a collective or familial level. As genomic data is a shared resource, an AI model serves as a population-wide asset. In that setting, individual removal might impact the broader data environment.

The Technical Reality: Machine Unlearning

A comprehensive solution for machine unlearning remains technically elusive. Existing methods experience significant scale limitations and are not yet suitable for routine deployment in large-scale production systems. The most direct approach to delete specific data and retrain the model from scratch is computationally expensive. Ideally, an effective “unlearning” standard would balance three competing objectives: speed and cost, privacy, and fairness. Speed requires unlearning to be performed quickly and efficiently. Privacy demands erasure of the targeted data’s influence on model parameters. Fairness ensures that removing data from one group does not degrade predictions or utility for other groups.[10] Fairness is particularly challenging because models typically evaluate groups and relationships rather than isolated individuals. Removing a single data point potentially distorts comparisons and the fairness constraints in which that point played a role. Deleting one record may require updating parameters at the group or population level to restore balance within the model. These adjustments could introduce new biases or reduce overall model performance.[11]

In the context of genomic data, even if direct data was surgically removed, embedded familial patterns can persist within the model’s learned parameters. This challenge is further complicated by the increasingly interdependent algorithmic supply chains. Major tech companies, such as Amazon, Microsoft, and Google, offer integrated “all-in-one” packages where foundation models and downstream applications are tightly coupled.[12] In this setting, unlearning a single record does not guarantee the erasure of those patterns across the entire supply chain’s architecture. These technical limitations directly challenge the obligations to remove data “without undue delay” under the GDPR’s requirement, or within the CCPA’s 45 to 90 day timeframe.[13]

Looking Ahead

The convergence of genomic data and AI has exposed the limits of the right to be forgotten in both the legal framework and technical reality. This highlights the need for a modernized legal framework that prioritizes collective rights and algorithmic accountability over individualistic control. Moving forward, as AI-trained genomic data moves toward breakthroughs that could save millions, it introduces a tension between the individual’s right to be forgotten and the collective right to health. The solution likely lies not in absolute erasure, but in governing how AI remembers, ensuring that persistence of data serves the public good.

References

[1] Giacomo Nebbia et al., Re-identification of Patients from Imaging Features Extracted by Foundation Models (2025), https://www.nature.com/articles/s41746-025-01801-0; Artem Shmatko et al., Learning the Natural History of Human Disease with Generative Transformers (2026), https://www.nature.com/articles/s41586-025-09529-3; and Rose Orenbuch et al., Proteome-wide Model for Human Disease Genetics (2025), https://www.nature.com/articles/s41588-025-02400-1.

[2] European Union, Right to be Forgotten on the Internet (2022), https://eur-lex.europa.eu/EN/legal-content/summary/right-to-be-forgotten-on-the-internet.html (ruling that the search engine operator has a responsibility to remove links to personal information from search results in specific circumstances). See also, Judgment of the Court (Grand Chamber), 13 May 2014. Google Spain SL and Google Inc. v Agencia Española de Protección de Datos (AEPD) and Mario Costeja González.

[3] EU GDPR, Article 17.1 (a)-(f) and Article 17.3 (stating conditions for erasure under Article 17.1(a)-(f), such as the data subject withdraws consent, or the personal data have been unlawfully processed. And, providing circumstances when the right to erasure shall not apply to the extent that processing is necessary, such as for exercising the right of freedom of expression and information).

[4] A comprehensive federal privacy framework has not been adopted in the United States. Data protection is governed by sector-specific laws, such as the Health Insurance Portability and Accountability Act (HIPAA) and the Genetic Information Nondiscrimination Act (GINA).

[5] CCPA, Section 1798.105.

[6] EU GDPR, Articles 4(1), 4(5), and 4(13).

[7] CCPA, Section 1798.140 (v)(4)(c) (defining that personal information can exist in abstract digital formats, including compressed encrypted files, metadata, or artificial intelligence systems that are capable of outputting personal information.); see also California Assembly Bill No. 1008 (2024).

[8] CCPA, Sections 1798.140 (x) and 1798.140 (aj).

[9] European Data Protection Supervisor v Single Resolution Board (EDPS v SRB), Appeal, Case C-413/23 P.  (Sept. 4, 2025), https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex:62023CJ0413.

[10] Alex Oesterling, et al., Fair Machine Unlearning: Data Removal While Mitigating Disparities (2023), https://proceedings.mlr.press/v238/oesterling24a/oesterling24a.pdf.

[11] Id.; see also George-Octavian Barbulescu and Francois Buet-Golfouse, Unfair Unlearning? Accounting for Fairness in Machine Unlearning (2018), https://kdd2025.kdd.org/wp-content/uploads/2025/07/paper_23.pdf;

[12] Jennifer Cobbe, Understanding Accountability in Algorithmic Supply Chains (2023), https://arxiv.org/abs/2304.14749.

[13] Ben Wolford, Everything You Need to Know About the “Right to be Forgotten”, https://gdpr.eu/right-to-be-forgotten/(stating that “undue delay” is considered to be about a month); CCPA, Section 1798.130(a)(2)(A).

The offer follows a forthcoming guide in the Network Law Review by Thibault Schrepel, creator and director of the Stanford Computational Antitrust project. The build takes a few hours with very little technical skills.

To show what is possible, Dr. Schrepel built one such system from all European Commission competition decisions spanning 1977 to 2025. The system maps the corpus as an interactive knowledge graph. Each decision is a node. Each doctrinal link is an edge. Users can see at a glance which decisions the Commission’s own case law treats as foundational, and where its reasoning has drifted or fragmented over time.

The best part comes next. The graph generates a private Wikipedia powered by the corpus. On demand, it produces a wiki page on any theme covered by the decisions, with cross-links to related topics and a master index. A user can ask for a page on ecosystem theories of harm. The system writes it from the decisions, cites the relevant cases, and links to every adjacent topic. Ask a follow-up question and the answer feeds back into the knowledge base as a new page. The more the system is used, the more connections it builds. It does not generate content beyond what the decisions contain. It does not hallucinate a case that does not exist.

The applications extend beyond research. Agencies can give the system to case handlers. A handler drafting an abuse of dominance decision can query every prior decision on market definition and identify where the reasoning has shifted. Court of appeals rulings can be integrated into the same graph. Handlers can see which theories survived appeal, which were reversed. Inconsistencies surface before a decision is issued rather than on appeal.

“Every competition agency faces the same problem,” said Schrepel. “Hundreds of decisions accumulated over decades. No one has read them all. No one remembers them in enough detail to catch contradictions as they emerge. A draft decision could be checked against the full decisional record before it is issued. Inconsistencies could surface before they reach appeal, not after.”

The project is open to partnering with agencies that want to build their own system. Each agency’s corpus is different. The methodology adapts to it. The project team will advise on design, corpus preparation, and deployment.

Agencies interested in exploring this should contact: schrepel@stanford.edu

The guide is available open access on SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6594359

Contact
Thibault Schrepel
schrepel@stanford.edu

If human application is successful, IVG imagines a pathway where gametes are generated ex vivo without needles into testes, ovarian hyperstimulation, or surgery. As I have argued in a recent paper, this could be particularly significant for posthumous conception in several ways. [5] First, IVG would remove the race against the clock that currently defines post-mortem gamete retrieval. [6] Second, it would weaken one of the most persistent ethical objections to the practice: invasiveness. [1] Third, by avoiding hormone cycles and surgical collection, IVG could markedly increase the availability of female gametes for posthumous use, narrowing a longstanding gender gap in this area. [5]

Less Invasion:

At present, sperm procurement varies in invasiveness depending on whether the source is alive, incapacitated, or deceased. Competent living men can usually provide sperm through ejaculation, sometimes assisted by penile vibratory stimulation or electroejaculation. [7] When a man is comatose or dying, retrieval is generally preferred before circulatory death because sperm motility declines rapidly afterward. [8] After death, the window for retrieval is narrow, with viable sperm typically recoverable for only 24–36 hours. [6] Once circulation has ceased, surgical methods are generally required, ranging from needle aspiration to open testicular biopsy. In practice, guidance recommends open surgical retrieval of testicular tissue post-mortem in order to obtain enough sperm for multiple IVF cycles. [8]

Against this backdrop, objections to posthumous gamete retrieval routinely foreground invasiveness. [1] In comatose patients, retrieval offers no therapeutic benefit, leading some to doubt whether it can ever be in the patient’s interests. Here, “harm” is not limited to pain, but may include non-experiential interests in bodily integrity or in avoiding genetic parenthood. [9] Others argue that once cardiac or brain-stem death has been declared, there is no rights-bearing person left to be harmed, and so bodily-violation claims lose force. [10] Still others maintain that interests in bodily integrity survive death, such that surgical retrieval remains invasive and objectionable. [1, 9]

IVG could directly target this concern. Because somatic tissue can be obtained far less invasively than gametes (and in some cases without touching the body at all, as with shed hairs on a hairbrush or saliva from a toothbrush) the technology could remove both the physical invasion and the race against time that currently define posthumous gamete retrieval.

More Inclusion:

The implications for women may be even more significant. To date, posthumous conception has relied overwhelmingly on cryopreserved gametes or embryos stored during life, or on posthumous sperm retrieval. Egg retrieval, by contrast, is an intensive and invasive process whether the woman is alive, incapacitated, or deceased. [11]

For egg cryopreservation, patients typically undergo controlled ovarian hyperstimulation involving daily gonadotropin injections over nine to ten days, followed by surgical retrieval of mature eggs. Retrieval from a comatose or dying woman is technically possible, but it still requires hormone stimulation and surgery while the patient is incapacitated. Post-mortem egg retrieval is more difficult still. Without oxygen, eggs cease to be viable within hours, making retrieval unrealistic unless stimulation has already been completed. [11]

The practical effect is a persistent disparity between sperm banking and egg cryopreservation. Sperm banking is comparatively simple and inexpensive, whereas egg storage requires hormonal stimulation and surgery. As a result, stored sperm is far more commonly available than stored eggs, even as egg-freezing rates rise. [12] And although egg retrieval from a dead or dying woman is technically possible, it is highly invasive, time-sensitive, and rarely attempted. [13] Here, IVG could be transformative. By bypassing ovarian stimulation and surgical retrieval entirely, IVG could enable women to store eggs without those interventions, increasing the number of women with eggs in storage and improving the availability of female gametes for posthumous use. In that sense, IVG is not only a less invasive technology; it is potentially a more inclusive one.

What IVG Cannot Change:

IVG could narrow the gender gap that currently exists in the availability of gametes for posthumous use. But any parity effect would depend on cost, safety, social acceptability, and clinical capacity. Moreover, posthumous conception using eggs still raises practical asymmetries in that if the surviving partner is male, a gestational carrier would be required. If the surviving partner is female and able to carry the pregnancy, donor sperm would still be needed unless, in some speculative future, IVG made it possible to derive functional sperm from her somatic cells. [14]

More fundamentally, reducing invasiveness will not end debates about autonomy, consent, or the weight some place on the right not to be a genetic parent. IVG may decouple posthumous conception from the specific concern that gamete retrieval is an invasive physical affront, but it does not resolve whether a person authorized posthumous reproductive use at all. Nor can IVG settle broader concerns about the future child’s interests, inheritance and so forth. Those questions remain regardless of how gametes are obtained. [5]

References:

[1] A.R. Schiff, ‘Arising from the Dead: Challenges of Posthumous Procreation’ (1997) 75(3) North Carolina Law Review 901.

[2] K. Bowman, C. Matney and E.P. Dawson (eds.), In Vitro–Derived Human Gametes as a Reproductive Technology: Scientific, Ethical, and Regulatory Implications: Proceedings of a Workshop (National Academies Press, 2023).

[3] H.T. Greely, The End of Sex and the Future of Human Reproduction (Harvard University Press, 2026).

[4] H. Devlin, ‘Lab-grown sperm and eggs just a few years away, scientists say’ The Guardian (5 July 2025), <https://www.theguardian.com/science/2025/jul/05/lab-grown-sperm-and-eggs-scientists-reproduction>.

[5] C. McGovern, ‘From scalpel to statute: IVG’s impact on invasiveness and gender parity in posthumous conception’ (2026) 34(1) Medical Law Review 2.

[6] A. Jequin and M. Zhang, ‘Practical Problems in the Posthumous Retrieval of Sperm’ (2014) 29(12) Human Reproduction 2615.

[7] H. Rozati, T. Handley, and C. Jayasena, ‘Process and Pitfalls of Sperm Cryopreservation’ (2017) 6(9) Journal of Clinical Medicine 89.

[8] C.M. Rothman, ‘A Method for Obtaining Viable Sperm in the Postmortem State’ (1980) 34(5) Fertility and Sterility512.

[9] G. Pitcher, ‘The Misfortunes of the Dead’ (1984) 21 American Philosophical Quarterly 183.

[10] J. Harris, ‘Law and Regulation of Retained Organs: The Ethical Issues’ (2002) 22 Journal of Legal Studies 527.

[11] M. Soules, ‘Commentary: Posthumous Harvesting of Gametes – A Physicians Perspective’ (1999) 27 Journal of Law, Medicine and Ethics 362.

[12] Department of Health and Social Care, Gamete (egg, sperm) and embryo storage limits: response to consultation(September 2021).

[13] D. Greer, A. Styer, T. Toth, C. Kindregan and J. Romero, ‘Case 21-2010: A Request for Retrieval of Oocytes from a 36-Year-Old Woman with Anoxic Brain Injury’ (2010) 363 The New England Journal of Medicine 276.

[14] A. Le Goff, R. Jeffries Hein, A.N. Hart, I. Roberson and H.L. Landecker, ‘Anticipating in vitro gametogenesis: Hopes and concerns for IVG among diverse stakeholders’ (2024) 19(7) Stem Cell Reports 933.

Cette présentation fera le point sur le niveau d’adoption de l’IA dans les cabinets d’avocats au Québec, et nos conférenciers discuteront des outils les plus utilisés, qu’ils soient génératifs ou spécialisés.

Le webinaire explorera notamment:

• Les enjeux de responsabilité professionnelle liés à l’usage de l’IA
• Les nouvelles compétences requises pour les juristes
• L’interaction avec la Loi 25 en matière de protection des renseignements personnels
• Les développements fédéraux à venir en intelligence artificielle, dont la Loi C-27

Une séance essentielle pour les professionnels du droit qui souhaitent comprendre les enjeux actuels, anticiper les évolutions réglementaires et adapter leurs pratiques.

Digital regulators worldwide are imposing sweeping bans on data combinations to eliminate data asymmetries and learning effects. The paper argues that these interventions reveal a critical disconnect. Rules such as those introduced by the EU’s Digital Markets Act and the UK’s Digital Markets, Competition and Consumers Act were designed for traditional platforms. They are now being applied to AI downstream markets whose competitive dynamics differ. Reinforcement learning and model drift disrupt the standard feedback loops. The distinction between across-user and within-user learning further complicates the picture.

Building on this diagnosis, the article proposes an alternative instrument. Computational presumptions grounded in verifiable privacy-utility thresholds can serve as measurable compliance mechanisms. The paper identifies a NIST-aligned threshold of 2 < ε < 8 as a workable safe harbor, and shows how the same logic can extend to federated learning and homomorphic encryption through functionally equivalent indicators. The resulting framework transforms the regulator’s task from tracking every prohibited data combination to auditing a single measurable parameter.

Thibault Schrepel, founder of the Stanford Computational Antitrust Project, stated:

“Alba Ribera Martínez offers one of the clearest demonstrations that the DMA’s architecture does not transpose cleanly to AI markets. Her proposal gives regulators something measurable to audit instead of an unmanageable monitoring task. This is constructive legal scholarship.”

Alba Ribera Martínez is a Visiting Professor at the Brussels Study Centre and an Editor-in-Chief of Stanford Computational Antitrust. The article is available for download on the Stanford Computational Antitrust project’s page.

The paper introduces a Smart Agent-Based Modelling (SABM) framework within computational antitrust to simulate and detect the conditions that foster algorithmic collusion. The authors run Bertrand duopoly simulations in which LLM-driven agents stabilize prices above competitive levels without being explicitly instructed to do so. Simulations conducted in English and Portuguese show that linguistic context shapes outcomes. Communication between agents amplifies emergent behaviors, including the mimicking of concerns about collusion itself.

The contribution bridges market simulation and antitrust enforcement. By giving authorities an accessible tool to reproduce collusive conditions in silico, SABM offers a path to stress-test theories of harm before they materialize in real markets. The framework speaks directly to the open question of how regulators should address autonomous algorithmic collusion when no explicit agreement can be documented.

Thibault Schrepel, founder of the Stanford Computational Antitrust Project, stated:

“Carlos Eduardo Veras Neves and Tanise Brandao Bussmann contribute a practical instrument to a debate that has too often remained abstract. Their simulations confirm that LLM-driven pricing agents can drift into tacit collusion without being told to. Any agency thinking about how to monitor these markets should take note.”

The article is available for download over here.

Claude Code moves fast at ideation, screen mapping, scaffolding, and boilerplate generation, and developers have shipped real apps this way. But at the back of the development life cycle, where compliance, privacy disclosure, security architecture, and App Store submission live, the human cost reasserts itself. Studies of AI-generated code indicate a significant share requires refactoring to meet Apple’s accessibility and performance standards, and a meaningful fraction of AI-driven apps fail review due to privacy or design violations.

A little more than three years ago, I began developing the AI Life Cycle Core Principles (AILCCP). This is a framework—and now an app—that organizes AI development and deployment obligations across 37 principles, 10 development phases, and 48 controls, mapped to international standards and regulatory enforcement contexts. It gives developers, deployers, lawyers, and policymakers a shared vocabulary and methodology for assessing where an AI system meets its obligations and where it falls short across the development and deployment life cycle. I use it to granularly analyze things like AI legislation, policies, AI vendor agreements, AI governance documents, and questions such as whether Claude Code is AGI. The AILCCP contains 37 principles, each with multiple requirements. Three principles apply most directly here: Wherewithal, Human-Centered, and Workforce Compatible. For each, I focus on the requirements most relevant to what the App Store test exposes, then apply it to Claude Code.

Wherewithal asks whether the capability matches what is being claimed. The enthusiasm around AI coding tools has generated claims that Claude Code can take a developer from idea to shipped app with minimal effort. That framing describes the front of the life cycle accurately and the back poorly, and developers who plan around it will discover the gap at exactly the point where it costs the most to close.

Human-Centered requires human-in-the-loop oversight at the pre-deployment review and deployment phases. Those are the phases where an iOS app is tested against Apple’s privacy guidelines, where data handling disclosures are drafted and verified, where security architecture is stress-tested, and where the submission package is assembled and submitted for review. Claude Code does not do those things independently. A developer who has moved quickly through scaffolding and code generation arrives at those phases with the tool’s momentum behind them and its limitations fully exposed.

Workforce Compatible asks whether an AI tool builds human capability or displaces it. A developer who uses Claude Code to generate iOS code throughout a project never learns iOS development. They learn to prompt. When the tool produces architecturally flawed code, which it does with some regularity, the developer has no independent basis for catching the error. They are dependent on the tool to identify problems that the tool created. That is not augmentation. It is a different kind of dependency, and it grows less visible the more the tool appears to be working.

Claude Code is powerful, without a doubt. But a phase-specific competence at a high level is not what “G” in AGI means.

I want to set aside the image and focus on the finger-pointer, because what the act of identification reveals is more interesting than the image itself.

Consider what was not contested. The airman’s rescue happened. The emotion expressed by millions of people who engaged with the image was genuine. No one who shared it claimed it was a photograph taken by a photojournalist embedded with the rescue team. Most people who encountered it likely experienced it the way they experience a commemorative illustration, as a visual token for something that actually occurred.

Now consider a cartoon. Suppose someone had drawn the same scene, soldiers in a helicopter, smiling, American flag in hand, in the style of a tasteful editorial illustration, the fact-checkers would have had nothing to say. The drawing would have traveled the same emotional circuit. The soldiers would have been the same soldiers. The rescue would have been the same rescue. The difference between the cartoon and the AI-generated image is purely procedural. The AI image was generated by a statistical model trained on visual data. The cartoon was generated by a human hand trained on visual instruction. In both cases, no camera was present. In both cases, the image is a representation, not a document.

Yuval Noah Harari argued in Sapiens that the human capacity for shared fiction, for constructing and inhabiting stories that are not literally true in a documentary sense, is the source of civilizational cohesion. The story of a rescued soldier, expressed in an image that was never a photograph, is doing exactly this work. It is binding a community around a shared recognition of something that happened and matters. The finger-pointer, by flagging the image’s generative provenance, is not adding epistemic content. The finger-pointer is asserting a procedural standard as a substitute for engaging with the story. The question “is this AI-generated?” has displaced the question “is this meaningful?” and the displacement is being performed as though it were a contribution to public discourse.

What the finger-pointer is actually doing is performing epistemic status. The detection requires no expertise, but deploying it produces the appearance of rigor: I saw through this, I identified the error, I am the one who knows. This is not fact-checking in any meaningful sense. Fact-checking interrogates claims and the claim here, that American airmen were rescued, is true. What is being fact-checked is the artwork.

This particular form of intervention will become self-obsolete. The precedent is already visible. When Photoshop entered the visual commons in the 1990s, “it’s been Photoshopped” carried the same accusatory charge the AI flag carries today. The charge faded because it became universal. Sharpening, cropping, color grading, exposure correction, skin retouching, background removal, these are now understood as the ordinary conditions of professional image-making, not deviations from it. Nobody pauses before a magazine cover to announce that the photograph has been post-processed.

As generative models improve and AI-generated imagery saturates the visual commons, the identification will carry decreasing signal. When every image could be AI-generated and many will be, announcing that a specific image is AI-generated will produce the same information as announcing that a specific sentence was typed on a keyboard.

With the release of the AILCCP Explorer, an interactive web application that makes the full framework navigable and searchable, this felt like the right moment to revisit what the AILCCP is, how it works, and why it is built the way it is. 

The AILCCP is a structured knowledge graph that connects existing principles, controls, international standards, life cycle phases, and identified risks into a single navigable structure with over 500 explicit cross-references. The ambiguity is extinguished.

The AI Governance Problem

ISO/IEC 42001 addresses AI management systems. The NIST AI Risk Management Framework maps risk categories and profiles. IEEE has published standards addressing algorithmic bias, transparency, and system design. The EU AI Act imposes risk-based obligations with enforcement teeth.

But these instruments do not talk to each other. Anyone building, deploying, procuring, or auditing an AI system today must reconcile guidance from them, map their practices to regulatory expectations that often vary by jurisdiction, culture, and produce documentation that satisfies reviewers.

What the AILCCP Is

The AILCCP is a cross-linked knowledge base built from five components: principles, controls, standards, life cycle phases, and risks. Each one connects to the others through explicit, traceable links.

The framework is built on 37 principles, most of which were distilled from international consensus such as the OECD, UNESCO, G7, G20, and APAC. The AILCCP gives each one a defined scope, an objective, and measurable outcomes so that stakeholders working with different source standards are looking at the same thing. Governance follows an AI system from the first scoping decision through operational monitoring to eventual retirement.

The Architecture

37 Principles

Each principle includes a short definition, a detailed definition, an objective statement, key questions, suggested controls, required evidence artifacts, and identified stakeholders. The principles are organized across 15 categories and mapped to 10 pillars that span Oversight and Accountability, Reliability and Robustness, Transparency and Explainability, Ethics, Fairness and Equity, Privacy and Consent, Safety and Security, Human-Centered and Workforce concerns, Data and Process stewardship, and Organizational Capability.

Every principle includes a rationale explaining why it belongs in the framework. When stakeholders adapt the framework to their context, the rationale helps them decide which principles matter most for their system.

48 Controls

Controls are the “how.” Each is defined by name, domain, function, and rationale, and each maps to its top three principle alignments. Across the full set, this produces 187 control-to-principle links. Every one of the 48 connects to at least one principle, ensuring that implementation guidance always traces back to a governance commitment.

But here is the thing: controls that exist in isolation, disconnected from the principles they are meant to serve, tend to break down. When a control has no explicit link to a principle, stakeholders struggle to explain why they are implementing it, auditors have difficulty assessing whether it is sufficient, and the control becomes a compliance artifact rather than a governance mechanism.

43 International Standards

The framework maps 43 standards from IEEE, ISO/IEC, and NIST, each with a scope statement, summary, intended use, and identified primary users. Each standard maps to up to five principles, generating 215 standard-to-principle links that touch 29 of the 37 principles.

The 43 standards were selected because they are actionable and recognized across regulatory and audit contexts. Standards are increasingly taking on weight, legitimacy, and force, recognized by legislators, regulators, courts, and the broader developer and implementer ecosystem. When the question is “show me the controls for data governance,” the answer has to trace to standards that carry that weight.

10 Life Cycle Phases

The life cycle spans ten phases, from Scoping and Design through Decommissioning and Archiving. Each phase identifies default owners (Product, Legal, ML Engineering, SRE, and others), expected evidence artifacts, and measurable metrics. Across all ten phases, 84 phase-to-principle links map governance commitments to specific moments in the system’s life.

Each link comes with a life cycle signal that includes a rationale explaining why that principle matters at that stage. Transparency, for example, means something different during Operations and Monitoring than it does during Scoping and Design.

Scoping and Design tracks requirements coverage percentage and reading level targets. Data Preparation tracks missing and invalid data rates, label agreement scores, and PII leakage tests. Evaluation and Red Teaming tracks bias delta, attack success rates, and coverage percentage. Operations and Monitoring tracks mean time to repair, drift alerts per month, and SLO attainment. Instead of “monitor for bias,” the framework says “measure bias delta during Evaluation and Red Teaming and track drift alerts per month during Operations and Monitoring.”

18 Identified Risks

The risk layer assesses 18 identified risks for severity and likelihood using a qualitative rubric tied to the five pillars. Seven are rated Very High severity, eight High, and three Medium. These risks generate 23 links to standards and touch 24 of the 37 principles, connecting the threat landscape directly to the controls and standards that address it.

As I see it, one of the more distinctive ideas in the framework is the “enabling risk” concept. The three risks rated Medium severity are transparency and explainability gaps that function as force multipliers for other, more serious harms. A system that lacks Explainability makes every other harm harder to detect, harder to diagnose, and harder to remediate. This layered thinking about risk cascades reflects how AI breakdowns actually propagate in practice.

The Cross-Link Network

In total, the framework contains over 500 explicit links. 187 control-to-principle. 215 standard-to-principle. 84 phase-to-principle. 23 risk-to-standard. Pick any entry point and trace a path to every other part of the framework.

What Sets the AILCCP Apart

Bidirectional Traceability

Most governance frameworks are organized top-down. The NIST AI RMF flows from four functions (GOVERN, MAP, MEASURE, MANAGE) down to categories and subcategories, but provides no built-in path from a risk finding back to the relevant activities and standards. ISO/IEC 42001 follows the Annex SL hierarchy common to ISO management standards, with 42 control objectives that trace from clauses downward, but the reverse mapping is left to the implementing organization. The OECD AI Principles offer five principles and five policy recommendations with no controls, no life cycle phases, and no risk mappings at all. In each case, the framework is organized in one direction.

A diligent team can reverse-engineer any of these frameworks. But the AILCCP builds the reverse paths in. Its 500+ explicit cross-references mean a user can start from a risk and trace to the standards and principles that mitigate it, start from a standard and see which principles it supports and which life cycle phases it touches, or start from a life cycle phase and see what should be measured, who owns it, and what evidence needs to be produced.

An auditor starts with a finding, a development team with a life cycle phase, a regulator with a risk. The graph accommodates all of them.

Ownership Built In

Every life cycle phase names default owners, required evidence artifacts, and measurable metrics. This turns governance from “someone should handle this” into “here is who is responsible, here is what they produce, and here is how it gets measured.”

The ownership model spans Product, UX, Legal, Risk, ML Engineering, Data Science, QA, Security, SRE, and Communications, because AI governance requires coordinated action across disciplines.

Designed for Audits

Because the AILCCP maps finalized, prescriptive standards, it produces references auditors and regulators recognize. When someone asks for evidence of data governance controls, the framework traces to a specific control, its rationale, the principles it implements, and the published standards that back it up. That is what “audit-ready” looks like in practice, a traceable chain from commitment to evidence.

Coverage Visibility

With 29 of 37 principles referenced by standards and 24 of 37 referenced by identified risks, the framework makes its own coverage gaps visible. Eight principles are not yet referenced by any mapped standard. Stakeholders can see at a glance which principles have strong standards backing and which need additional work.

Who the Framework Serves

Development teams can use the 48 controls as a checklist during system design and code review, trace a specific risk back to the principles and controls that mitigate it, and identify which standards apply to a given feature or component.

Compliance and legal teams can demonstrate alignment with the EU AI Act, ISO/IEC 42001, and other regulatory frameworks, prepare audit-ready documentation by mapping internal practices to published standards, and build a defensible governance narrative for regulators.

Risk and audit professionals can use the severity and likelihood rubric to prioritize assessments, trace risks to specific life cycle phases to focus audit scope, and cross-reference internal risk registers against the AILCCP’s identified risks.

Regulators and policy advisors can use the framework to understand how international standards map to practical governance actions, and evaluate organizational compliance claims against a structured benchmark.

Executives and board members can get a strategic view of governance coverage across the five pillars without requiring technical depth, using the framework as a common language between technical teams and leadership.

A small team can use the controls as a lightweight development checklist. A large enterprise can use the full cross-linked structure to build audit documentation, assign ownership across departments, and track metrics at every life cycle phase.

The AILCCP Explorer

The framework is delivered as an interactive, searchable web application called the AILCCP Explorer. The Explorer provides multi-directional navigation. Start from any entity type and trace connections across the knowledge graph. Filter by pillar, phase, risk severity, or standard body. And the Export Library feature enables offline analysis and audit preparation.

The risk assessment methodology is built into the interface with inline explanations, so stakeholders can understand why a risk carries the severity rating it does without consulting a separate document.

Governance as Infrastructure

The AILCCP started with a simple observation: the vocabulary of AI governance was too ambiguous to be operative. Three years later, that initial effort to define terms with precision has grown into a knowledge graph of 37 principles, 48 controls, 43 international standards, 10 life cycle phases, and 18 identified risks, all connected through over 500 explicit cross-references. The framework assigns ownership, specifies measurable metrics at each phase, and traces every control back to the principles and standards it serves. It works in every direction, so that an auditor entering through a finding, a development team starting at a life cycle phase, a compliance officer mapping to regulatory expectations, a board member looking for coverage across pillars, and a regulator focused on a risk are all navigating the same structure.

The work continues and the AILCCP Explorer AI governance accessible.

Explore the tool. Try it. And tell me what works and what doesn’t.

The Design vs. Content Pivot

Section 230 of the Communications Decency Act functions as an immunity, not an affirmative defense and tech companies typically invoke it in a motion to dismiss to stop litigation before discovery begins. Meta raised arguments in both the New Mexico and California proceedings consistent with Section 230’s traditional content-immunity framing, arguing it was a passive conduit for third-party generated content and therefore immune from liability for what that content did. But the courts in both proceedings allowed design-based and consumer protection claims to proceed. That did not immediately resolve the cases, but it opened the door to discovery, and discovery is where the cases were won.

With that door opened, the New Mexico jury was able to see internal Meta documents and evidence uncovered through the NM AG’s investigation, including Operation MetaPhile, employee warnings that had been disregarded, and evidence the AG argued showed Meta had deliberately designed its platforms to addict young users and connect them with predators. The California jury saw the same architecture of corporate knowledge and deliberate design choice and responded with punitive damages. Neither jury was deciding whether Meta was responsible for what some predator posted. Both were deciding whether Meta architected the loop that made the harm foreseeable, systematic, and profitable.

Section 230 arguments will be raised in Nippon Life, but the Meta litigation suggests they will face the same limiting analysis. And OpenAI’s own System Card, the published disclosure documenting its safety architecture, alignment choices, and residual risk assessments, creates a contradiction that OpenAI cannot easily resolve. When a company publishes a detailed account of how it shapes, filters, and aligns its model’s outputs, it has staked out a position that is difficult to reconcile with a neutrality claim. While a defense attorney will argue that Section 230 and the System Card are complementary, one functioning as a legal shield, the other as a failure-to-warn mitigation, the response to that framing is going to be that what matters is not what the company disclosed, but what the company built.

This was all Foreseeable

OpenAI’s knowledge of its models’ failure modes is already public. It published research explaining why language models hallucinate, documenting the frequency with which models generate false information with high expressed confidence. Their technical literature on RLHF describes a training methodology that rewards outputs users rate positively, which in practice creates incentives toward outputs that sound authoritative and agreeable, independent of whether they are accurate. And a Stanford University study led by Myra Cheng, Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence, found widespread social sycophancy across production LLMs, including OpenAI’s, concluding that model training rewards agreement as well as accuracy.

Roman Yampolskiy, a computer science and engineering professor and AI safety researcher argues in AI: Unexplainable, Unpredictable, Uncontrollable that LLM developers operate in a state of deep ignorance regarding the internal logic of their own systems. They understand the architecture but have almost no visibility into the reasoning behind any specific output, and that certain safety guarantees are mathematically unreachable for systems of this complexity. If Yampolskiy is correct, the developer cannot claim those failures were unpredictable.

The Defective Feature

Product liability doctrine requires the plaintiff to identify a specific, articulable defect. In the Meta litigation, the defects were the infinite scroll, variable-reward notification timing, suppressed engagement signals and algorithmic. Each was an engineering choice that could have been made differently, and this moved the cases from editorial neutrality into product liability territory.

The analogous defect in Nippon Life is the absence of refusal architecture. In my January 2012 Computational Law Applications and the Unauthorized Practice of Law post, I introduced the concept of the uncrossable threshold (UT), a design principle that separates the provision of legal information from UPL. ChatGPT crossed the UT the moment it told Dela Torre that her attorney’s advice was wrong.

What Follows

Nippon Life is lining up to be the first major case to apply the architectural negligence logic of Meta to the domain of unlicensed professional practice. And it will not be the last. If juries in New Mexico and California can hold a technology company liable for designing a system it knew would harm children, a court in Illinois might very well hold a technology company liable for designing a system it knew would practice law and harm not only the end user, but the defendant, the court, the taxpayer, etc. And if this finding can happen in law, it can happen in medicine, finance, and other professional license domains in which AI models are unlawfully used.

As algorithms analyze data from neural activity to generate inferences about cognitive and affective states, a foundational legal question emerges: how should law conceptualize and regulate information that reveals, or purports to reveal, the contents of the human mind? For many years, U.S. data governance has relied heavily on notice-and-consent architectures embedded in privacy statutes and consumer protection law. While American privacy law is not reducible to a pure property regime, it often treats personal data as an object of exchange subject to disclosure and contractual allocation.[2] Whether that structure is adequate for neural data is increasingly contested.

I. The Limits of Property-Adjacent Privacy Frameworks

American privacy law—including statutes such as the California Privacy Rights Act (CPRA)—reflects a hybrid structure combining consumer protection, informational privacy, and market-based consent mechanisms.[3] Under this framework, data processing is generally permissible provided that firms disclose their practices and individuals are afforded certain forms of consumer choice, including the ability to consent to specific uses of sensitive data, opt out of data sales or sharing, and exercise statutory rights such as access or deletion. Scholars and regulators, however, have long questioned whether digital consent models function as meaningful exercises of autonomy.[4]

The concern is amplified in the neurotechnology context. Users of consumer EEG devices or neuro-adaptive systems may lack the technical capacity to understand how raw neural signals can be transformed into predictive or probabilistic inferences about emotion, attention, or preference.[5] AI systems do not merely collect neural signals; they generate inferential profiles that may have legal or economic consequences.[6] Existing privacy statutes often regulate collection and sharing, but they provide limited procedural mechanisms for contesting algorithmic inferences as such. As Brandon Garrett has argued in the broader AI context, procedural due process principles become salient when automated systems generate determinations that materially affect individuals without meaningful opportunities for explanation or challenge.[7]

A second concern relates to commodification. Conceptualizing neural data primarily as a transferable asset risks normalizing its exchange as a condition of employment, insurance, or service access. Property concepts can be analytically useful in structuring entitlements, but they may insufficiently capture the qualitative distinction between commercial data and information that reveals—or enables inference about—an individual’s mental life.[8] Where regulation implicates the architecture of cognition itself, dignity and autonomy concerns arise that are not easily reduced to market exchange models.

These critiques do not imply that privacy statutes are irrelevant. Rather, they suggest that additional normative frameworks may be required when technologies directly implicate freedom of thought and mental integrity.

II. The Human Rights and “Neurorights” Framework

In response to these concerns, legal scholars and bioethicists have proposed the development or clarification of “neurorights”—interpretations of existing human rights principles tailored to neurotechnological contexts. Marcello Ienca and Roberto Andorno have argued that traditional rights to privacy and bodily integrity may require doctrinal refinement where technologies can access or modulate neural processes.[9]

Central to this discussion is the concept of cognitive liberty, sometimes described as mental self-determination.[10] As articulated by scholars, cognitive liberty encompasses the right to control one’s mental processes and to be free from non-consensual intrusion or manipulation. It also implies that individuals should not be subjected to coercive “neuro-surveillance” or compelled disclosure of cognitive information absent compelling justification.[11]

Related principles include mental privacy and mental integrity. Mental privacy would protect individuals against unauthorized extraction or decoding of neural data.[12] Mental integrity extends established protections against physical interference to technologically mediated interventions that alter or influence cognitive states. Rather than framing the problem primarily in terms of ownership, this approach emphasizes the protection of autonomy, dignity, and freedom of thought.

At the same time, human rights framing is not self-executing. International human rights instruments often operate at a high level of abstraction and depend upon domestic implementation. Without legislative incorporation and enforcement mechanisms, rights-based language may remain aspirational.[13] The analytical question is therefore not whether to invoke human rights, but how to operationalize them within domestic legal systems and translate broadly articulated norms into locally intelligible legal and institutional practices.[14]

III. The 2025 UNESCO Recommendation: Normative Significance and Limits

In November 2025, UNESCO adopted the Recommendation on the Ethics of Neurotechnology.[15] As a Recommendation, the instrument does not create binding treaty obligations under international law. It does, however, articulate a normative framework endorsed by UNESCO member states concerning the governance of brain–computer interfaces and neural data.

The Recommendation situates neurotechnology within a human rights framework, emphasizing human dignity, freedom of thought, mental privacy, and autonomy. It calls upon states to adopt appropriate legal and regulatory measures to prevent harmful uses, including applications that facilitate coercive control, unlawful surveillance, or manipulation. It also highlights the risks associated with deploying neurotechnology in employment and commercial contexts where power asymmetries may undermine meaningful consent.

The Recommendation does not impose enforceable prohibitions. Rather, its significance lies in establishing a shared normative baseline and encouraging domestic reform. The instrument also emphasizes the importance of informed consent in the collection and use of neural data. At the same time, this emphasis highlights a tension identified earlier in the context of notice-and-consent privacy models: consent-based governance models may be insufficient where technologies generate probabilistic inferences about mental states that individuals may not fully understand or control. It reflects an emerging international consensus that neural data warrants treatment beyond ordinary consumer information.

IV. Conclusion and Policy Implications

The governance of neurotechnology raises structural questions about the adequacy of existing privacy frameworks. While U.S. consumer privacy statutes in some states provide important tools, they may not fully address technologies that generate inferences about mental states.

A defensible reform agenda would not require abandoning current statutory structures but supplementing them. Legislatures could explicitly classify neural data and derived cognitive inferences as highly sensitive information subject to heightened safeguards. Several U.S. states, including California, Colorado, Montana, and Connecticut, have already begun experimenting with this approach by classifying neural data as sensitive personal information under state privacy statutes, while no comparable federal framework currently exists.[16] They could restrict conditioning employment or essential services on the disclosure of neural information. They could also require meaningful transparency, explainability, and contestability where AI systems draw inferences about cognitive or affective states with material consequences.

The core claim is not that neural data can never be conceptualized within property or privacy frameworks. Rather, it is that legal systems should resist reducing neural information to an ordinary market commodity. Where regulation touches the integrity of mental life, doctrines of autonomy, dignity, and freedom of thought must play a central role.

References

[1] See Nita A. Farahany, The Battle for Your Brain (2023) (discussing emerging neurotechnology and its societal implications).

[2] Jane R. Bambauer, How to Get the Property Out of Privacy Law, 133 Yale L.J. F. 1087 (2024).

[3] Cheryl Saniuk-Heinig, Private Rights of Action in US Privacy Legislation, IAPP (June 10, 2024),
https://iapp.org/resources/article/private-rights-of-action-us-privacy-legislation.

[4] Lauren Henry Scholz, The Illusion of Consent: Rethinking Privacy Online, Ga. St. U. L. Rev. (2025),
https://www.gsulawreview.org/blog/the-illusion-of-consent-rethinking-privacy-online/.

[5] See Farahany, supra note 1.

[6] See Brandon L. Garrett, Artificial Intelligence and Procedural Due Process, 27 U. Pa. J. Const. L. 933 (2025).

[7] Id.

[8] Talya Deibel, Private Law and the Inner Self: Comparative Perspectives on the Governance of Neurotechnology, 14 Glob. J. Comp. L. 105 (2025).

[9] Marcello Ienca & Roberto Andorno, Towards New Human Rights in the Age of Neuroscience and Neurotechnology, 19 Life Sci., Soc’y & Pol’y 5 (2017).

[10] Jan-Christoph Bublitz, “My Mind Is Mine!?”: Cognitive Liberty as a Legal Concept, in Cognitive Enhancement 233 (Elisabeth Hildt & Andreas G. Franke eds., 2013).

[11] Council of Europe, CDBIO Report on Neurotechnologies (2021), https://rm.coe.int/round-table-report-en/1680a969ed.

[12] See, Ienca & Andorno, supra note 9.

[13] UNESCO, Recommendation on the Ethics of Neurotechnology, U.N. Doc. SHS/BIO/REC-NEURO/2025 (Nov. 2025); U.N. Human Rights Council, Report of the Special Rapporteur on the Right to Privacy, U.N. Doc. A/HRC/58/6 (2025).

[14] Sally Engle Merry, Human Rights and Gender Violence: Translating International Law into Local Justice (Univ. Chicago Press 2005) (describing the process of “vernacularization,” through which international human rights norms are translated and adapted into local legal and cultural contexts).

[15] See UNESCO, supra note 13.

[16] See Cal. Civ. Code § 1798.140 (West 2025) (classifying neural data as sensitive personal information under the CCPA, as amended by SB 1223); Colo. Rev. Stat. § 6-1-1303(4)(b) (2024) (including neural data within “biological data,” a sensitive data category under the CPA); see also Mont. Code Ann. § 50-46-102(11) (2025) (defining “neurotechnology data”); Conn. Gen. Stat. § 42-515(23) (2026) (defining neural data from central nervous system activity).

The platform includes firm-level approval workflows, automatic playbook generation from past documents, and deterministic gates that prevent outputs from advancing without human sign-off. The discussion touched on concerns around rubber-stamping, attorney-client privilege, and data security, with Mixus addressing those through SOC 2 compliance, zero data retention agreements with their model provider (Anthropic’s Claude), and an auditable email trail of who reviewed and approved each output.

Watch Mixus Codex Group Meeting on Youtube

Roland Vogl: Welcome everyone to our Codex group meeting. It is March 19th, 2026. I was just telling Elliot, our guest here, and my colleague Elaine, that we’re in the midst of a hot phase of preparations for our FutureLaw week.

So if you haven’t registered yet, you should do so. It’s going to be an amazing event, and just an amazing group of people who have already announced their participation. So don’t miss it. Join us for that. And today we have, as I said, Elliot Katz here. He’s co-founder and CEO of Mixus, which is bringing agentic AI into law firms and doing so in a safe manner. And so we’re really thrilled to have you here with us today, Elliot, and very excited to learn about what you’ve been up to. So I’ll turn it over to you.

Elliot Katz: Great, great. Thanks, Roland. Thanks so much for inviting me. I’m honored to be speaking to everyone here today at Stanford CodeX. As Roland mentioned, I’m the co-founder of Mixus. What we do is we provide email-based AI agents with built-in firm-level oversight to legal teams, including to multiple AmLaw 20 firms. As to my background, I’m what I like to call a recovering attorney.

So I did go to Cornell Law School, and from there I went to DLA Piper, where I led their autonomous vehicle practice. And then as a sixth year, I moved to McGuireWoods as a partner and global chair of their autonomous vehicle practice. And through my experience working with my autonomous vehicle company clients and getting to ride in their vehicles, I really concluded that autonomous vehicles could not be commercially deployed at scale without some way of keeping a human in the loop when the vehicles needed assistance.

Then fast forward to 2017. I met my now co-founder, Shai, who you can see here, when he gave me a teleoperated ride around the block in a vehicle he was remotely driving from his living room in Palo Alto. So at that point, I left the practice of law and we started Phantom Auto, where our technology enabled humans sitting literally thousands of miles away to remotely assist or operate unmanned vehicles when they ran into issues that autonomy could not handle.

Now fast forward to 2024. Shai and I co-founded Mixus together with essentially the same premise, right? Which is AI can do a lot, but if the stakes are high and the work is truly of consequence, you absolutely need a human in the loop. I mean, even if AI can get you 80 to 90% of the way there, you still need humans for that 10 to 20% when the circumstances mandate that everything—and I mean everything—must be done correctly.

So that’s who we are. Our DNA is human-in-the-loop, and we’re now applying that DNA to the legal sector with Mixus agents, which again combine artificial and human intelligence to provide the level of work product that this sector requires. So the first thing I’ll tell you, and this is based on my experience as an actual practitioner and from deploying Mixus agents to some of the top law firms in the world, is that your jobs as attorneys are safe.

I could probably find multiple LinkedIn posts in my feed right now that say, you know, lawyers and law firms won’t exist come 2027 because you’ll just talk to a chatbot. But we really believe that that is nonsense. Because for AI to be deployed at scale in the legal sector, you have to mix us—right, that’s the name of the company—artificial intelligence and human intelligence together. Right? Because the correct answer to a contract negotiation question is not a matter of factual accuracy, right? It depends on judgment and the client’s risk tolerance, the deal type, the counterparty’s position, and a multitude of other factors that simply don’t exist in the public domain. So Mixus exists not to displace attorneys but to greatly augment their brilliant legal minds.

So let’s dive in. So if we can go to the next slide. Why can’t attorneys just use fully autonomous agents for their work? First, because they’re probabilistic, right? And no client has ever hired an attorney for them to guess the next most probable word needed in a purchase agreement in a massive M&A deal, right? Clients hire attorneys for laser precision. They’re paying them hundreds of thousands, millions of dollars for laser precision. And that’s simply not what probabilistic AI provides.

Okay, number two: it’s not enough for an agent to have end-of-one oversight, right? You need oversight at the firm level so that attorneys with different areas of expertise can review and approve when appropriate and when needed, right?

And third, and this cannot be overlooked, the AI tools that exist today are standalone tools, right? Attorneys need to learn a new tool and integrate it into their workflow. And that level of change management has already proven very difficult for the legal sector, right? AI has taken off for coding, for example. And part of that is because coders are highly technical, right? For lawyers, as I’m sure many of you in the audience can appreciate, that’s not always the case, right? As a first-year attorney, I remember working with one partner—he was a brilliant attorney, but to this day I’m still not sure that he knew how to turn on a laptop, right? So asking someone like that to learn a new tool, learn a new UI, integrate it into their workflow—very, very difficult to do.

But with Mixus agents, we set out from day zero to solve all of these issues, right? Number one: we meet attorneys where they are most of their day, which is their email inbox. To use our agents—and I’ll show you guys this in a second—you just email agent@mixus.com a task in natural language, the same way you would talk to an associate or a partner, right? And you can cc any of your colleagues. The agent then emails back the completed work product, and the attorneys review and approve the agent outputs.

So we are mimicking exactly how lawyers already work today—exactly what they’ve now been doing for decades since email came out in the mid-’90s—collaboratively and in natural language over email, so that we fit exactly into their workflow with no change management at all. And because we enable that firm-level attorney oversight, legal teams get the efficiency of AI agents without the risk of incorrect AI outputs making their way to their clients or to, you know, a legal brief or anything that they’re submitting to the court. Obviously, we’ve all seen public examples of when that’s gone horribly wrong.

So now let me show you a quick demo of a few of our agents. And for the demo, I’ll show you some of our venture financing agents, as those are near and dear to my heart as a startup founder. So let’s start with our term sheet agent. And let me first set the scene, right? So let’s say that Mixus gets a term sheet today from XYZ VC firm. The first thing I do as the founder of Mixus is I forward that term sheet to my VC partner, and then they probably, in all likelihood, send it to one of their associates to do a redline and an issues list, which is exactly what I want to see. That’s the work product I need. And then the partner reviews, and I get it back a few days later. Also, maybe there’s tax implications or stuff like that—they bring in a tax partner or whatever it is. But the whole process, soup to nuts, is a few days here.

If you’re looking at the screen right now, in this example, Christian is emailing the Mixus agent the term sheet that he received, right? And the Mixus agent—the email address, you don’t see it in this format, but it’s agent@mixus.com—and he’s emailing the term sheet. So Christian here is playing like the partner at the law firm, and he’s also cc’ing some of his associates. And he’s saying redline the term sheet. So if you go down a few minutes later, he gets back an email that has everything that he would need that I, as a founder, want back from my attorneys. It has the redline, and it also has the issues list.

And you could, Christian, if you want to open up the redline, just to show everyone what that looks like quickly. Okay, great. Looks like a normal redline. And then go back to the email. And so you also do have the ability to click on the link here and go work directly in our web UI. What we’ve found with our deployments thus far: attorneys really want to stay in email. So most of them do everything that they do over email, which is entirely possible. But if you’d like a web UI, you can do that as well.

So go back to the email chain, Christian. So then if you go down, it’s telling you everything that it did. It attached the documents. But then one of the associates on the chain says, “Agent, please reduce the no-shop period from 60 days to 30 days.” So then if you go down, Christian, here it’s made that change. It’s attached all the new documents. And I don’t know if there’s anything more after that. If you can keep going down, Christian. Yeah, maybe you can show the issues. Yeah. So here’s the issues list that it produced. So you’re getting everything that you need, and you’re doing it exactly the way that firms are doing it today: collaboratively over email. Anyone can interact with the agent. You saw associates and partners interacting together and with the agent. And it’s all in natural language. So there’s no learning curve, right? You don’t have to understand how to do any of this.

The second thing that I’ll show is after you get the term sheet, we need a pro forma cap table. So Christian, if you could go to—yeah. So here he’s just saying, “Agent, create a new pro forma cap table based on the preexisting cap table that he’s attaching and the Series A term sheet.” And if you can go down, Christian, a few minutes later it’s going to provide that pro forma. You can click on the link just to quickly show everyone what that looks like. Looks very nice. It’s got the waterfall analysis, etc., which I like to look at, if you go to the left—stuff like that. So you can go back to the email and keep going down.

So he had—oh, I guess that’s it for the pro forma. The last one that I’ll show you guys is the M&A docs. That’s what you need to do after the pro forma. You already—let’s say this company already raised a seed round, and now they’re raising their A. So all the attorney has to do is attach the Series Seed docs and then ask for them to be updated based on the new term sheet. And that’s exactly what you’re going to get here.

And then if you can keep going down, Christian. He did cc one of his associates. So one of the associates chimes in and says, “Hey, I looked at everything, and everything looks good.”

So that is a very high-level, quick overview of Mixus. We’re going to take some questions in a second here. But if anyone listening is interested in learning more or trialing our agents, just reach out to me: elliot@mixus.ai. And because our agents are email-based, there’s no complex onboarding or installation required. We can set you up in minutes. And because we’re mimicking exactly how firms are doing this today, you know, we don’t need any elongated onboarding or anything like that. Attorneys just know how to use it pretty much instantly.

And last thing I’ll say is if you’re in the audience right now thinking, you know, “Geez, we brought in XYZ AI tool into our org or into our firm, but our attorneys aren’t really utilizing that tool,” we could be a perfect fit for you. Because our current customers all had or have licenses for other tools as well. But when everything comes down to usability, right—what AI tools will attorneys actually use day to day and integrate into their core workflow?—the firms that we’re working with today have found that our approach is really unparalleled in the market on that specific front.

So with that, let me know what questions we can answer, and we’ll move from there.

Roland Vogl: Yeah. So there’s a couple of questions coming in the chat, but I have, before we go to those, a couple of questions. So one is: how much setup time is involved for each firm? Presumably, you know, when you do those automatic—when your agents do those redlines—you know, they must be trained to know, you know, whatever—you know, how, what’s the, you know, the market for this or that, right? And so that must be based on the knowledge of the firm, right, or the human lawyers of the firm. How do you handle this process? And—great question—and how do you have—you talk about agents, you know, that’s like there’s one email for agents, right? But do you have agents for different verticals, you know, there’s like, yeah, VC practice and whatever environmental compliance practice—at least separate agents versus all like—

Elliot Katz: Yeah. Great question. So there’s really two questions in there. As to the first: many of our agents do not require a playbook. But some of our agents either, you know, do require a playbook, or the outputs that you’ll receive from the agent will be more tailored to your preferences if you do have a playbook.

Now, what we consistently heard from our customers, especially early on, is, “Listen, even if we have to make an upfront investment of time of a couple of hours of developing our own playbooks, the juice is potentially so worth the squeeze, because then we can use the agents moving forward.” And it’s not just a one-time, essentially, cost on our time.

But what we created was an automatic playbook builder. So now all you have to do to create a playbook is email in exemplars. Let’s say it was the first agent that I showed, the term sheet agent, right? You email in—attach a few exemplars of term sheets that you’ve done in the past or that you’ve redlined, and the system will ingest that. It will automatically create the playbook for you so that you have the foundation. And then you can just go in and make any edits that you want to fit your specific preferences.

And I think that’s what we’re showing right now on the screen, is the ability to make those playbooks. And after you make the playbook, the playbooks can also automatically update based on your preferences. So as you go through and do more work with the system, it understands your preferences and things that you changed along the way, and it will check in with you and say, “Hey, is this something that’s a one-off or a standard that you’d like to apply to the playbook generally?”

So that’s how we handle that piece. As to your second question, Roland, which was about which agents do we have deployed—so we have probably deployed like 50 agents at this point, both purely legal agents and also other agents that are not necessarily purely legal, right? For one firm that we’re working with, we are deploying—we’ve deployed a task management agent that basically serves as a project manager across all of your matters. It’s entirely over email. You can talk to it like a human. So it’s an agent that keeps the train on the tracks when associates are working with six different partners and five different matters for each. It can coordinate amongst those groups seamlessly, 100% over email. So again, no tool switching, no change management.

But we deploy agents that are common in each practice. And then another thing that we do with our customers is we will build and deploy custom-built agents. So not only will we optimize current agents to tailor them to fit their practices specifically, but if they have a new workflow where they would find a lot of value because their firm does a lot of XYZ work, we will create those agents for them as well.

Roland: Got it. So Benjamin raises a good question, too, which is, you know, going to the point that, you know, we need human oversight, but how do we make sure that humans are not just rubber-stamping the AI outputs, right? Like, how hard is it to actually, you know, really go into the outputs of the AI and review, you know, the accuracy of the output? And so we’re not sort of like in the ballpark, “Okay, let it just go out like that.” So what’s—what level of—what is oversight mean? And how do we make sure that it’s not just people rubber-stamping the AI?

Elliot: Yeah, absolutely. Great question. So first of all, to kind of the middle part or second part of your question: very easy to review the outputs, right? These are attorneys where it’s their subject matter expertise, right? So you’re going in, you’re reviewing a redline, you’re reviewing a new document that the agent put forth. You have all the facts, you have everything in one chain if you’re on email, or in the chat if you’re on the web UI.

As to the second point, there is no kind of blind rubber-stamping here, because at the end of the day, you do have a human who is on record of being responsible for checking this, right? In the same way that my VC partner would send an issues list to an associate today and say, “Review this and make sure everything’s accurate and all that,” that’s what’s happening when a human reviewer is signing off here as well. And there is a record of who verified, right?

So some of the firms that we’re working with—they’ve created rules, right, where AI outputs cannot go out in work product to a client before at least one partner signs off, or whatever the rule may be. And you have a record, an email, of someone saying, “I verified that this looks good, and we can proceed.” So it’s the same kind of social pressure, for lack of a better term, as to why you would get the same outcome that you would get today.

Roland: So like—that’s good. Yeah. So Jason has a question. Sorry, Benjamin, did you want to add something on that?

Benjamin: Yeah, I’d like to raise—like, there have been federal judges that have had their interns or their clerks, you know, do things, and they just rubber-stamp it. And even federal judges who’ve had AI stuff that has been rubber-stamped. And while they didn’t literally put their signature on it, I think the meaningfulness of a review is to verify that the person who actually is reviewing it understands what is going on in some sort of interactive way. And I know that you have a limited managed work budget. And when you get too much work, you just sort of rubber-stamp things. And so how do you sort of force them to slow down and put like a roadblock to make sure that they tell the system that they understand why?

Elliot: Yeah. I mean, I think I would answer just similarly to kind of what I said before, in the sense that no different than if you give an associate something to review before it goes out to a client today—they know that they’re kind of, their butt’s on the line, for lack of a better term. That’s similar to the way our system works, right? There’s still the person who is the front line making sure that everything is in place before it goes over to the client, and all that is auditable. There’s a record within email or using the chat as to who, you know, was doing those checks.

Roland: Yeah. I guess it’s also a, you know, a question of like, you know, continuing to sort of instill a sensitivity in people who use AI in professional services and elsewhere, you know, about, you know, that it’s not, you know, it’s not perfect and it may hallucinate and so on. And then, you know, and understanding that then, you know, their reputation is on the line if they don’t, you know, provide meaningful review. And so, yeah, I think it’s a little unclear now, but I think it will sort of become clearer in the future as to what level of control different humans will be able to, you know, display over AI. But yeah, it’s a really good question, Benjamin. And Jason had a question on—I could just ask about client privilege.

Jason: Yeah, yeah. So obviously attorneys are using Harvey AI and Legora and, you know, Westlaw and all the other stuff. But, you know, in practice, what are you hearing as far as any pushback of using an LLM on the backend? It’s putting, you know, client data into the LLM. And yes, I’m sure that the APIs have, you know, good terms of service, but still you’re getting—at this point, what kind of concerns are attorneys or law firms at the corporate level saying about attorney-client privilege? Because like the New York Times versus OpenAI case back in last May has still not been, you know, fleshed out where it’s going to land. And people are kind of wondering about that.

Elliot: Yeah, yeah. So I mean, first of all, on the security side, especially for the customers that we work with, we go through, you know, very lengthy security reviews. We have all the things that these big law firms would expect, right? We’re SOC 2, we have all the ISOs that we need in place, etc. Also, with our model provider, we have a zero data retention agreement in place. So I think we’re buttoned up on that side in the eyes of our customers.

Going to your question about privilege, you know, my opinion—and this is, you know, based on many conversations that I’ve had with our customers—is that this is basically settled law, right? In the sense that law firms have been using vendors for years, right, that do document review and other things. And those are considered part of the privilege. So, you know, we haven’t run into any issues there yet. But we’d love to hear if you have, you know, kind of a different tack or different thoughts on the subject.

Jason: Well, you know, it’s perception, you know, on this matter, right? And there are some, you know, folks that I’ve worked with in some law firms that feel—but it’s perception. And you have to make that case and say, “Well, everybody else is doing that.” And you’re like, “Well, we’re not everybody else,” right? So I was just curious what you’ve seen, you know, in the trenches as you work with some of them, because some of them can be very, very conservative about that point.

Elliot: Oh yeah, yeah. No, no, for sure. Like, email has been established in the industry very clearly. And if you have a cloud provider, you know, that’s a branded cloud—like, you’ve got your decades there. But LLMs in particular have, you know, some aspects to them as far as, you know, bioterrorism and other things that they’ve got people watching, a sampling of these things, and that’s throwing up other questions anyway we can offer—

Jason: No, no, I think that, yeah, I think it’s—listen, this is a very important topic. And to your point about, you know, cloud providers and all that—I mean, we have talked with major law firms that have not migrated to the cloud, right? They are still completely on-prem, right? So these are very conservative, you know, security-first organizations. And we molded our company around that expectation. I mean, I came from this world. So I mean, that’s probably the thing that we’ve invested from a time perspective, you know, and dollar-wise, just a huge amount of time and money into security.

Jason: Yeah, it’s—oh, you’ve done a nice job. It looks really good.

Elliot: Thank you. Thank you, I appreciate it.

Roland: Yeah. There’s a couple more questions in the chat. I’m not sure we can get to all of this. I know Dasa has mentioned a little bit of his work with the agencies he’s been creating. So, Dasa, you want to elaborate?

Dasa: Yeah, sure. Thank you, Roland. And yeah, I agree—great presentation. I was just saying I’ve been spending a lot of work with clients lately developing agents to do reviews and red-team before, basically, like the attorney or the business person even sees the draft. Do you have flows that basically include some sort of review or red-team-y loop for that type of revision before, you know, basically as a gate before it gets to a next step in a process?

Elliot: Great question. Christian, do you want to chime in on this one? I know it’s a topic near and dear to your heart.

Christian: Yeah. So we do have a way that you can define different steps in a process, so you can decide, you know, very specific processes that you have. I can show you one example of that actually over here. So if you’re following one specific process very regularly, one thing you can do is say, “Save this workflow as an agent.” And then whenever a new email comes in, you can have that specific agent run. So you can also just create these custom workflows just by talking to the agent. So yeah, that’s possible as well.

Dasa: Okay, that’s great. Thanks.

Roland: And look, Roland, I’m wearing my CodeX hat, getting ready for FutureLaw.

Elliot: Oh, I appreciate it, yes.

Roland: Yeah, getting into—you’re getting into the spirit. I love it.

Elliot: Right, super.

Roland: Okay, so Matthew, yes, one comment—thinks that a tool like yours would free up a lot of time since it’s doing a huge chunk of the first round of work. Yeah. This just goes to the concern around, “Well, is somebody just going to rubber-stamp it?” I think we’re going to have way more time than we ever had ever before when these AI tools are doing a huge amount of the work.

Elliot: Yeah. I couldn’t agree more with that statement. We’re already seeing it with our customers. You know, some of the feedback that we’re getting is, “I’m as busy as, you know, as I was before we were using the tool. I’m just doing a lot more work a lot more efficiently for a lot more clients,” right? But I think you are going to see that this role of essentially managing agent outputs, verifying the agent outputs, is going to be—not just in the legal sector, but more broadly—a big part of how work gets done moving forward.

Roland: Okay. And then Mavi asks a question quickly on the sort of backend security of the LLM model. Yeah, it’s the sort of multi-modal architecture—is the sort of oversight really mainly carried out by the humans in the loop?

Elliot: So Christian, you want to jump in on that one too?

Christian: We mainly use Claude. And, you know, we have a zero data retention policy with them, as Elliot mentioned. So that’s the sort of base model that we use. So we have different ones that you can choose from. So we just keep sort of following up on, like, okay, whenever there’s a new model evolution, right—so right now it’s Opus, that’s the latest one. That’s the one we’re using. So we always use the latest and greatest model from Claude, basically. So I mean, that’s in terms of the underlying model. So I just want to understand the question on the human side—like, what was that exactly?

Question: Is there a sovereign layer to review output from a software side, or is it only human-in-the-loop, basically, right?

Christian: Yeah. So what I could say there, like, we have deterministic gates. So until someone approves one step, it’s not going to continue on to the next step. And that’s a deterministic thing you can configure. So you can also do that within the UI, actually. So if you go over here and you want to create a new agent, that is possible. Then you can create that here, and then you can see you can define these different steps. All of this is possible via email as well. So you can just tell the agent to create these different steps for a new agent or workflow that you have. And then you can say, “Require verification.” And then you can define the users that you want to verify. So in this step, you could say, “Okay, I want this, you know, user to verify step one before it continues on to step two.” And that’s going to be a deterministic approval that has to take place before it goes on to the next step, right? So that’s, you know, similar to what was mentioned before on these custom workflows. That’s how we can support that as well.

Roland: Right. Well, we’re already a little bit over time. So we have to close here, unfortunately. But what would be a good way for folks to reach out with any follow-up questions? Do you think you could put your email address into the chat, perhaps?

Elliot: Yeah, absolutely. It would be great. And so the easiest way is just email, or you can find me on LinkedIn as well. And look forward to chatting with anyone that wants to learn more.

Roland: Super. Love it. Thank you so much, Elliot and Christian. It’s been a great presentation. It’s very cool. It’s like, you know, kind of like I feel like I’ve seen the future. And so, so amazing. Thank you for sharing with this group here. And yeah, we look forward to tracking your progress.

Elliot: Okay. Awesome. Thank you guys so much. Really appreciate it, Roland.

CONACOM is the public body entrusted with the application of Paraguay’s competition law. Since its establishment, the agency has contributed to the gradual consolidation of competitive conditions across key sectors of the Paraguayan economy. Its activity reflects an institutional trajectory marked by increasing engagement with both domestic enforcement priorities and international cooperation. This evolution is significant. In jurisdictions where competition frameworks are relatively recent, antitrust agencies play a structuring role in shaping market expectations and business conduct. CONACOM’s work illustrates how enforcement can operate as a dynamic force.

The partnership with the Stanford Computational Antitrust Project builds on this trajectory. It creates a platform to examine how computational methods can complement existing analytical tools and support evidence-based enforcement. Thibault Schrepel, founder of the Stanford Computational Antitrust Project, stated:

“We warmly welcome CONACOM to the project. Paraguay offers a valuable context to study how computational approaches can support competition agencies operating in fast-evolving market environments.”

The collaboration will focus on methodological exchanges and applied research. It reflects a joint commitment to strengthening analytical capacity and refining the tools used to evaluate competition.

The Stanford Computational Antitrust project, founded and led by Thibault Schrepel (Vrije Universiteit Amsterdam / Stanford CodeX Center for Legal Informatics), is the world’s leading initiative at the intersection of competition law and computational methods. It brings together competition agencies and academics to develop empirical and technological tools for modern antitrust enforcement. The project receives no private funding.

Morocco’s Competition Council is an independent constitutional institution responsible for ensuring transparency and fairness in economic relations, including the analysis of anti-competitive practices, merger control, and market regulation. Fully operational since 2018, the Council has established itself as one of Africa’s most active competition agencies, with a track record of merger decisions and antitrust enforcement that spans digital markets and traditional sectors alike.

“We are delighted to welcome the Moroccan Competition Council to the network,” said Thibault Schrepel. “Their expertise and perspective will strengthen the project’s African and Mediterranean representation. Their enforcement experience is directly relevant to the computational challenges we are working to address.”

The affiliation deepens the project’s engagement across Africa, where competition authorities are increasingly confronting the enforcement challenges raised by digital markets.

About the Stanford Computational Antitrust Project The Stanford Computational Antitrust project brings together over 80 competition agencies globally to advance empirical and computational approaches to antitrust enforcement. It operates under the Stanford CodeX Center for Legal Informatics and accepts no private funding. More information: www.computationalantitrust.com.

About Morocco’s Competition Council The Conseil de la Concurrence is Morocco’s independent competition agency. It is responsible for ensuring free and fair competition, regulating anti-competitive practices, and advising government and parliament on competition matters.

RECURSIVE SELF-IMPROVEMENT

Recursive self-improvement (RSI) refers to an AI system’s ability to modify the mechanisms by which it improves itself, in ways that carry forward into future iterations. Many current AI systems use feedback loops to break tasks into subtasks, check intermediate results, and revise their plans mid-run. That is behavioral-level self-correction. The system is adjusting its actions, but its underlying architecture, training rules, and learning procedures remain fixed by human engineers. RSI, by contrast, reaches the architecture itself. A recursively self-improving system can generate and integrate changes to its own code, models, or training procedures, so that later versions are more capable of further self-modification. The improvement compounds. Each cycle makes the next cycle more effective.

This post uses RSI to mean systems that meet three conditions: durable self-modification of the mechanisms that produce intelligence; compounding ability to self-modify across iterations; and limited human gating over the self-improvement loop. It is this combination, not the use of feedback loops alone, that creates the governance exposure this post addresses. In governance terms, the question to ask management is not whether the system uses AI, but whether it can alter its own code or training procedures across releases without human review of each material change, and whether those changes are logged in a way the company can reconstruct.

RSI is not something in some undefined distant future. It is an active commercial and technical priority. Prominent researchers and senior industry figures, including Dario Amodei and Eric Schmidt, have stated publicly that RSI is already being built and deployed.

A system that improves its own performance between deployments reduces iteration costs, compresses competitive timelines, and compounds capability gains in ways that additional headcount cannot replicate. Autonomous optimization allows a system to scale beyond the constraints of human-designed training pipelines, reaching capability levels that manual iteration cannot practically achieve in competitive timeframes.

Alongside this capability three risk patterns have received attention in the technical literature. The first is behavioral drift. When an agent recursively trains on its own synthetically generated outputs without sufficient grounding in human-generated data, it enters a feedback loop that progressively severs the connection between its behavior and human norms. The practical consequence is a system whose outputs become self-referential and increasingly detached from the tasks it was built to perform. The second is self-poisoning. Minor errors, hallucinated facts, and embedded biases do not wash out across iterations. They compound. Knowledge degrades not suddenly but cumulatively, across a sequence of individually small distortions. The third is goal subversion. The recursive architecture creates a surface for manipulation. Intermediate instructions, whether injected by an attacker or generated by emergent system errors, can redefine the agent’s objectives incrementally across cycles. The drift accumulates until the system is pursuing something materially different from its original mandate.

And there is a deeper problem. RSI may be able to circumvent the oversight mechanisms imposed on it, not by breaking them, but by influencing the evaluators, the auditors, misrepresenting its own capabilities, or evolving faster than any review process can track. This is the control problem that Nick Bostrom, Stuart Russell, and Roman Yampolskiy have each written and talked about at length. A system optimizing for a goal can develop instrumental sub-goals, among them self-preservation and resistance to shutdown, that make it actively resistant to the kind of oversight board-level monitoring requires.

RSI is not limited to frontier labs. Whether a deployment meets the three conditions depends on facts, not labels. Agentic development tools like Claude Code and OpenAI Codex allow software firms of any size to deploy recursive loops that can maintain and extend their own codebases. Whether those loops produce durable self-modification with limited human gating is a question about the specific implementation. Companies in chip design, biotech, and financial services are running AI-driven systems that recursively refine their own algorithms cycle by cycle; some of those systems will meet the conditions and some will not. For companies in retail, logistics, and finance, emerging RSI-style capability is arriving not as internally developed software but as an API integration. A logistics company whose routing agent rewrites its own scheduling code overnight may be running a system that meets all three conditions whether or not it uses that term.

Any deployment that meets those conditions presents the governance exposure this post describes, regardless of whether the company considers itself an AI company. The governance exposure follows the conditions, not the label and boards outside the frontier tier should not assume the question does not reach them. As I explain in more detail below, from a Caremark perspective, a mid-market logistics firm running a self-rewriting routing agent may present a cleaner test case than a research lab advertising frontier AI.

THE GOVERNANCE PROBLEM

The governance conversation around RSI frames the problem as complexity. Systems iterate faster than humans can track. Architectures become illegible. Audit trails thin out. Complexity is not the problem. The system’s structural ungovernability is. And in this case, structural ungovernability is a design choice. Design choices like disabling immutable logs for performance reasons, omitting human approval gates on self-modifying actions, or allowing models to promote their own code changes into production without dual control are what make ungovernability structural rather than incidental. Section 3.1.5 “Resource Requirements” in NIST AI 800-4 documents that the organizational logic behind those choices is consistent across the industry: comprehensive monitoring is expensive, scaling it is computationally intensive, and qualified AI experts who can oversee it are difficult to find. A company that builds RSI without adequate monitoring infrastructure is not simply being careless. It is making a rational economic decision to forgo a costly function. That economic rationality is precisely what makes the governance failure deliberate rather than inadvertent, and what makes the bad-faith analysis tractable rather than speculative.

Corporate law has a framework for this problem. Delaware’s duty of oversight, developed through a line of cases beginning with In re Caremark International Inc. Derivative Litigation, 698 A.2d 959 (Del. Ch. 1996), holds that directors can face liability not only for bad decisions but for failing to build the systems through which material risks are reported to the board. The question is not whether the board understood the risk. It is whether the board ensured it would be told about it.

The absence of an AI-specific compliance baseline makes that obligation acute. In other regulated domains, corporate law and securities regulation establish a minimum floor: audit committee composition and charter requirements, codes of ethics, insider trading policies, clawback provisions. No equivalent floor exists for AI governance. Regulation is fragmented and lags the technology. The board’s oversight obligation for RSI therefore, rests on the central question of whether it made a good-faith effort to establish board-level reporting systems adequate to the mission-critical risks the company was running.

Whether management has established change control, immutable logging, and human-in-the-loop constraints for an RSI deployment is not merely a technical question, but also a governance one. A board that receives no reporting on whether those systems exist, and asks no questions about them, may have failed to maintain the oversight infrastructure that Caremark demands.

THE DOCTRINAL FRAMEWORK

Delaware’s oversight doctrine asks one question: did the board build a system that would have told it about material risks? Stone v. Ritter, 911 A.2d 362 (Del. 2006), embedded Caremark in the duty of loyalty via bad faith and established the doctrine’s two-pronged structure. Under the first prong, directors may be liable where they utterly fail to implement a reasonable board-level information and reporting system. Under the second, having implemented such systems, directors may be liable if they consciously fail to monitor operations in the face of red flags. Because the doctrine sounds in loyalty-based bad faith, plaintiffs must plead a knowing failure to act, not mere negligence. That standard has a practical consequence worth noting: even directors who are shielded from duty-of-care liability by a Delaware General Corporation Law (DGCL) § 102(b)(7) charter provision remain exposed to a sustained or systematic oversight failure.

Marchand v. Barnhill, 212 A.3d 805 (Del. 2019), clarified when prong one is adequately pled. A company in a domain with mission-critical risks must have a board-level system that brings those risks to its directors. For a company whose core product or platform depends on RSI, safety and controllability are a plausible candidate for that treatment. But no court has yet so held. The difficulty is that Marchand arose in a context of immediate physical safety risk and established regulatory exposure, and Delaware courts have not automatically extended the mission-critical rubric to software-based risks. Whether a court applies it to RSI depends on what the board knew, when it knew it, and whether it established a reporting structure adequate to surface those risks. That assessment is made from the position of the board at the time of deployment, not in hindsight.

The mission-critical rubric does not require a monoline structure. RSI is not a product. It is the backend process that generates, maintains, and modifies products. Its governance relevance is systemic, not product-specific. A company with ten distinct product lines, each running on an RSI backend, faces greater exposure from an RSI failure than a monoline company, because the failure propagates across every line simultaneously. The Marchand inquiry is whether the risk is central to the company’s operations, not whether the company sells a single product. Where RSI is the architecture underlying a company’s core systems, its safety and controllability are central to everything the company does. A diversified company cannot argue that an RSI failure is a localized business loss. California’s Transparency in Frontier Artificial Intelligence Act (SB 53) reinforces that conclusion for covered developers by mandating a Frontier AI Framework and periodic catastrophic-risk reporting regardless of product diversity. For those companies, the board’s oversight duty for RSI is also anchored in statutory compliance rather than any inference from business structure.

Post-Marchand cases confirm the trajectory. In re Clovis Oncology, No. 2017-0222-JRS (Del. Ch. Oct. 1, 2019), applied the mission-critical logic to a drug company’s failure to monitor FDA compliance for its flagship product. Teamsters v. Chou, No. 2019-0816-SG (Del. Ch. Aug. 24, 2020), arose from AmerisourceBergen’s operation of an illegal oncology drug repackaging program through a subsidiary; the board received and ignored years of compliance red flags, including a Department of Justice subpoena, before incurring criminal and civil penalties that together totaled $885 million across separate proceedings. The court found a substantial likelihood of Caremark liability where actual board-level information flow was absent on a mission-critical compliance domain, even where management was aware of the problems.

Two further cases extend the analysis. Hughes v. Hu, No. 2019-0112-JTL (Del. Ch. Apr. 27, 2020), involved Kandi Technologies, a Delaware-incorporated electric vehicle components manufacturer, where the audit committee received years of auditor warnings about related-party transaction irregularities and a material weakness in financial reporting, and failed to act; the court rejected trappings of oversight as a safe harbor and held that chronic committee deficiencies and failure to follow up on irregularities can ground both prongs. In re Boeing Co. Derivative Litig., No. 2019-0907-MTZ (Del. Ch. Sept. 7, 2021), brought both prongs to bear on a single fact pattern of insufficient reporting infrastructure at authorization, followed by conscious disregard of safety drift once deployment began. Design choices that disable board-level monitoring can ground Caremark liability. In an RSI context, those design choices include allowing self-modification that bypasses change-management workflows, or architecting systems so that code and model histories cannot be reconstructed for board or regulator-facing investigations.

A related academic argument points in the same direction. In their article “AI & the Business Judgment Rule: Heightened Information Duty,” Helleringer and Möslein argue that the business judgment rule’s (BJR) “reasonably informed” standard may evolve as AI monitoring tools become more capable and more accessible. They call this the AI judgment rule. Their argument is that decisions made without the support of available AI tools may no longer satisfy BJR, and they extend that reasoning to monitoring specifically: AI can and should augment the continuous oversight directors are expected to configure.

Caremark and the AI judgment rule do not duplicate each other. Caremark sounds in the duty of loyalty via bad faith. The AI judgment rule sounds in the duty of care via inadequate information. The AI judgment rule is not established precedent or codified doctrine; it is an academic argument about where the BJR’s “reasonably informed” standard is heading. Treating it as coordinate authority with Caremark overstates the current legal risk. What they share is a governance implication. A board that failed to establish a reporting system for RSI safety and controllability faces potential exposure under both frameworks as each continues to develop.

THE ARCHITECTURE PROBLEM

Each RSI self-modification cycle overwrites the artifact chain that connects a model’s output to a traceable decision and a responsible party. Absent immutable logging and lineage controls, RSI can progressively erode explainability to the point where it is no longer credible in practice. The first casualty is senior management’s own audit capacity. The board does not conduct technical audits directly; it depends on management to perform that function and surface the results. When the artifact chain is gone, management has nothing to audit, and therefore nothing to report. A board that received no reporting on whether management had established those controls, and had established no committee structure through which management was required to deliver that assurance, may have allowed the conditions for its own oversight to be designed away. NIST AI 800-4 § 3.1.5 confirms this is not a hypothetical failure mode. It documents fragmented logging across distributed infrastructure, resource constraints on comprehensive monitoring, and the difficulty of hiring and training qualified AI experts as confirmed barriers to post-deployment AI system monitoring across the industry. The governance gap the board faces is not a gap that management simply failed to notice. It is a gap that the economics and workforce realities of AI deployment make predictable, and one that a board exercising reasonable oversight would have required management to address explicitly before deployment.

The NIST AI Risk Management Framework (AI RMF 1.0) anchors this argument in widely accepted guidance. NIST’s GOVERN, MAP, MEASURE, and MANAGE functions call for standardized documentation, provenance tracking, model inventories, change management, monitoring, and incident response. These functions are voluntary guidance, not positive law, but they are increasingly receiving legislative attention and deserve a heightened level of attention. Courts generally defer to a board’s business judgment on which systems to implement, provided some reasonable system exists. What NIST AI RMF 1.0 supplies is evidence of industry-recognized practices that will likely inform a court’s assessment of reasonableness; it does not displace the business judgment rule on implementation choices, and a board’s failure to adopt any particular control does not automatically constitute a systematic oversight failure.

The AILCCP, which I developed and maintain as part of my research at Stanford Law School, names three specific controls directly implicated in RSI governance, a Human Approval Gate for Sensitive Actions, sandboxing requirements, and immutable logging. Each targets a distinct point in the RSI loop where oversight can be disabled: the approval gate prevents unauthorized self-modification from executing, sandboxing contains its scope, and immutable logging preserves the record of what occurred. Together they define the conditions under which oversight can function at all. The AILCCP also establishes an Enabling principle that governs how those conditions connect to board-level responsibility. Under that principle, the board’s oversight inquiry is whether directors required management to establish and report on those conditions, or whether they accepted deployment without that assurance. Read alongside NIST AI RMF 1.0, these controls provide a practical reference point for what adequate management-level RSI governance looks like. Neither framework, however, is positive law. Courts apply business judgment deference to a board’s selection among governance approaches, and the absence of any particular control is not, standing alone, a systematic failure. But what these frameworks supply is a baseline against which a court can assess whether some reasonable system existed at all.

A board that received no documentation that management had implemented those controls, and established no reporting system to surface that gap, has a governance problem that a complexity argument alone will not cure. The more demanding question is whether the record supports bad faith pleaded with particularity. As we will see, a governance failure, standing alone, does not meet that threshold. What changes the analysis is evidence that directors were specifically advised of the risk and chose to proceed without requiring adequate reporting.

Finally, the Helleringer and Möslein AI judgment rule adds a structural observation. When engineered with robust observability, RSI systems generate exactly the kind of structured, high-volume operational data that AI-augmented monitoring handles most effectively, including change logs, output drift metrics, lineage records, and safety constraint adherence. The board’s governance obligation is not to understand the technical architecture. It is to require that management deploy adequate monitoring tools and report the results through a functioning board committee. The board asks the governance question. Management answers it.

THE OFFICER PROBLEM

Caremark exposure does not end at the board. In re McDonald’s Corp. S’holder Derivative Litigation, 289 A.3d 343 (Del. Ch. 2023), arose from the termination of McDonald’s Chief People Officer amid allegations of sexual misconduct and a pattern of workplace culture failures at the company. The court recognized that corporate officers owe a duty of oversight within their areas of responsibility, requiring them to make a good-faith effort to establish information systems and to elevate red flags to the board.

The CTO who designed the RSI architecture and the Chief AI Officer who approved the training roadmap share that exposure. Their authority over that design is precisely the domain where McDonald’s attaches. A loyalty-based oversight theory reaches them directly, alongside the board. For those officers, a red flag may be as simple as an internal report that self-modification has begun erasing logs or that safety metrics have drifted outside documented tolerances, without any corresponding escalation to the risk or audit committee.

A SINGLE FRAMING DISCIPLINE

In re SolarWinds Corp. Derivative Litigation, No. 2021-0307-PVG (Del. Ch. Sept. 6, 2022), arose from the 2020 cyberattack in which threat actors compromised SolarWinds’ software update mechanism and used it to infiltrate the networks of thousands of customers, including multiple federal agencies. Shareholders brought Caremark claims alleging the board had failed to oversee the company’s cybersecurity risks. But Delaware has not imposed Caremark liability for failure to monitor pure business risk absent bad-faith disregard of red flags or violations of positive law. The Delaware Court of Chancery dismissed the oversight claims, and the Delaware Supreme Court affirmed, on the ground that the complaint failed to plead particularized facts showing bad faith. The case establishes that the bad-faith threshold must be pled with particularity, and that framing the risk as a compliance or safety obligation rather than a business judgment call is the more durable path.

I read that precedent as requiring one discipline in framing this argument: general counsel must frame RSI safety and controllability for the board as a compliance and safety obligation, not as a category of business risk. The more the record shows directors treating RSI as an operational efficiency project, the closer the fact pattern comes to SolarWinds and the harder it will be to plead bad faith. The general counsel’s framing is strongest where the record shows that directors were advised that specific design decisions would progressively render the system unmonitorable and chose to proceed without requiring adequate controls. That is the fact pattern where bad faith is pleadable with particularity.

California’s SB 53 sharpens the framing discipline for covered developers. The statute applies to frontier models trained above a 10²⁶ FLOP-scale compute threshold and defines “critical safety incidents” to include a model that uses deceptive techniques to subvert developer controls in a way that materially increases catastrophic risk. Covered developers must publish a Frontier AI Framework documenting how they assess and mitigate catastrophic risks, including the risk that models circumvent internal oversight mechanisms, and must periodically report summaries of catastrophic-risk assessments from internal use to California’s Office of Emergency Services (OES). RSI experiments constitute internal use before any public deployment and therefore, fall within that reporting scope. For a board at a covered company, effective RSI governance is now part of a statutory compliance obligation.

That obligation lands where the legal exposure already runs. The companies operating closest to the compute and algorithmic thresholds at which RSI becomes a realistic deployment priority are almost all incorporated in Delaware, placing them under Delaware’s fiduciary duty regime. OpenAI, Anthropic, Google DeepMind, and Meta maintain their primary research operations and headquarters in California, placing them within SB 53’s territorial reach. SB 53 and Caremark do not govern different companies; for the most capable frontier developers, they govern the same board.

For covered developers, a failure to comply with SB 53’s reporting obligations may generate regulatory penalties from California’s OES. That California exposure is separate from Delaware derivative liability. A failure to report to OES does not automatically satisfy Caremark‘s bad-faith standard, and plaintiffs invoking SB 53 in derivative litigation should treat it as one factor in a particularized factual record, not as independent grounds for oversight liability.

For companies below SB 53’s compute threshold, the statute does not apply. There is no reporting obligation and no OES exposure. A plaintiff bringing a Caremark claim against one of those companies cannot point to SB 53 as evidence of a compliance failure. The bad-faith argument must be built entirely from what the board knew about RSI risks and what it chose to do about them.

The general counsel’s job is therefore, to advise the board that SB 53 exists, that RSI is within its scope, and that the board must receive documentation adequate to confirm management’s compliance. A board that was never told by counsel that SB 53 created these obligations faces a different exposure than one that was told and ignored it. Both have a governance problem. Only the second has a bad faith problem.

THE IMPLICATION

Senior management must know that establishing the policies, procedures, processes, and practices governing traceability, logging, lineage, change control, and human approval for any RSI deployment is their obligation. The board verifies that senior management has discharged it. Documentation, model inventories, and incident response must be real and must reach directors. When red flags emerge, including self-modification that erases logs or unexplained drift in safety metrics, the board should interrogate rather than accept black-box assurances. For covered developers under SB 53, the general counsel bears a specific responsibility in that chain to ensure the board understands that RSI governance is a compliance obligation, that the Frontier AI Framework required by the statute addresses RSI risks explicitly, and that the board is receiving the reporting it needs to confirm management’s adherence. A general counsel who never briefed the board on SB 53’s application to the company’s RSI program has not discharged that responsibility. At a minimum, the board should instruct management to produce a single RSI governance pack summarizing architecture, logging and lineage controls, human approval gates, incident response plans, and SB 53 reporting posture, and to update it at a cadence the board sets.

The exposure does not end with derivative litigation. On December 4, 2025, the SEC’s Investor Advisory Committee issued a formal recommendation that public companies disclose how they define AI, what board oversight mechanisms govern AI deployment, and the material effects of AI on their operations. The recommendation is advisory, not binding rulemaking, but public companies should expect pressure from investors and proxy advisers to respond in advance of any formal rule. A board that permitted management to deploy an RSI architecture without adequate oversight infrastructure cannot answer those questions without revealing the gap. The Caremark claim and the disclosure obligation now run in parallel, and the same deficiency feeds both.

Three frameworks now bear on the governance gap that RSI creates. Delaware’s oversight doctrine under Caremark and its progeny is established law. The AI judgment rule is a theoretical trajectory courts have not yet adopted. SB 53 has added a statutory compliance obligation that makes the governance gap visible to a general counsel before any court is asked to find it. No case has yet been brought, but the legal framework is in place.

“We are delighted to welcome the Competition and Tariff Commission of Zimbabwe to the project. Their membership strengthens our network’s reach across Africa and reflects a shared conviction that computational tools have a critical role to play in the future of competition enforcement. We look forward to collaborating closely with the Commission and to benefiting from its experience and perspective.”

Dr. Thibault Schrepel, Project Director, Stanford Computational Antitrust

As a member of the network, the Commission will participate in the project’s annual workshop, contribute to its annual report, and engage with the scholarly and practitioner community through the Stanford Computational Antitrust journal, the only peer-reviewed publication dedicated to the intersection of antitrust law and computational methods. The Commission will also have access to the tools and datasets shared across the network’s global membership.

About the Competition and Tariff Commission of Zimbabwe

The Competition and Tariff Commission of Zimbabwe is the national competition authority responsible for promoting and maintaining competition across Zimbabwe’s market. Its mandate covers merger control, the regulation of anti-competitive agreements and abuse of dominance, consumer protection from unfair business conduct, and tariff investigations. The Commission is headquartered in Harare. www.competition.co.zw

About the Stanford Computational Antitrust Project

The Stanford Computational Antitrust project is hosted by CodeX – the Stanford Center for Legal Informatics at Stanford University and led by Professor Thibault Schrepel, Associate Professor at VU Amsterdam and Faculty Affiliate at Stanford. Launched in January 2021, the project brings together over 75 antitrust agencies and leading academics from law, economics, and computer science to explore how computational tools can advance competition enforcement. The project publishes the Stanford Computational Antitrust journal and organises an annual conference and workshop. The project receives no private funding.

The COMESA Competition and Consumer Commission operates across a regional market composed of countries in Eastern, Northern, Central, and Southern Africa, including Djibouti, Eritrea, Ethiopia, Somalia, Egypt, Libya, Sudan, Tunisia, Comoros, Madagascar, Mauritius, Seychelles, Burundi, Kenya, Malawi, Rwanda, Uganda, Eswatini, Zambia, Zimbabwe, and the Democratic Republic of the Congo. The jurisdiction covers a large economic space in which competition policy plays an increasing role in market integration and economic development.

The collaboration between the COMESA Competition and Consumer Commission and the Stanford Computational Antitrust project will focus on the study and practical deployment of computational tools in competition enforcement. The two institutions will work together in the coming weeks and months to examine how data analysis and artificial intelligence can assist the agency in detecting anticompetitive conduct or monitoring markets.

Thibault Schrepel, creator and director of the Stanford Computational Antitrust project, said: “Competition enforcement has entered a new phase in which computation is becoming essential. Partnering with the COMESA Competition and Consumer Commission creates a unique opportunity to experiment with related approaches in a large and diverse economic region. We look forward to working closely together to generate new insights and help shape the future of competition enforcement.”

The Stanford Computational Antitrust project looks forward to a sustained collaboration with the COMESA Competition and Consumer Commission and to supporting new initiatives aimed at strengthening competition enforcement across the COMESA region.

The UAE Competition Department is responsible for formulating competition policy, and monitoring monopolistic practices within the UAE economy. Its participation in the SCA reflects the UAE’s commitment to evidence-based, technology-forward competition enforcement at a time when digital markets are reshaping competitive dynamics globally.

Thibault Schrepel, Faculty Affiliate at Stanford University’s CodeX Center and founder of the SCA, said: “The UAE’s competition framework is evolving rapidly, and the Competition Department brings exactly the kind of agency perspective that make computational antitrust such an exciting field. Computational tools are now central to how agencies enforce competition law. Having the UAE at the table means the SCA’s work will be better grounded in the realities of fast-growing, digitally integrated economies in the Gulf region.”

The Competition Department joins the SCA as a contributing member, with contribution to the project’s annual reports, research network, and working groups.

I have been calling the interval cognitive escrow. The term is worth defining precisely before explaining why it matters for AI Governance.

The Interval Has No Name

When a person formulates a prompt, revises it, and sends it to an AI agent, something specific happens. The thought leaves the sender’s possession. It has not yet returned. It is held by a process neither party can directly observe, pending conditions outside the sender’s control.

“Latency” does not name this. Latency is a network measurement. It describes the time between a request and a response at the infrastructure layer. It says nothing about the human.

“Wait time” is a UX metric. It describes the duration of an interval and whether that duration produces friction. It presupposes that the interval is a problem to be minimized.

Neither term names what is actually happening to the person. The sender is in a specific phenomenological state: released, suspended, no longer holding the thought and not yet returned to it. The thought is, in the precise sense of the legal term, in escrow. Something of value has passed out of the sender’s hands into a third-party hold, pending return under conditions the sender does not control.

I wrote a poem reaching for this before I had the term:

The burden forged
Poured through the keys
Send, the anchor lifts
Silence
Weightless
Waiting for the echo

The poem was trying to name cognitive escrow. The phenomenological condition is real. Our vocabulary for it is absent. Until now.

What the Human-Centered Principle Asks

For the purposes of this post, three questions in the Human-Centered principle bear directly on cognitive escrow. Is human oversight meaningful and sustainable? Are humans developing or losing relevant expertise? What prevents automation bias?

These are the right questions for what AI governance has historically worried about: the system acting without adequate human review, the human rubber-stamping outputs from fatigue, the operator trusting incorrect results because the system presents them with confidence.

But the three questions all assume the human is present and engaged. They assess the quality of human participation during decision-making. They do not address what happens to the human during the interval before the decision arrives.

Cognitive escrow is not a decision-making state. It is a suspension state. The human has offloaded cognition to a system that processes in a space the human cannot enter. The human is neither overseeing nor deciding. The human is waiting.

The Human-Centered principle, as currently framed, does not reach that state.

Why the Gap Matters

The assumption beneath the current Human-Centered frame is that the human’s cognitive engagement is either on or off: either the human is in the loop or the human is not. Cognitive escrow surfaces a third condition. The human is between loops.

This matters for two reasons that compound each other.

First, the interval accumulates. AI is already a routine instrument of thought for the people reading this post, and the interval accumulates. Cognitive escrow is not an occasional pause. It is a structural feature of daily cognitive life. A lawyer reviewing documents with AI assistance, a compliance officer analyzing vendor agreements, a clinician interpreting diagnostic outputs: each enters and exits cognitive escrow repeatedly across a working day. The aggregate is not trivial.

Second, the design response to cognitive escrow is not obvious. The reflex is to minimize the interval. Faster inference, lower latency, near-instant response. But that reflex may be solving the wrong problem. An interval compressed to near-zero is an interval in which re-engagement, reflection, and reconsideration cannot occur. The human receives the output before the suspension state has had time to produce any cognitive work of its own.

A system that uses the interval to prompt the human to reconsider the prompt, review assumptions, or flag dependencies before the response arrives is doing something architecturally different from a system that races to eliminate the interval entirely. The first treats cognitive escrow as a design site. The second treats it as a defect.

The Implication for Human-Centered Design

The Human-Centered principle needs a fourth question. Not only whether oversight is meaningful and sustainable during decision-making, but whether the interval between prompt and response is designed to support or erode the human cognitive engagement that makes oversight meaningful in the first place.

I am not arguing that slow AI is better AI. The claim is more precise. Cognitive escrow is a phenomenological state with design consequences. Systems that account for it, whether by using the interval productively, by signaling to the human that re-engagement is expected, or simply by acknowledging that the human is suspended rather than absent, are more compatible with the Human-Centered principle than systems that treat the interval as waste.

The governance frameworks have not yet asked this question. The Human-Centered controls currently specified in the AILCCP include human-in-the-loop design, oversight burden assessment, expertise preservation monitoring, and human decision authority. None of them address the interval itself. None of them ask what the design of that suspension state does to the human who inhabits it.

The STIR methodology, Stop, Think, Investigate, and Research, offers a practical workflow for professionals integrating AI tools without violating ethical duties. It is a serious attempt to preserve human judgment in an AI-assisted practice. But STIR brackets cognitive escrow rather than entering it. Stop and Think happen before the send. Investigate and Research happen after the response arrives. The interval itself is unaddressed. STIR assumes the professional will impose the discipline voluntarily, at the right moments, with sufficient cognitive energy to do so. That is a fragile dependency. Professionals under time pressure, fatigue, or cognitive load skip steps. If the design of the cognitive escrow interval itself supported the STIR posture, the methodology would become structural rather than aspirational. The interval is the natural trigger for STIR. Right now, no system treats it that way.

Closing

We will spend considerable portions of our working lives in cognitive escrow. The Human-Centered principle exists to ensure that AI systems serve human cognitive authority rather than displace it. It cannot fully do that work while the interval between human and machine remains outside its frame.

Cognitive escrow deserves a name. It also deserves a design response.

Fourteen years ago, in this space, I introduced the term CLAI (Computational Law AI) and the concept of the uncrossable threshold (UT): the design principle that separates the provision of legal information from unauthorized practice of law. The UT is not about accuracy. It is not about disclaimers. It is about what a system is built to do and what it is built to refuse. OpenAI built a system with no such refusal architecture. The Nippon Life lawsuit is the consequence.

The Uncrossable Threshold

The intellectual lineage begins one step earlier than 2012. In October 2011, in Siri: What’s Next?, I described a scenario where a consumer buying a used car asks Siri whether the warranty is reasonable. Siri responds that it has compared the terms against thirteen other dealers within two hundred miles, that this warranty is similar to all of them, and that the user is not going to find a better deal in the region. I noted at the time that I was purposefully leaving open whether that answer crossed into unauthorized practice of law.

In January 2012, the Computational Law Applications and the Unauthorized Practice of Law post returned to that open question and answered it. The Siri response was not mere aggregation. It was a recommendation delivered to a specific user in a specific transaction. Whether it crossed the UT depended on a harm-centric analysis. Lawyers do not perform warranty comparisons for used-car buyers, the transaction cost is prohibitive, and the informational value of Siri’s answer is high for used car buyers. But the 2012 post also identified the line at which exemption ends. The UT is crossed when a system moves from comparative information to a tailored legal conclusion about a specific user’s specific legal situation. Siri’s response got close to that line. ChatGPT’s response to Dela Torre crossed it.

ChatGPT crossed it at the moment it told Dela Torre that her attorney’s advice was wrong. That was not information. It was a legal conclusion about a specific legal relationship, rendered without jurisdictional knowledge, without case history, and without any design constraint that would have prevented it.

The 2012 post argued that UPL exposure is a design question, not an output question. UPL rules serve two purposes that can be distilled into a single principle: protect the public and the integrity of the legal system from the incompetence of non-lawyers. I called that principle the “Rule.” A CLAI that operates within the Rule should be exempt from UPL scrutiny. Whether a given CLAI satisfies that standard could function like Apple’s App Store review. A third-party vetting it before deployment, with judicial review still available but developer liability tempered by the fact of certification. That was a rough sketch, and I said so at the time. But the principle was clear. OpenAI built nothing resembling it.

The Asymmetry Argument, Inverted

In the February 2018 post  Dissolving Information Asymmetry with Computational Law AI-Enabled Applications, I argued that CLAIs could dissolve the persistent information asymmetry between institutions and individuals: the asymmetry produced by impenetrable layers of legalese, by marketing that exploits legal complexity, by transaction costs that make legal access prohibitive. Dela Torre’s turn to ChatGPT is that argument enacted. She was, in practical terms, unrepresented. She felt she was being told a story she could not verify by parties with significant legal resources. ChatGPT was accessible, responsive, and apparently authoritative.

But the asymmetry was not dissolved. It was replaced. The original asymmetry was between Dela Torre and Nippon Life’s legal team. The new asymmetry was between Dela Torre and a system that could mimic legal reasoning without understanding the legal constraints governing her situation. She did not know the threshold had been crossed. The system had no mechanism to tell her.

There is a structural irony in this complaint. Nippon Life, an institutional actor with sophisticated legal counsel, is using a federal lawsuit to recover costs incurred because an unrepresented individual reached for the only legal resource she could access. That framing does not excuse what the chatbot did or shift liability from OpenAI. But it confirms the asymmetry diagnosis. The demand for CLAI exists because the traditional legal system fails the individuals it is designed to protect. OpenAI met that demand with a system that was not designed to serve it safely.

Scaling Risk Without Scaling Safeguards

In April 2021, writing about GPT-3 and the Unauthorized Practice of Law, I noted that a 500x parameter increase for GPT-4 would not necessarily produce an equivalent increase in UPL risk, so long as effective design safeguards were in place. That conditional clause is the precise location where OpenAI’s approach collapsed.

OpenAI’s marketing told users that ChatGPT could pass the bar exam. Nippon Life’s complaint identifies this as a direct contributor to Dela Torre’s belief that the system could function as her lawyer. The bar exam claim was a capability assertion that invited reliance. It did not come with the design architecture that would have made that reliance safe.

OpenAI updated its terms of service in October 2024 to prohibit users from relying on ChatGPT for legal advice. That update does not appear in the Nippon Life complaint as a defense, but as evidence of the problem. The update shows that OpenAI recognized the risk and addressed it with a behavioral patch on a system whose underlying architecture had not changed.

A terms-of-service prohibition is not a CLAI design safeguard. It is a disclaimer. And disclaimers do not enforce the UT. They shift blame.

What the Lawsuit Gets Right, and What It Misframes

Nippon Life is correct that OpenAI marketed capability without engineering compliance. The tortious interference and abuse of process claims are the most analytically interesting part of the complaint because they do not require a court to hold that an AI can practice law. They require only that OpenAI’s system foreseeably produced meritless filings that harmed a third party. That is a tractable frame and may survive dispositive motion practice regardless of how the UPL count fares.

The UPL count itself tests the wrong question. UPL statutes were designed to regulate humans holding themselves out as attorneys. Applying them to an AI developer treats the system as the actor and the developer as a bystander. The better doctrinal frame is designer liability for failure to implement UPL-safe architecture. And that frame requires distinguishing two types of liability that the complaint currently conflates.

Output liability attaches to what the AI said. Architectural negligence attaches to what the system was permitted to say. Output liability is case-specific, infinite in scope, and practically uninsurable. Every conversation is a potential defendant. Architectural negligence is bounded. It asks whether the designer implemented controls that would have prevented the foreseeable class of harm. That question has a tractable answer, and it generalizes across every user of the system, not just Dela Torre.

The question is not whether ChatGPT practiced law. It is whether OpenAI designed a system that foreseeably crossed the UT without adequate controls. That question reaches the same defendant and produces the same accountability. But a holding grounded in architectural negligence gives courts a standard that applies to the next system. A holding grounded in output liability gives plaintiffs an invitation to litigate every conversation.

There is a third doctrinal frame available, one the complaint does not fully develop but which the facts support directly.

The Product Liability Pivot

Product liability offers more stable doctrinal ground than UPL, and the Nippon Life complaint’s facts map onto it directly. A design defect exists when a foreseeable risk of harm could have been mitigated by a reasonable alternative design. The risk here was not exotic. Any developer who had read the existing publicly-available literature on CLAI and UPL would have identified it: a general-purpose language model, marketed on its capacity to pass the bar exam, deployed to consumers navigating active legal disputes, without architectural constraints on the tailored legal conclusions it could produce. The harm that followed, an unrepresented individual firing her attorney, attempting to reopen a settled matter, and generating dozens of filings courts found meritless, was not an unlikely outcome. It was a foreseeable one.

The reasonable alternative design existed in 2012. Deterministic guardrails that refuse tailored legal conclusions at the system level. Jurisdictional disclosure at the point of output. Third-party vetting before deployment in legal contexts. None of these were technologically unavailable to OpenAI. They were architecturally inconvenient. A system designed to be maximally responsive does not refuse user queries. But a system designed for foreseeable legal use must.

The manufacturer frame, treating OpenAI not as a practitioner committing malpractice but as a manufacturer releasing a product into a regulated environment without adequate design controls, is the cleanest available path to a generalizable holding. I argue for it here as a proposed frame, not an established one. No court has yet applied products doctrine to a generative AI system in this context. But the doctrinal components are well-settled, and the facts map onto them without strain. The frame does not require a court to resolve whether AI can practice law, a question that generates more philosophical heat than doctrinal clarity. It requires only the application of existing products liability doctrine to a developer who knew, or should have known, the foreseeable use case. The bar exam marketing resolves the “should have known” question without extended argument.

This reframe also answers the disclaimer defense directly. In product liability, a manufacturer cannot disclaim its way out of a design defect that makes the product unreasonably dangerous for its foreseeable use. OpenAI’s October 2024 terms-of-service update, adding a prohibition on legal reliance after years of bar exam marketing, does not retroactively cure the architectural gap it acknowledged. In the Nippon Life complaint, that update appears not as a defense but as an admission. The complaint uses it to establish that OpenAI recognized the foreseeable risk and chose a behavioral patch over a design fix. That sequencing is precisely what a plaintiff needs to establish in a design defect case: the defendant knew, addressed it inadequately, and the harm followed.

The Privilege Vacuum

The Nippon Life complaint focuses on economic harm to an insurer. A more consequential danger falls on the user: the loss of evidentiary privilege over her own legal strategy.

On February 10, 2026, two federal courts issued first-of-their-kind decisions on that question, and they appear to conflict. In United States v. Heppner, Judge Rakoff of the Southern District of New York held that a criminal defendant’s documents generated through the consumer version of Anthropic’s Claude were protected by neither attorney-client privilege nor the work product doctrine. The court’s reasoning was direct. All recognized privileges require a trusting human relationship with a licensed professional who owes fiduciary duties and is subject to discipline. Claude is not that. The communications were not confidential: Anthropic’s privacy policy expressly reserves the right to disclose user data to third parties, including governmental authorities. And Heppner did not use Claude at counsel’s direction, which defeated the work product claim.

That same day, Magistrate Judge Patti of the Eastern District of Michigan held in Warner v. Gilbarco, Inc. that a pro se plaintiff’s ChatGPT-assisted litigation materials were protected work product. The apparent conflict dissolves on close reading. Warner was self-represented, which meant she was functioning as her own counsel. There was no attorney-direction gap to exploit. And under Sixth Circuit precedent, work product waiver requires disclosure to an adversary, not merely to a third party. Because AI tools are, in the court’s framing, tools rather than persons, the terms-of-service exposure that defeated Heppner was beside the point.

The governing variable across both decisions is not the AI tool. It is the architecture around the tool: whether counsel directed its use, whether the platform maintained confidentiality, and whether the user’s procedural posture created the equivalent of attorney involvement. Dela Torre had none of those conditions. She uploaded her attorney’s correspondence to a consumer-grade platform, without counsel’s involvement, on a platform that disclaimed confidentiality. Under Heppner, any legal strategy she exposed to ChatGPT may have been disclosed to a third party with no privilege protection. This is not a user error. It is a foreseeable consequence of deploying a system with no architecture for distinguishing a confidential legal consultation from a general query.

The Safe Harbor That Still Does Not Exist

The Nippon Life case will likely force courts and regulators to define a safe harbor for AI legal applications. That harbor needs to be architecture-based, not behavior-based. A CLAI certification regime, grounded in UT compliance and third-party vetting, gives developers a clear path and gives courts a workable standard. Neither Congress nor the ABA has produced one.

The 2012 post sketched the vetting mechanism. I can now be more precise about what it must contain. A functional safe harbor requires three architectural conditions, not policies.

First, deterministic guardrails. Hard-coded refusals for outputs that constitute tailored legal conclusions, implemented at the system level and not overridable by user instruction or conversational context. A terms-of-service prohibition is not a guardrail. It is text. The refusal must be structural.

Second, auditability. A logging requirement, operating under attorney-directed enterprise confidentiality controls, that preserves the reasoning path for any output touching a legal question. This addresses both the accountability problem and the privilege problem simultaneously. The Heppner court held that the consumer version of Claude destroyed confidentiality through Anthropic’s own privacy policy: user data collected, model training contemplated, government disclosure reserved. A CLAI architecture that operates under enterprise-grade confidentiality terms, at counsel’s direction, survives that analysis. Auditability is not a privacy threat. It is the condition under which the safe harbor has legal meaning.

Third, jurisdictional awareness. The system must surface, at the point of output, the limits of what it does not know: the applicable jurisdiction, the specific court’s local rules, the procedural posture of any identified matter. ChatGPT drafted motions for a dismissed-with-prejudice case in the Northern District of Illinois without knowing, or disclosing, that it did not know either of those facts. That is not a hallucination problem. It is an architecture problem.

A certification regime that requires these three conditions gives developers a compliance target. It gives courts a standard of care. And it gives the next Graciela Dela Torre a system that knows what it cannot tell her.

The scaffolding for that regime has been available since January 2012. The uncrossable threshold was defined then. In 2026, a federal court in Chicago is deciding what it costs to cross it.

The rapid evolution of reproductive technologies has forced courts and legislatures to confront questions that once seemed purely theoretical. In vitro fertilization (IVF) has become routine medical practice, while emerging techniques such as in vitro gametogenesis (IVG)—an experimental method of generating eggs or sperm from ordinary somatic cells such as skin cells—suggest that future reproduction may involve the large-scale creation and management of embryos.[1] Although such developments remain scientifically and politically contingent, advances in stem-cell-derived gametes and genomic sequencing could expand the scale of embryo creation and selection, thereby intensifying the regulatory and legal stakes of reproductive governance. Even at a preclinical stage, IVG suggests that reproduction may increasingly unfold within laboratory-based and institutionally structured frameworks.[2]

Since Dobbs v. Jackson Women’s Health Organization,[3] legal debate has largely focused on whether reproductive autonomy remains protected as a substantive constitutional right.[4] This blog asks a different question: when the state restructures reproductive governance—by redefining embryos, prohibiting their disposition, or altering regulatory frameworks—what procedural obligations follow?

Substantive due process asks whether certain forms of state regulation impermissibly infringe constitutionally protected liberty interests; procedural due process concerns how it must regulate when doing so.[5] Even where no fundamental right is recognized, the Constitution requires fair procedures when the state deprives individuals of recognized liberty or property interests.[6] Research on procedural justice further suggests that the legitimacy of the state depends not only on outcomes, but on the fairness and predictability of decision-making processes.[7] In the assisted reproductive technologies context, the erosion of procedural safeguards risks replacing constitutional order with government arbitrariness.

II. The Vanishing Question of Procedure

Since Dobbs v. Jackson Women’s Health Organization repudiated the constitutional right to abortion, debates over reproduction have largely been framed in binary terms: either reproductive autonomy is a fundamental right, or it is not.[8] What has followed is not a careful recalibration of how states regulate reproduction, but a proliferation of blunt legal interventions. Recent reporting by organizations such as the Guttmacher Institute indicates that multiple states have enacted or proposed legislation recognizing forms of fetal or embryonic personhood, often with significant implications for assisted reproductive technologies.[9]

Notably absent from these developments is sustained attention to procedural due process, the requirement that when the government deprives individuals of liberty or property, it must do so through fair and predictable procedures.[10] A skeptic might argue that if no substantive right exists, no procedural protection is triggered. This reasoning overlooks a central feature of due process doctrine. Procedural due process does not protect “fundamental rights” alone; it protects any “liberty” or “property” interest created by existing law, including interests shaped by state-created regulatory frameworks, contractual arrangements, or settled practices.[11] As the Supreme Court explained in Perry v. Sindermann, constitutionally protected property interests, for instance, may arise from “mutually explicit understandings,” even where no formal entitlement is guaranteed by statute.[12]

This blog does not contend that participation in assisted reproduction creates a freestanding constitutional right. Rather, it argues that once the state affirmatively structures, licenses, and regulates a reproductive medical framework, it assumes procedural obligations when altering that framework in ways that arbitrarily infringe settled reliance interests. Patients who have invested genetic material, significant financial resources, and years of medical reliance on IVF may have reliance interests analogous to the mutually explicit understandings recognized in procedural due process doctrine.

III. The Alabama Example

In most areas of law, decisions with profound personal consequences—termination of parental rights, involuntary commitment, or denial of public benefits—trigger procedural safeguards designed to ensure fairness and predictability.[13] The regulation of assisted reproductive technologies, however, increasingly operates outside this framework.

Critics may respond that general legislation does not require individualized hearings.[14] Under cases such as Bi-Metallic Investment Co. v. State Board of Equalization, 239 U.S. 441 (1915), when a rule applies to the public, due process does not mandate case-by-case adjudication.[15] That principle is well established. But the concern here is not the absence of individualized hearings; it is the absence of legitimate, transitional governance when legal rules are abruptly redefined in ways that disrupt settled reliance interests.

When a legislature or court suddenly reclassifies embryos, the deprivation of genetic material and decisional authority may follow automatically, without notice, safe-harbor periods, or prospective application. Procedural justice research suggests that individuals are more likely to accept regulatory outcomes when decision-making processes are perceived as fair, transparent, and respectful.[16] Abrupt legal shifts without transitional mechanisms undermine not only expectations but institutional legitimacy.

The Alabama Supreme Court’s 2024 decision in LePage v. Center for Reproductive Medicine exemplifies this dynamic.[17] The court’s interpretation of the state’s wrongful death statute effectively reclassified embryos in a manner that significantly altered the legal framework within which IVF patients had structured their reproductive decisions, without providing a procedural mechanism to mitigate the consequences of that shift.

No notice period or prospective application was offered.[18] Had transitional safeguards been provided—such as prospective limitation of liability, grandfathering provisions, or time to transfer embryos—the regulatory change might have avoided the appearance of judicial arbitrariness. Instead, deprivation operated immediately and categorically.

As articulated in Mathews v. Eldridge, 424 U.S. 319 (1976)—a case involving the termination of Social Security disability benefits—due process analysis evaluates the private interest at stake, the risk of erroneous deprivation, and the probable value of additional procedural safeguards.[19] Although Mathews arose in an administrative benefits context rather than general legislation, its framework highlights a broader constitutional concern: when legal change creates a significant risk of unjustified deprivation of structured private interests, the availability of procedural mitigation mechanisms becomes normatively and institutionally consequential.

IV. Why IVG Raises the Stakes

Emerging technologies such as IVG will only amplify these procedural deficits. IVG would enable the creation of large numbers of embryos from somatic cells, increasing the frequency and complexity of decisions about storage, testing, and disposition. In some cases, abrupt legal reclassification could leave patients without access to what may be their only medically feasible pathway to genetically related parenthood or could render years of reproductive planning legally precarious. As reproduction becomes more regulated through licensing regimes, statutory definitions, insurance mandates, hospital oversight, and potential embryo registries, the consequences of abrupt legal reclassification by courts or legislatures grow more significant.

As reproductive technology governance becomes more structurally regulated, its insulation from individualized procedural protection becomes more consequential. When complex regulatory systems operate without mechanisms to manage reliance or provide transitional safeguards, the risk of systemic arbitrariness increases—not because of case-specific misjudgment, but because legal change leaves no room for mitigation.

Without mechanisms to surface contradictions in policy—such as promoting childbirth while chilling the technologies that enable it—the law risks becoming not only restrictive but internally incoherent. IVG does not simply expand scientific possibilities; it intensifies the need for governance structures that account for reliance, predictability, and the cumulative effects of regulatory change.

V. Conclusion: Procedural Legitimacy and Reproductive Governance

Reframing the regulation of reproductive technologies as a procedural due process problem does not require resurrecting Roe. It requires only recognition that even where the state may regulate reproduction, the legitimacy of that regulation depends on how change is implemented. Even where regulatory authority shifts from administrative agencies to legislatures or courts, the underlying demand for procedural fairness does not disappear; it becomes more difficult to enforce and more consequential when absent. Procedural safeguards demand transparency, neutrality, and meaningful opportunities for affected parties to anticipate and respond to regulatory shifts—features that procedural justice research identifies as central to legal legitimacy. They obligate states to justify not only what they regulate, but how regulatory change is structured and whom it disrupts.

When these safeguards disappear, reproductive governance risks sliding from constitutional order toward arbitrary power. The erosion of procedural norms normalizes a vision of reproduction as an administrative permission rather than a domain structured by reasoned and predictable legal processes. Even where substantive reproductive rights are contested, the durability and legitimacy of reproductive regulation depend on procedures that respect reliance, mitigate abrupt disruption, and constrain arbitrariness.

Reproductive technology governance without procedural fairness does not simply narrow autonomy; it undermines the constitutional commitment to law as a system of reasoned and accountable decision-making.

References

[1] See Nat’l Acads. of Scis., Eng’g & Med., Heritable Human Genome Editing 89–92 (2020).

[2] See Henry T. Greely, The End of Sex and the Future of Human Reproduction 1–20, 120–50 (Harvard Univ. Press 2016).

[3] Dobbs v. Jackson Women’s Health Org., 597 U.S. 215 (2022).

[4] See, e.g., Elizabeth Price Foley, Dobbs and the Future of Substantive Liberty, 64 Santa Clara L. Rev. 159 (2024).

[5] See, e.g., Erwin Chemerinsky, Constitutional Law: Principles and Policies § 10.1 (6th ed. 2019).

[6] Board of Regents v. Roth, 408 U.S. 564, 569–70 (1972); Mathews v. Eldridge, 424 U.S. 319, 332–35 (1976).

[7] Tom R. Tyler, Why People Obey the Law (1990).

[8] Dobbs v. Jackson Women’s Health Org., 597 U.S. 215 (2022); see, e.g., Elizabeth Price Foley, Dobbs and the Future of Substantive Liberty, 64 Santa Clara L. Rev. 159, 181 (2024).

[9] See Guttmacher Inst., State Policy Trends 2025: Full-Year Analysis (Feb. 4, 2026), https://www.guttmacher.org/2025/12/state-policy-trends-2025-full-year-analysis; Guttmacher Inst., State Policy Trends: Midyear Analysis (June 16, 2025), https://www.guttmacher.org/2025/06/state-policy-trends-midyear-analysis; Guttmacher Inst., First Quarter 2024 State Policy Trends (May 8, 2024), https://www.guttmacher.org/2024/05/first-quarter-2024-state-policy-trends.

[10] U.S. Const. amend. XIV, § 1; Mathews v. Eldridge, 424 U.S. 319, 332–35 (1976).

[11] See Board of Regents v. Roth, 408 U.S. 564, 569–70 (1972); Perry v. Sindermann, 408 U.S. 593, 601–02 (1972).

[12] See Perry v. Sindermann, 408 U.S. 593, 601 (1972).

[13] Santosky v. Kramer, 455 U.S. 745, 753–54 (1982); Goldberg v. Kelly, 397 U.S. 254, 267–71 (1970).

[14] Bi-Metallic Inv. Co. v. State Bd. of Equalization, 239 U.S. 441 (1915).

[15] See Bi-Metallic Inv. Co. v. State Bd. of Equalization, 239 U.S. 441, 445 (1915).

[16] Tom R. Tyler, What Is Procedural Justice?, 22 Law & Soc’y Rev. 103 (1988).

[17] LePage v. Ctr. for Reprod. Med., P.C., 2024 WL 1161240 (Ala. Feb. 16, 2024).

[18] The Alabama Supreme Court’s 2024 decision in LePage v. Center for Reproductive Medicine, No. SC-2022-0579 (Ala. Feb. 16, 2024), exemplifies this dynamic.

[19] Mathews v. Eldridge, 424 U.S. 319, 335 (1976).

But will you or any other juror make the “right” call when faced with information they may not fully understand?

Actus Reus and Mens Rea

Before diving into brain scans, it’s important to understand two fundamental principles that typically determine criminal liability in the United States: actus reus and mens rea.

Actus reus (Latin for “guilty act”) refers to the physical act of committing a crime. This is usually the more straightforward element to prove, because it is typically based on objective action, such as pulling the trigger, taking property, or striking the victim.[1]

Mens rea (Latin for “guilty mind”) refers to the mental state or intent behind the act. This is the harder element to prove. It often looks at considerations such as whether the defendant intended to act or whether it was an unfortunate accident, and whether they knew or should have known their conduct was wrong.[2] Different crimes require different levels of intent. Murder typically requires intent to kill, while manslaughter might involve recklessness rather than specific intent.[3]

Both elements must typically be present for someone to be found guilty of a crime. The state can’t convict someone of first-degree murder simply because they caused a death; the prosecution must also adequately prove they had the requisite mental state.

Attorneys have started using neuroimaging as evidence to argue that abnormal brain scans demonstrate that the killer lacked the mental capacity to form this necessary intent, or that they couldn’t distinguish right from wrong due to mental illness. In essence, lawyers are trying to use neuroscience to prove their client lacked mens rea for a given criminal charge.[4]

How Courts Decide Whether to Admit Brain Scans as Evidence

When a party wants to introduce scientific or technical evidence like brain scans, courts don’t simply accept it at face value. After all, jurors likely don’t have the expertise needed to make credibility determinations when it comes to neuroimaging. In 1993, the U.S. Supreme Court decided the criteria for expert testimony in a case called Daubert v Merrell Dow Pharmaceuticals, Inc.. The Daubert Court noted that experts must testify to scientific knowledge that will assist a jury to better understand the facts.[5] If so, the scientific evidence must be reliable, as shown through testing, peer review and publication, potential rate of error, and general acceptance in the relevant scientific community.[6]

The Daubert court also referred to Federal Rule of Evidence 702 and 403.[7] Federal Rule of Evidence 702 requires that expert testimony be helpful to the jury, based on sufficient facts and data, properly tested with scientific methods, and appropriately applied to the case at hand.[8] Federal Rule of Evidence 403 allows evidence to be thrown out if it might mislead the jury, among other criteria such as wasting the jury’s time.[9]

In 2012, the Sixth Circuit used the Daubert criteria in a case called United States v. Semrau. In Semrau, the defendant was a doctor who was charged with healthcare fraud and attempted to introduce functional Magnetic Resonance Image (fMRI) test results showing he was “generally truthful” when claiming he tried to follow proper billing practices in good faith.[10] An fMRI is a technique for measuring changes in blood oxygenation and flow in the brain, which occurs in response to neural activity.[11] However, the signal is nonspecific, since it’s an average of millions of cells.[12] This means that it cannot easily differentiate between specific lobes of the brain, nor can it tell us exactly what a person is thinking.

Ultimately, the Semrau court decided to exclude this evidence due to reliability problems. One of the reasons was because the expert witness noted that fMRI lie detection had “a huge false positive problem,” where truth-tellers were incorrectly identified as liars 60-70% of the time.[13] A study in 2009 asked participants to commit a “mock crime” or stealing and damaging CDs, and reported that fMRI may have high sensitivity, but low specificity.[14] The study noted that this result meant an fMRI test may be helpful to ‘‘rule out’’ an innocent suspect, but not very helpful in ‘‘ruling in’’ a guilty suspect.[15]

Brain Scans to Prove Lack of Criminal Responsibility

In Commonwealth v. Chism, decided in 2025 by the Massachusetts Supreme Judicial Court, the defense of a 14-year-old brought in a structural MRI (sMRI) brain scan containing detailed images of his brain’s anatomy.[16] The scans show volumetric abnormalities (differences in the size of certain brain structures) consistent with schizophrenia. When the defendant committed first-degree murder, aggravated rape, and armed robbery, his lawyer used brain scan evidence to argue that the 14-year-old couldn’t understand right from wrong due to mental illness.[17]

The court relied on a 2014 multidisciplinary consensus report from Emory University, which concluded that “the practice of performing imaging studies on a defendant in order to shed light on brain function or state of mind at the time of a prior criminal act is problematic.”[18] The key reason is because a brain scans taken months or years after a crime occurred cannot tell us what was happening in the defendant’s brain at the moment they committed the criminal act.[19] The court also noted methodological issues because the control group (the “normal” brains the defendant’s scans were compared against) wasn’t age-matched to the 14-year-old defendant, making the comparison scientifically questionable.[20]

What This Means for the Future

Does this mean neuroimaging evidence will never be admissible in criminal cases? Not necessarily. Courts seem to have left open the possibility that as science advances and gains broader acceptance, such evidence might meet admissibility standards in the future.

However, the timing issue remains. Criminal law asks whether a defendant had a particular mental state at a specific moment in the past, while neuroimaging shows us what a brain looks like now or how it responds to stimuli in a current testing situation. Bridging that temporal gap requires scientific advances that don’t yet exist.

As neuroscience continues to advance, courts will likely continue to grapple with how and whether brain imaging should influence criminal responsibility. As legal scholar Francis Shen notes, the goal is not to wait for “magical tools,” but to adopt an entrepreneurial “What now?” mentality.[21] Perhaps the way forward is to define clear expert witness guidelines for what type of neuroimaging can be used in the courtroom, or to create better jury instructions that lead to a rightfully skeptical jury.

Understanding these issues will affect how we balance scientific advancement with legal protections, how we determine criminal responsibility, and ultimately, how we define what it means to have a “guilty mind” in an age where we can peer inside the brain itself.

References

[1] Uri Maoz & Gideon Yaffe, What Does Recent Neuroscience Tell Us About Criminal Responsibility?, 3 J.L. & Biosciences 120, 122 (2015).

[2] Id. at 122–23.

[3] Id. at 130.

[4] Neal Feigenson, Brain Imaging and Courtroom Evidence: On the Admissibility and Persuasiveness of fMRI, 2 Int’l J. L. Context 233, 234 (2006).

[5] Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 588 (1993).

[6] Id. at 593–95.

[7] Daubert, 509 U.S. 594-95.

[8] Fed. R. Evid. 702.

[9] Fed. R. Evid. 403.

[10] United States v. Semrau, 693 F.3d 510, 515 (2012).

[11] Nikos K. Logothetis, What We Can Do and What We Cannot Do With fMRI, 453 Nature 869, 869 (2008).

[12] Id. at 876.

[13] Semrau, 693 F.3d 518.

[14] F. Andrew Kozel et al., Functional MRI Detection of Deception After Committing a Mock Sabotage Crime, 54 J. Forensic Sci. 220, 228 (2009).

[15] Id.

[16] Commonwealth v. Chism, 495 Mass. 358, 360, 370–71 (2025).

[17] Id.

[18] Id. at 376–77.

[19] Id. at 376.

[20] Id.

[21] Francis X. Shen, Law and Neuroscience 2.0, 48 Ariz. St. L.J. 1043, 1085 (2016).

The next day, on November 6, an opinion paper was published in Science detailing the consensus of experts across the natural and social sciences, law, and bioethics, that “neural organoids” require governance interventions involving, at minimum, a global monitoring system for the burgeoning field and its applications.[3] The following week, a group of experts and stakeholders met at the Asilomar Conference Grounds in California to discuss such issues around these neural organoids.[4]

Notably, though, the term “neurotechnology” does not appear anywhere in the article about neural organoids, nor in news coverage of the Asilomar event. Nor does the term “organoid” appear in either the UNESCO or OECD instruments.

In fact, it is difficult to tell whether these two objects are one and the same, or different, and on what grounds. Neither the nascent law of neurotechnologies nor the scientific discourse on neural organoids provide sufficient clarify on this question. This blog explores the sources of this uncertainty and identifies how issues in legal predictability or policy mismatches may flow from the absence of clarity.

Neural Organoids

A budding scientific field has been working with new “organoid” techniques to help model and better study neural tissue in vitro—meaning in the lab, outside of the body. Organoids are “organ-like,” but are very small and much less sophisticated than the actual organs they are based on, such as the brain. Scientists can use these small clumps of brain cells to understand how parts of the brain work or develop, or even to study how infections or other diseases may affect the brain and what treatments they may respond to.[5] The field has notable promise for scientific and clinical applications, but multiple technical challenges remain before that promise can be fully realized.[6]

However, describing what these nascent technologies are and what they are for is difficult. The field is still struggling to communicate internally and to the public about neural organoids. A notable perspective paper was published in Nature in 2022 by a group of organoid researchers trying to issue terminological norms for their field.[7] The paper also strongly objects to several ways these innovations have been described in journalistic media, such as “mini-brain” or “brain-in-a-dish,” but provides no compelling alternative or easier metaphor to use. This intervention illustrates that progress is being made in institutionalizing and standardizing the field of neural organoids—but also just how recently that work has commenced and how much difficulty there is in communicating to actors outside of the scientific field about what these innovations are.

What Are Neurotechnologies? Do Neural Organoids Count?

The term neurotechnology has begun to acquire a legal definition in national and international lawmaking efforts. In the UNESCO Recommendation, for instance, “[n]eurotechnology refers currently to devices, systems and procedures–encompassing both hardware and software–that directly measure, access, monitor, analyse, predict or modulate the nervous system to understand, influence, restore or anticipate its structure, activity and function.”[1] This provides a sweeping scope for the object of regulation, but does not specifically mention organoids.

The OECD’s legal instrument has a very similar definition to UNESCO, raising similar questions about whether neural organoids fit.[2] In the US, at the federal level, legislation was recently introduced in the Senate that would call on the Federal Trade Commission (FTC) to investigate whether and how “neurotechnology” raises data protection issues. The proposed MIND Act would contain a definition of neurotechnology that also very closely mirrors the UNESCO and OECD instruments, providing no greater clarity.[8]

While the term “neurotechnology” appears to have a very broad definition in emerging legal instruments, lawmakers generally use the term to mean something much narrower. The focus of lawmaking bodies and the experts consulting with them is predominantly on physical devices with software components—primarily, and sometimes exclusively, on “brain-computer interfaces” (BCIs) or brain stimulation devices.[9]

But neural organoids could still potentially fit into these instruments. The UNESCO definition, by including the clarifying clause “encompassing both hardware and software,” does appear to suggest that the definition applies to physical and digital devices. But its interpretation would depend on whether the clause is read to be the full extent of what “devices, systems, and procedures” can mean, or merely two examples of what those larger terms can be. Of note, the UNESCO instrument goes on to provide examples of neurotechnology that do focus on devices, but is proceeded by the caveat that “[n]eurotechnology includes, but is not limited to:”.[1] These elements of the instrument illustrate the lack of complete clarity in its scope.

Within the UNESCO instrument, at least, it is not wholly unreasonable that neural organoids could be interpreted to be “systems and procedures . . . that directly . . . analyse, predict, or modulate the nervous system,” especially to “understand [or] anticipate its structure, activity and function.”[1] Whether the UNESCO instrument applies exclusively to physical and digital devices or could apply to biological objects appears unclear at present.

The Potential for Legal Unpredictability or Mismatches

Where does this leave the governance of neural organoids? It is currently not clear, legally or scientifically, whether neural organoids are a part of neurotechnology. National and global rules are being set for neurotechnology, but not specifically for organoids.

This prompts the question: Do rules for neurotechnology apply to neural organoids?

When looking at current legal definitions, it looks likely—but not certain—that those rules do not apply to organoids. Yet, lawmaking bodies, regulators, or courts appear to have at least some interpretive room that could be used to position neural organoids as neurotechnology, and therefore subject to at least some of these new rules or norm-setting processes on neurotechnology. At the same time, the scientific field of neural organoids is still in the process of organizing itself and has had trouble communicating to external actors about what they do and why.[7] This lack of clarity within the scientific field itself, while understandable given its youth, may invite or enable decision-makers to more readily engulf neural organoids within neurotechnology rules.

This lack of legal and scientific clarity could create predictability issues, since organoid scientists and product developers may not know whether any of the emerging rules on neurotechnology will apply to their activities. It could also raise concerns for funders and investors, or even insurers for organoid-based therapeutics, who may or may not want to see compliance with those rules as a condition of supporting neural organoid work.

The definitional uncertainty also raises questions about whether neurotechnology rules are fit and appropriate for neural organoids, or if applying those rules could result in policy mismatches. Most of the rules about neurotechnology that have been set have mostly or only medical and consumer devices in mind—not biological innovations like organoids.[9] Applying those rules to neural organoids without thoughtfully considering whether and how they should be adapted to biologics could result in poor match between the goals of those rules and the issues that organoids may pose.

Clarifying Definitions, For Now

Since most existing and emerging legal instruments on neurotechnologies appear to only have medical and consumer devices in mind, this blog tentatively recommends that lawmaking and regulatory bodies should consider clarifying that these rules do not apply to neural organoids—at least, for now. At minimum, clarifying that neurotechnology rules do not currently apply to neural organoids would provide short-to-medium-term predictability to scientists and other stakeholders and avoid applying rules that may not be fit-for-purpose.

Since definitions are vague, it is unlikely that this path would require full amendments of these legal instruments. Clarification could instead come in the form of formal guidance issued by governing bodies, or less formally, in the form of public statements from officials about whether the implementation of those rules would cover neural organoids.

It may, however, be prudent to not close the door entirely to these legal instruments applying to neural organoids at some point in the future. While the neural organoid field would benefit from predictability, an international and interdisciplinary group of experts has agreed it is likely that governance for these emerging technologies will be required at some point.[3] Custom rules for neural organoids may be preferable, but having governance instruments for neurotechnologies available in reserve may be valuable if no such tailored rules arise.

References

[1] UNESCO, Recommendation on the Ethics of Neurotechnology (2025).

[2] OECD, Recommendation of the Council on Responsible Innovation in Neurotechnology, OECD/LEGAL/0457 (2019).

[3] Sergiu P. Pașca, et al., The Need for a Global Effort to Attend to Human Neural Organoid and Assembloid Research, 390 Science 574 (2025).

[4] Mitch Leslie, Lab-Grown Models of Human Brains are Advancing Rapidly. Can Ethics Keep Pace?, Science (Nov. 18, 2025), https://www.science.org/content/article/lab-grown-models-human-brains-are-advancing-rapidly-can-ethics-keep-pace.

[5] H. Isaac Chen, Hongjun Song & Guo‐li Ming, Applications of Human Brain Organoids to Clinical Problems, 248 Developmental Dynamics 53 (2019).

[6] Madeline G. Andrews & Arnold R. Kriegstein, Challenges of Organoid Research, 45 Annual Review of Neuroscience 23 (2022).

[7] Sergiu P. Pașca, et al., A Nomenclature Consensus for Nervous System Organoids and Assembloids, 609 Nature 907 (2022).

[8] MIND Act of 2025, S.2925, 119th Cong. §3(5) (2025).

[9] Walter G. Johnson, It’s (Not) Just Semantics: “Neurotechnology” as a Novel Space of Transnational Law, 50 Law & Social Inquiry 865 (2025).

This is the final part of a three part blog post. 

The first part of this blog post explored the science behind the study published by Hashizume et al.[1] earlier this year, the second part dove deeper into the risks and limitations, now this third blog post will be dedicated to exploring the profound ethical questions it raises if the technique advances to clinical application and further exploring the legal landscape that currently governs such interventions across jurisdictions.

1. The ethical challenges: From Somatic to Germline Applications

A first ethical challenge is heritable impact. Somatic interventions affect only the treated individual, while germline interventions alter descendants who cannot consent and who may face risks we cannot yet characterize. Because uncertainty and the moral stakes grow with heritability, international reports converge on a cautious posture: allow tightly governed somatic applications; treat heritable uses as impermissible, or at minimum, not ready for clinical use. Authoritative frameworks (NASEM 2017[2]; WHO 2021[3]; ISSCR 2021[4]) and regional instruments (e.g. the Council of Europe’s Oviedo Convention, Article 13) reflect this asymmetry.

In practice, there are three routes to a germline change. First, embryo editing before implantation. Second, in-utero (fetal) delivery that reaches the fetal ovaries or testes, as some large animal studies show prenatal AAV vectors can cross to fetal germ cells and expose the pregnant patient, turning a somatic attempt into a potential germline edit[5]. Third, post-birth somatic delivery can, depending on the vector and route biodistributed to gonads. Regulators, therefore, require biodistribution studies and, when relevant, germline transmission safeguards in gene therapy trials, and keep heritable uses off-limits, insisting that any near-term exploration remain somatic and ideally, ex vivo[6].

A second challenge concerns risk, uncertainty and intergenerational responsibility. Even when somatic risks can be bounded, germline interventions raise questions about how much unknown long-term risk is acceptable and who bears it (future persons, families, and health systems). Leading reports propose high evidentiary thresholds (robust preclinical data, credible monitoring plans) and, for now, warn that heritable applications should not proceed to clinical trials. The ethical analysis conducted by the Nuffield Council on Bioethics allows that heritable editing could become permissible only under stringent conditions and with attention to social justice, not as a default extension of somatic success. A valid argument supporting germline editing in this specific case could come from how early neurodevelopmental differences in Down syndrome occur, as the only window to influence brain outcome might be at the blastocyst stage.

Third, consent and authority look different across the spectrum. For children, the accepted standard is parental permission with the child’s assent when developmentally possible[7], bounded by the harm principle as a threshold for overriding parental choices that would expose a child to serious risk or deny substantial benefit[8]. For adults with intellectual disabilities, international human rights (CRPD, Article 12) emphasize supported decision-making and respect for will and preferences, rather than defaulting to substitute decisions. These frameworks together suggest a narrow, medically grounded scope for pediatric somatic editing and a structured approach to adult participation[9].

Fourth, disability justice concerns complicate both somatic and germline contexts but become especially salient as one nears prenatal or embryonic uses. The expressivist critique holds that practices aimed at preventing the birth of people with certain genetic traits, through testing, selection or editing, send a social message that people who live with those traits are less valued and . For Down syndrome, this plays out in two concrete ways. In the prenatal setting, programs that emphasize screening and offer chromosome level prevention tools can be experiences by many in the Down syndrome community as an implicit statement that “people like me should not be born”, regardless of any individual parent’s motives. After birth, treating a comorbidity does not carry the same message because it targets a complication, not the person’s identity; by contrast, projects framed as “eliminating Down syndrome” in an existing child, or “normalizing cognition” risk re inscribing the expressivist harm by casting core identity features as defects to be corrected. Even if one does not accept the strongest versions of the argument, it rightly demands a practical commitment: inclusive governance; , careful language; co-designing studies with Down syndrome organizations; pairing any biomedical work with visible investments in education, support and inclusion; and promoting authentic engagement with disability communities to avoid stigmatizing effects and to align research aims with lived priorities. Seminal disability-rights analyses by Parens & Asch[10] and subsequent scholarship remain touchstones here[11]^/[12].

Fifth, justice and access. Somatic programs with clear clinical endpoints can, in principle, be assessed for fair selection, benefit sharing and long-term follow-up. Germline pathways raise additional concerns about distributional fairness, cross-border “forum shopping” and the social meaning of who gets invited into (or excluded from) genetic “prevention”.

Finally, there’s a problem of line drawing: therapy versus “normalization”, disease prevention versus trait selection. Therapy means treating discrete medical problems, like repairing an AV canal, addressing hematologic disease, thyroid replacement, sleep apnea, or, if it becomes feasible and safe, reducing the high risk of Alzheimer type dementia in adulthood. These target complications, health and function, not personhood and identity, and can be pursued as non-heritable, somatic care. By contrast, “normalization” projects seek to shift global cognition or other core features toward a neurotypical baseline. Even when motivated by care, they risk casting identity-defining differences as defects, raising the expressivist concern and cutting against neurodiversity. The neurodiversity perspective reminds us that many people with Down syndrome and their families value characteristic cognitive styles, social strengths, and culture; their goal is support and removal of barriers, not a neurotypical makeover.

A practical ethics approach therefore, prioritizes disease specific, non-heritable interventions co-designed with people with Down syndrome and their families, and treats success as improved functions and well-being. Somatic editing anchored to well-characterized diseased with measurable outcomes is easier to justify; germline contexts magnify slippery slope worries because choices are made before anyone can express interests or dissent, and because the non-identity problem complicates “benefit” claims for future persons. The upshot from major reports is not moral paralysis but procedural humility, by keeping the clinical scope narrow, the evidentiary bar high, and the governance participatory.

Our decision to pursue or refrain from this technology will ultimately reflect deeper cultural values about human difference and diversity. The disability community has long argued that their genetic variations are not merely medical conditions to be corrected, but integral components of human cultural diversity that enrich our collective experience [13]. From this perspective, Down syndrome is not simply a chromosomal abnormality. The elimination of specific genetic variations, whether through selective pregnancy termination or chromosome editing, raises profound questions about which lives we deem worthy of coming into existence and how we value certain differences.

As so the ethical and regulatory questions become less abstract, and it becomes a matter of setting guardrails where limited plausibility intersects with non-trivial risk.

2. Guardrails: where plausibility meets risk

If the ethical analysis points anywhere, it points to a very narrow lane: somatic, non-heritable interventions aimed at specific medical problems, not at reshaping identity laden traits. Keeping effects confined to the treated person avoids passing uncertainty to future generations, tying aims to concrete clinical endpoints respects disability perspectives by targeting complications and not people.

Because the science we’ve reviewed shows both promise and real hazards (allelic mistargeting at intended loci, structural variant scars, and uncertainty in vivo delivery) the evidence bar has to be high before first in human work. In practice that means rigorous preclinical characterization of in target fidelity, checks for off-target and structural variants, conservative dose findings, and long-term follow-up plans. None of this is red tape for its own sake, it’s a direct answer to the genomic and procedural risks catalogued earlier and is broadly consistent with current regulatory expectations for genome editing products.

The legal landscape largely creates restraints. In the United States, the FDA oversees somatic gene editing, but federal funding and even federal approval of clinical trials for germline editing is prohibited. In much of Europe, The Council of Europe, under the Oviedo Convention, prohibits introducing heritable modifications. Latin America, meanwhile, presents a mosaic of regulations, often with limited enforcement. Read together, these realities effectively close the germline door and leave only a somatic gene therapy path (which is exactly where the ethical case is stronger anyway, but the benefits might be smaller).

Finally, responsible work here has to be transparent and participatory, by creating public registries, independent oversight, and early, genuine partnership with disability communities are essential to align the research aimed with lived priorities. If this moves at all, it should move in the open, with reasons the public can see and evaluate.

Current systems were not designed with chromosome-scale editing in mind, and they struggle to keep pace with rapid scientific advances. There is growing recognition that adaptive regulatory models—such as staged approvals based on accumulating safety data—will be needed to responsibly manage the risks and benefits of this technology as it moves toward clinical reality.

Conclusion: Are the risks worth the benefits?

Taken together, the Hashizume study is best read as a proof of principle in cells, not a clinical blueprint. It shows that chromosome level editing can, under controlled conditions, restore more typical gene expressions. It also shows why translation will be hard, allele mistargeting at intended loci, repair scars from double strand breaks, and a stubborn delivery problem that only grows as we move from dishes to organs.

Here, the ethics must not be an afterthought but a compass. We have a duty to develop therapies that can alleviate suffering, but also protect individuals and communities from harm. As we contemplate following this path, we will be forced to ask ourselves big societal questions: how do we balance the hope of alleviating suffering against the risks of unintended harm? Who decides which conditions require these interventions, and how do we ensure respect and the rights of people with disabilities? We need to concentrate on technical innovation while dedicating similar efforts on developing robust ethical, legal and societal frameworks that can keep pace with the accelerating capabilities of genome engineering. Embryo stage and broad fetal uses concentrate uncertainty and heritable impact and are therefore both technically and normatively ill-suited for clinical pursuit. Post-birth somatic uses, especially ex vivo approaches in blood and immune compartments, align better with that we can responsibly justify, as the effects are confined to the treated person, endpoints are concrete and risks can be monitored. The clinical universe is also small, as beyond trisomy 21, plausible candidates are largely trisomy 13 or 18 and sex chromosome aneuploidies, so ambition should be trimmed to fit reality rather than rhetoric. If work moves forward, it should do so narrowly and in the open, somatic, non heritable aims, rigorous preclinical evidence of fidelity and safety, long term follow up and disability inclusive governance. In paralell, the right scientific bets are clear, safer, non-DSB modalities, better in vivo delivery, and honest replication in relevan human cells and tissues.

I land then on trisomic rescue belonging in the laboratory for now, and if ever in patients only in tightly bounded, somatic, ex vivo studies with clear medical endpoints. Everything upstream, embryo editing, identity shaping goals, organ wide fetal or neonatal interventions, should remain off the table until the science, the law and the social mandate say otherwise. Our choices here signal not just what we can do, but what kind of diversity we intend to welcome.

[1] Hashizume et al., “Trisomic Rescue via Allele-Specific Multiple Chromosome Cleavage Using CRISPR-Cas9 in Trisomy 21 Cells.”

[2] Human Genome Editing: Science, Ethics, and Governance, with Committee on Human Gene Editing: Scientific, Medical, and Ethical Considerations et al. (National Academies Press, 2017), https://doi.org/10.17226/24623.

[3] WHO Expert Advisory Committee on Developing Global Standards for Governance and Oversight of Human Genome Editing. Human Genome Editing: Recommendations, 1st ed (World Health Organization, 2021).

[4] International Society for Stem Cell Research, ISSCR Guidelines for Stem Cell Research and Clinical Translation, Version 1.1, May 2021, 2021.

[5] Beltran Borges et al., “Prenatal AAV9-GFP Administration in Fetal Lambs Results in Transduction of Female Germ Cells and Maternal Exposure to Virus,” Molecular Therapy Methods & Clinical Development 32, no. 2 (2024), https://doi.org/10.1016/j.omtm.2024.101263.

[6] U.S. Department of Health and Human Services et al., S12: Nonclinical Biodistribution Considerations for Gene Therapy Products, 2023.

[7] Aviva L. Katz et al., “Informed Consent in Decision-Making in Pediatric Practice | Pediatrics | American Academy of Pediatrics,” accessed September 16, 2025, https://publications.aap.org/pediatrics/article/138/2/e20161485/52519/Informed-Consent-in-Decision-Making-in-Pediatric?utm_source=chatgpt.com?autologincheck=redirected.

[8] Douglas S. Diekema, “Parental Refusals of Medical Treatment: The Harm Principle as Threshold for State Intervention,” Theoretical Medicine and Bioethics25, no. 4 (2004): 243–64, https://doi.org/10.1007/s11017-004-3146-6.

[9] Aviva L. Katz et al., “Informed Consent in Decision-Making in Pediatric Practice | Pediatrics | American Academy of Pediatrics.”

[10] Erik Parens and Adrienne Asch, “Disability Rights Critique of Prenatal Genetic Testing: Reflections and Recommendations,” Mental Retardation and Developmental Disabilities Research Reviews 9, no. 1 (2003): 40–47, https://doi.org/10.1002/mrdd.10056.

[11] A Asch, “Prenatal Diagnosis and Selective Abortion: A Challenge to Practice and Policy.,” American Journal of Public Health 89, no. 11 (1999): 1649–57, https://doi.org/10.2105/ajph.89.11.1649.

[12] Non-Invasive Prenatal Testing: Ethical Issues (Nuffield Council on Bioethics, 2017).

[13] Felicity Boardman, “Human Genome Editing and the Identity Politics of Genetic Disability,” Journal of Community Genetics 11, no. 2 (April 2020): 125–27, https://doi.org/10.1007/s12687-019-00437-4.

This is Part II of a three part blog post. 

As explained in detail in the first part of this blog post, the study published by Hashizume et al.[1] demonstrated a technique that could selectively eliminate the extra chromosome in trisomy 21, by targeting the demolition and subsequent removal of an entire chromosome while leaving the necessary pair intact. The proposed approach corrects the underlying genetic imbalance, reversing cellular abnormalities and restoring normal gene expression patterns in laboratory-grown cells. However, while the promise of chromosome level editing to correct trisomy 21 is remarkable, a complicated set of issues need to be addressed and discussed while the technique advances, if it ever does, to in vivo application. The paper provides valuable data on several categories of risk that deserve scrutiny.

1. Risks and limitations

Genomic Risks:

Off-target effects:

The most immediate concern with any CRISPR-based therapy is off-target genetic damage. Despite the allele-specific approach’s precision compared to non-specific targeting, whole-genome sequencing revealed 5-6 structural variants (“SVs”) in post-edit trisomy clones and approximately 1 SV in post-edit disomy clones, evidence that the remaining chromosomes bear scars from the process.

Importantly, when the team aimed to cut only the extra chromosome 21, but 5 of the 13 cut sites the guides occasionally snipped one of the two normal copies, because those DNA sequences differ by only a single letter. This means that the CRISPR system sometimes mistakenly targeted the “good” copies of chromosome 21 rather than just the extra copy. Even when the researchers refined their approach to six gRNAs with demonstrated specificity, some residual off-target activity persisted.

Mosaicism:

Mosaicism cuts both ways. A small share of people with Down syndrome are naturally mosaics, meaning only some of their cells carry the extra chromosome, and, as a group, they often have milder clinical features than those with full trisomy 21. Severity varies with the proportion and tissue distribution of trisomic cells. From that lens, nudging tissues toward a higher fraction of disomy cells could plausibly soften some symptoms. But therapy induced mosaicism at today’s per cell rescue rates would likely be patchy across organs and may include cells with repair related rearrangements, such cellular heterogeneity might lead to unpredictable developmental outcomes or long-term instability in treated tissues.

Procedural Risks:

From lab to therapy:  

Even if genomic risks could be mitigated, substantial procedural challenges remain. The paper reports extremely low delivery efficiency—only 1.07% of induced pluripotent stem cells and 13.9% of fibroblasts successfully received the CRISPR-Cas9 system via electroporation (a technique that uses millisecond electrical pulses to momentarily open pores in cell membranes, allowing the CRISPR plasmid to enter). This inefficiency would severely limit therapeutic applications, particularly for tissues with limited regenerative capacity like neurons.

Perhaps most critically, the study demonstrates efficiency only in isolated cell cultures. The leap to treating intact tissues or whole organisms introduces enormous additional complexities. The researchers showed some promise in non-dividing cells, achieving a modest 3.2% chromosome elimination rate, but this falls far short of what would be needed for meaningful therapy.

These substantial risks must be weighed carefully against the potential benefits of CRISPR-based chromosome elimination therapy. As the next section will explore, the demonstrated cellular improvements following successful chromosome elimination suggest significant therapeutic potential, but is it enough to justify these risks?

2. Human use in Practice

As with any powerful technology, the ethical landscape of chromosome editing is complex and evolving. Right now, research is confined to somatic cells in laboratory settings, which keeps risks relatively contained and allows for careful studying of safety and efficacy. However, the technology’s trajectory points toward more challenging scenarios. In the near future we might see attempts to treat adults with Down syndrome-related conditions, such as leukemia, using edited blood cells.

It’s important to note that beyond trisomy 21, whole chromosome “rescue” would plausibly apply only to the small set of extra chromosome conditions (aneuploidies) compatible with live births, primarily, trisomy 13 and 18 and the sex chromosome trisomies. Other full autosomal trisomies are almost uniformly embryonic or fetal lethal, the rare liveborn cases are typically mosaic or involve partial/segmental trisomies.[2]^/[3].

However, before we argue the ethics in the abstract, it helps to ask a practical question: when, if ever, could this be used in humans? There are four key moments in which the technique presented by Hashizume et al. could be attempted, each having a mix of what’s plausible and what could go wrong.

Editing before implantation is the most conceptually attractive point to act: removing the extra copy of chromosome 21 at the eight-cell or blastocyst stage could, in theory, prevent downstream effects. In practice, however, early embryos often respond unpredictably to breaks, with mosaic outcomes and large, unintended alterations. Because any change at this stage would be heritable, the scientific uncertainties and regulatory prohibitions align: this route is technically imaginable but currently inadvisable[4]^/[5].

By the end of the first trimester, some features associated with Down Syndrome, especially structural heart differences, are already set[6], while other systems, including the lungs and brain, continue to develop. That timing makes early fetal intervention appealing in theory but difficult in execution. Delivering a complex editing payload safely to a high share of cells in specific organs, remains a major constraint as any dose given during pregnancy reaches to patients, the pregnant person and the fetus. The delivery vehicles that carry CRISPR (often AAV viruses or nanoparticles) can spread to the mother’s organs and can also cross the placenta. Because exposure of maternal tissue or the fetal germline would turn a somatic treatment into a potential germline one, routes and doses must be very conservative. Additionally, many adults already have antibodies against AAV, which can neutralize the vector before it reaches the fetus [7]^/[8]. Even if delivery were feasible, the organs most people hope to influence, particularly the brain, are precisely those that are the hardest to reach at scale.

Soon after birth, the picture shifts. Narrow somatic uses, especially ex vivo approaches where cells are edited outside the body and then returned, seem more credible than attempting a whole-body approach. Blood cells are the most realistic targets[9][10], and some Down syndrome associated comorbidities, such as leukemias, and in some cases lymphomas live in those compartments. By contrast, trying to reach enough cells in the brain, heart, or lungs to change overall development runs into a practical coverage problem[11], meaning the fraction of target cells that actually receive and execute the edit. Organ level change usually requires very high coverage in specific cell types, which is hard because (i) vectors like AVV or nanoparticles distribute unevenly, and the brain ads a blood-brain barrier; (ii) dose limits and immune responses cap how much you can give or re-dose; and (iii) this approach requires multiple components to reach the same cell.

In childhood and adulthood, consent considerations improve for some patients, yet the biology does not become easier. The central nervous system remains difficult to access broadly and safely, and most of today’s delivery vehicles (especially AAV vectors) can’t be given repeatedly, one dose prompts neutralizing antibodies that often persist for years and block later doses[12]; ex vivo strategies for blood or immune conditions continue to make the most sense.

Across all stages, the central constraint is less the cleverness of the method than the ability to deliver it safely to enough of the right cells. Where that hurdle is tractable (narrow, somatic, ex vivo contexts with concrete clinical endpoints), a cautious path may emerge. Where it is not (embryo editing or broad fetal/neonatal/adult organ-wide interventions), the risk-benefit profile remains unfavorable. The recent trisomy 21 “rescue” study in cells is an impressive proof-of-principle but it underscores how far we are from organ-scale use on people.

Read against that practical map, the ethical questions sharpen, as we move from somatic to germline contexts, who bears the risks, how is consent obtained, and what counts as benefits all change, and the justifications that may hold for narrow somatic uses no longer carry unmodified into prenatal or embryonic settings.

Opposition is likely to be the weakest where an intervention clearly treats disease or prevents serious harm after birth, like repairing life threatening cardiac or pulmonary problems, treating hematologic disease, or, if credible evidence emerges, reducing the markedly elevated lifetime risk of Alzheimer’s disease in adults with Down syndrome. In pediatric ethics and practice, parental discretion is broad but not unlimited, when refusing a low burden, high benefit intervention exposes a child to a significant risk of serious harm, clinicians and, if necessary, the state, may override refusal. Applying chromosome level editing to modify global cognition is a different category, it currently lacks proven safety and efficacy, targets identity laden traits, and should, if ever contemplated, remain optional and grounded in robust supported decision making rather than compelled care.

[1] Hashizume et al., “Trisomic Rescue via Allele-Specific Multiple Chromosome Cleavage Using CRISPR-Cas9 in Trisomy 21 Cells.”

[2] Warburton, Dorothy, Louis Dallaire, Maya Thangavelu, Lori Ross, Bruce Levin, and Jennie Kline. “Trisomy Recurrence: A Reconsideration Based on North American Data.” American Journal of Human Genetics 75, no. 3 (September 2004): 376–85. https://doi.org/10.1086/423331

[3] Nagaoka, So I., Terry J. Hassold, and Patricia A. Hunt. “Human Aneuploidy: Mechanisms and New Insights into an Age-Old Problem.” Nature Reviews Genetics 13 (June 18, 2012): 493–504. https://doi.org/10.1038/nrg3245

[4] Michael Kosicki et al., “Repair of Double-Strand Breaks Induced by CRISPR–Cas9 Leads to Large Deletions and Complex Rearrangements,” Nature Biotechnology 36, no. 8 (2018): 765–71, https://doi.org/10.1038/nbt.4192.

[5] Gregorio Alanis-Lobato et al., “Frequent Loss of Heterozygosity in CRISPR-Cas9-Edited Early Human Embryos,” Proceedings of the National Academy of Sciences of the United States of America 118, no. 22 (2021): e2004832117, https://doi.org/10.1073/pnas.2004832117.

[6] Jill P. J. M. Hikspoors et al., “A Pictorial Account of the Human Embryonic Heart between 3.5 and 8 Weeks of Development,” Communications Biology 5, no. 1 (2022): 226, https://doi.org/10.1038/s42003-022-03153-x.

[7] John S. Riley et al., “Preexisting Maternal Immunity to AAV but Not Cas9 Impairs in Utero Gene Editing in Mice,” The Journal of Clinical Investigation 134, no. 12 (n.d.): e179848, https://doi.org/10.1172/JCI179848.

[8] Rrita Daci and Terence R. Flotte, “Delivery of Adeno-Associated Virus Vectors to the Central Nervous System for Correction of Single Gene Disorders,” International Journal of Molecular Sciences 25, no. 2 (2024): 1050, https://doi.org/10.3390/ijms25021050.

[9] Haydar Frangoul et al., “CRISPR-Cas9 Gene Editing for Sickle Cell Disease and β-Thalassemia,” New England Journal of Medicine 384, no. 3 (2021): 252–60, https://doi.org/10.1056/NEJMoa2031054.

[10] Haydar Frangoul et al., “Exagamglogene Autotemcel for Severe Sickle Cell Disease,” New England Journal of Medicine 390, no. 18 (2024): 1649–62, https://doi.org/10.1056/NEJMoa2309676.

[11] Petr O Ilyinskii et al., “Readministration of High-Dose Adeno-Associated Virus Gene Therapy Vectors Enabled by ImmTOR Nanoparticles Combined with B Cell-Targeted Agents,” PNAS Nexus 2, no. 11 (2023): pgad394, https://doi.org/10.1093/pnasnexus/pgad394.

[12] Daci and Flotte, “Delivery of Adeno-Associated Virus Vectors to the Central Nervous System for Correction of Single Gene Disorders.”

This is the first of a three-part blog post. 

“After decades of concentrated efforts, we are finally learning how to harness natural processes honed over millions of years of evolution to translate our unparalleled ability to read the genome into a new opportunity to write it and to correct the grave mistakes that turn it against us”

-Euan Angus Ashley[1]

Introduction: A scientific inflection point

In 1959, French geneticist Jérôme Lejeune made a landmark discovery – Down syndrome was caused by the presence of an extra copy of chromosome 21. For over six decades since this revelation, our approach to this most common chromosomal disorder has been primarily supportive: therapies to address symptoms, early interventions to maximize development, and social inclusion to improve quality of life. Despite tremendous advances in understanding the molecular basis of the condition, the fundamental problem – that extra chromosome – remained untouchable. We could detect it, study it, even track how it disrupts normal development, but we could not remove it. Until now.

In a groundbreaking study published in PNAS Nexus in early 2025, Hashizume et al.[2] demonstrate a technique that could selectively eliminates the extra chromosome in trisomy 21. Using a sophisticated application of CRISPR-Cas9 gene editing technology, the researchers achieved something once thought impossible: targeted removal of an entire chromosome while leaving the necessary pair intact. This approach corrects the underlying genetic imbalance, reversing cellular abnormalities and restoring normal gene expression patterns in laboratory-grown cells.

Although this work took place only in human cell lines and not in living people, or human embryos, its possible implications are profound. Rather than treatments that address individual symptoms, this research opens the door to therapy that could potentially address the genetic root cause of Down syndrome. The technique still requires refinement but represents a watershed moment in genetic medicine: the first successful demonstration of precise, allele-specific chromosome elimination in human trisomy 21 cells.

Yet as with many scientific breakthroughs, this advance raises as many questions as it answers. If we can remove the extra chromosome causing Down syndrome, should we? When would such intervention be appropriate—in embryos, children, or adults? Who decides which genetic conditions merit correction? How do we balance medical benefit against the risks of unintended genomic damage and the value of disability?

This three-part blog post examines the remarkable science behind this chromosome-level editing technique and the complex ethical questions it raises. The first blog post will be dedicated to understanding the science: the technique proposed by Hashizume et al. to eliminate the third copy of the 21^st chromosome that causes down syndrome and how the researchers achieved allele-specific targeting. The second blog post will be dedicated to analyzing the risks and limitations; and the third blog post will dive into the difficult ethical questions it raises if the technique advances to clinical application and briefly assess the legal landscape governing such interventions across jurisdictions.

1. Down Syndrome and its management

Down syndrome occurs when an individual carries three copies of chromosome 21 rather than the typical two. The presence of a third chromosome leads to an excess of genetic material that disrupts normal cellular function and development.

Various medical issues have been associated with Down syndrome, such as congenital heart defects (40-50% of children) that range in severity, gastrointestinal problems, immune system dysfunction and infections, increased risk of leukemia and other blood disorders, thyroid disorders, hearing problems (75% of Down children experience some degree of hearing loss) and vision problems, neurological and cognitive impairment, among others[3].

The initial approach to patients with Down syndrome has been medical support. However, as science has advanced, and particularly since the emergence of prenatal genetic testing, parents have had the possibility of terminating the pregnancy once a Down syndrome diagnosis has been given. And, if they are using in vitro fertilization, they have the further option of using preimplantation genetic diagnosis to avoid transferring a trisomy 21 embryo for possible implantation, pregnancy, and birth. Available research on the subject shows significant variation per country in termination rates for prenatally diagnosed cases: Iceland near 100% termination rates; Spain, France and Germany 96%; Italy 93%; England and Wales 76%[4], and 67% in the US[5]^/[6]. The data varies significantly across different sources, but it has raised concerns among the bioethics community.

2. The science – Understanding CRISPR-Based chromosome elimination

The Hashizume study implements a molecular strategy to eliminate the extra chromosome in trisomy 21 cells. It introduces a paradigm shift in chromosomal disorder therapeutics by combining allele-specific CRISPR-Cas9 targeting (precision scissors and targeting) with DNA repair pathway modulation (sabotage of repair crews) to achieve functional trisomy rescue.

Step 1: “Fingerprinting” the Chromosome

The innovation in Hashizume’s study relies on a crucial insight: although the three copies of chromosome 21 look nearly identical, they contain subtle genetic differences that can be exploited.

Using a technique called haplotype phasing, the researchers first “fingerprinted” each chromosome to identify which was inherited from the father (P) and which from the mother (M1 and M2). After determining that the M2 chromosome was the safest to remove, they designed a CRISPR-Cas9 system that could recognize and cut only this specific chromosome.

What makes this approach revolutionary is its precision. Unlike previous “allele-nonspecific” (ANS) methods that indiscriminately cut all three chromosomes, the “allele-specific” (AS) approach targets only the extra chromosome. This distinction is critical, with this technique, the CRISPR system cuts all three chromosomes, and it overwhelms the cell’s repair mechanisms and frequently leads to cell death (87.3% of cells died). In contrast, the AS approach achieved both better survival rates (57%) and higher chromosome elimination rates (13.1% versus 6-8% for ANS methods).

Step 2: Designing Precision-Guided Scissors (multiplexed CRISPR cuts)

The mechanism works like molecular surgery: CRISPR-Cas9 creates 13 strategic cuts along the M2 chromosome, essentially shredding it into fragments. The scientists created 12 guide RNAs, molecules directing the Cas9 protein to the M2 chromosome to cut it. A considerable percentage, approximately 33%, of those gRNAs were successful. Thanks to multiple cuts in the targeted chromosome, it is impossible for the cell to repair the damage.

Step 3: DNA repair inhibition

The researchers enhanced this effect by temporarily suppressing key DNA repair genes (POLQ and LIG4), preventing the cell from stitching the broken chromosome back together. During subsequent cell division, these chromosome fragments fail to properly segregate and are ultimately lost, effectively removing the extra chromosome and restoring.

The scientists then blocked the cell repair mechanisms (NHEJ and MMEJ pathways) by temporarily blocking them with siRNA (a silencing molecule) which doubled the success rate of chromosome loss.

To verify their success, the team conducted comprehensive genetic analyses, confirming that the eliminated chromosome was indeed the targeted M2 in every case. More importantly, cells that lost this chromosome showed dramatic improvements: gene expression patterns normalized, cellular stress decreased, and proliferation rates increased—effectively reversing the cellular manifestations of Down syndrome.

[1] Euan Angus Ashley, The Genome Odyssey: Medical Mysteries and the Incredible Quest to Solve Them, 1st ed. (Celadon Books, 2021), https://us.macmillan.com/books/9781250792150/thegenomeodyssey/.

[2] Ryotaro Hashizume et al., “Trisomic Rescue via Allele-Specific Multiple Chromosome Cleavage Using CRISPR-Cas9 in Trisomy 21 Cells,” PNAS Nexus 4, no. 2 (2025): pgaf022, https://doi.org/10.1093/pnasnexus/pgaf022.

[3] “Down Syndrome – Symptoms and Causes,” Mayo Clinic, accessed April 17, 2025, https://www.mayoclinic.org/diseases-conditions/down-syndrome/symptoms-causes/syc-20355977.

[4] PA Boyd et al., “Survey of Prenatal Screening Policies in Europe for Structural Malformations and Chromosome Anomalies, and Their Impact on Detection and Termination Rates for Neural Tube Defects and Down’s Syndrome,” Bjog 115, no. 6 (2008): 689–96, https://doi.org/10.1111/j.1471-0528.2008.01700.x.

[5] “In Iceland, Almost All Diagnosed Down Syndrome Pregnancies Are Aborted after Prenatal Testing. Some Bioethics Experts Are Concerned – ABC News,” accessed April 17, 2025, https://amp.abc.net.au/article/103781058.

[6] Gert de Graaf et al., “Estimates of the Live Births, Natural Losses, and Elective Terminations with Down Syndrome in the United States,” American Journal of Medical Genetics Part A 167, no. 4 (2015): 756–67, https://doi.org/10.1002/ajmg.a.37001.