Judgment Proof

The Reign of the Safety Machine versus the Recovery of Human Wisdom

Seth Oranburg

Copyright © 2026 by Seth Oranburg

All rights reserved. No part of this book may be reproduced in any manner without written permission except in the case of brief quotations embodied in critical articles and reviews.

This is a work of nonfiction. Some names and identifying details have been changed to protect the privacy of individuals involved.

ISBN: [Pending]

Library of Congress Control Number: [Pending]

First Edition

To those who are willing to bear the weight.

“Baby, don’t hurt me / Don’t hurt me / No more.”

Haddaway, “What Is Love” (1993)

“Legislators make the citizens good by forming habits in them, and this is the wish of every legislator, and those who do not effect it miss their mark, and it is in this that a good constitution differs from a bad one.”

Aristotle, Nicomachean Ethics, Book II, 1103b

“The perfection of the body is antecedent to the perfection of the soul… But the perfection of the soul is the ultimate end.”

Moses Maimonides, The Guide for the Perplexed, Part III, Chapter 27 (c. 1190)

Contents

Preface

Part One: The Machine and Its Origins

Chapter 1: Haddaway’s Law

Chapter 2: The Confusion

Chapter 3: The Great Retreat

Chapter 4: Prudence as Transaction Cost

Chapter 5: Harm-Prevention Architecture

Part Two: The Epistemology of the Problem

Chapter 6: Wisdom of Not Knowing

Chapter 7: Nobody Knows Everything

Part Three: The Architecture of Recovery

Chapter 8: Boeing

Chapter 9: The Safety Machine

Chapter 10: The Gymnasium

Chapter 11: Aristotle’s Algorithm

Chapter 12: The Hardware

Chapter 13: The Silicon Havruta

Chapter 14: Anti-Utopia

Notes and References

Preface

I teach business law to students who will spend their careers inside the safety machine that this book describes.

Every semester, I watch them learn compliance. They arrive with the intuition that law has something to do with justice, and they leave with the professional understanding that law has something to do with documentation. I do not say this with contempt. The curriculum reflects what the profession actually demands. My students learn to identify risk, construct defensible paper trails, ensure that every decision their future employers make can be traced to a rule rather than to a person. They learn to become judgment-proof.

I trained at the University of Chicago Law School, where I absorbed the law-and-economics framework that Chapter 1 describes. Cost-benefit analysis replaces the messiness of judicial discretion. Legal rules are evaluated by whether they maximize social wealth. The conviction was that law could be practiced as a science rather than an art. I believed it, taught it for years, and built my published scholarship on it. When I tell you in this book that the Chicago School’s framework was elegant and that its best practitioners were clear-eyed about its limitations, I am speaking from inside the tradition. Chicago gave me the analytical tools I still use, and those tools helped me see what was breaking. What they could not show me was what the breaking had cost: what had been lost when the framework succeeded on its own terms and the institutions it built stopped producing the judgment they depended on.

The argument of this book grew gradually, but the experience that forced it into focus was specific.

In the summer of 2022, I was a law professor at the University of New Hampshire Franklin Pierce School of Law. The school advertised itself as offering the most flexible and accessible path to a JD in the nation. That summer, I was assigned to work a schedule that included Saturdays. I told the administration I could not work on Saturday because I am a Jew. Shabbat is a requirement of my identity, not a preference and not a scheduling inconvenience. In any functioning institution the response would have been immediate and uncomplicated.

What followed was a bureaucratic process so tangled that the people running it could not determine, over a period of weeks, whether they would accommodate an observant Jew’s refusal to work on the Sabbath. I do not mean they refused. I mean they could not figure out whether they would. The institution that claimed to be the most flexible law school in America had built an administrative architecture so rigid that a straightforward request (one that any competent principal, any community leader, any person with ordinary judgment would have resolved in a conversation) became a Kafkaesque procedural trap. I did not work on Saturday. But the resolution came despite the institution, not through it. Most of the people involved were sympathetic. They simply could not act, because the institution had been designed so that acting was not something the structure permitted. The structure permitted process, and process is what I received.

I already had, by that point, an intuition about where to look for an alternative. I had been studying Judaism seriously as a practice rather than as an academic exercise, and the tradition I was learning operated on principles that were the precise inverse of the compliance architecture I was living inside at work. When I brought the Shabbat problem to my Jewish community, the response was immediate and unequivocal. Nobody needed to consult a policy manual. The tradition had already worked out, over centuries of practice, what Shabbat observance requires and what a community owes to someone who observes it. In the vocabulary of the tradition, this was not a question. The clarity came not from simplicity but from a legal and moral tradition that had preserved, in its institutional design, the conditions under which judgment could be exercised.

The contrast was unbearable. On one side, a modern American institution with a diversity office, an accommodation policy, a Title VII compliance framework, and a bureaucracy that could not, over the course of weeks, figure out whether to let a Jewish professor observe the Sabbath. On the other, a tradition that had been governing questions like this since the twelfth century and could answer in an afternoon.

That contrast sent me to Maimonides. Not immediately; I did not open the Guide for the Perplexed that summer and find the answer on page one. But I knew, from my study of Jewish law, where to look when I needed a framework that could explain what I had experienced. Maimonides was a physician, a philosopher, and the most systematic legal thinker of the medieval Jewish tradition. His distinction between tikkun ha-guf (the material infrastructure of social order, the physical security and institutional stability that make everything else possible) and tikkun ha-nefesh (the cultivation of understanding, moral capacity, and practical wisdom) gave me the structural vocabulary I could not find in the contemporary literature on regulation. Law serves two purposes. Compliance can build the foundation, but compliance alone cannot raise the scaffolding.

Maimonides was not alone. Aristotle had made the same structural distinction. Aquinas had built institutional architecture around it. Elinor Ostrom had documented, empirically, across cultures and centuries, the design features that made institutions capable of preserving the judgment their governance required. The havruta — the Talmudic study practice in which your partner’s function is to challenge your reading — encodes an institutional design principle that Ostrom’s commons research validated independently. The Western tradition, and the Jewish tradition within it, had already solved a problem that modern regulatory theory does not know how to pose.

I direct a program at the Classical Liberal Institute at NYU Law, and I should be transparent about where I stand. The “classical liberal” label fits parts of my argument but not all of it. None of the thinkers this book draws on most heavily — Aristotle, Maimonides, Aquinas — would have recognized it. All three were institutionally serious about the cultivation of human judgment in a way that precedes and exceeds any modern political category. This book’s argument is that the architecture can be rebuilt (not demolished, rebuilt) so that the institutions we depend on can do what they were designed to do: cultivate the judgment their own legitimacy requires.

The argument begins, of course, with a Eurodance hit song.

Chapter 1: Haddaway’s Law

Where Fun Comes to Die

I arrive at the University of Chicago Law School in August wearing shorts. This turns out to be a mistake. The wind off Lake Michigan cuts through the campus with a cold that feels deliberate, as though the weather itself has internalized the school’s unofficial motto: Where fun comes to die. I go home freezing and underdressed. I am not disappointed. I have not come for fun.

Chicago promises clarity in an age of confusion. The law, as I encounter it before arriving, is a mess: competing values in collision, vague standards proliferating, judicial discretion that feels arbitrary, outcomes that vary wildly depending on which judge draws which case. People are hurt by this unpredictability. Real people, suffering real consequences from what amounts to an institutional lottery.

Chicago offers an answer: its version of law and economics (the school that Coase and Posner and Becker built) applies mathematical rigor to legal questions that had previously been resolved by intuition, precedent, and what Cardozo described as the trained practical instinct, the accumulated wisdom of experienced judges brought to bear on the resolution of disputes.¹ The Chicago School’s core insight is that legal rules produce consequences, that those consequences can be measured, and that measurement allows us to evaluate rules by something more reliable than judicial temperament. I remember the moment I feel the full appeal of this. The class is analyzing Hawkins v. McGee, the hairy hand case, a staple of first-year contracts. George Hawkins burns his hand on an electrical wire. A physician named McGee persuades him to undergo a skin graft, guaranteeing a “perfect hand.” The surgery fails. McGee grafts skin from Hawkins’s chest onto his palm. The chest skin grows thick hair. The hand is worse than before.

Traditional legal analysis spirals into questions that resist quantification: How does one compensate a young man for a disfigured hand? What is the worth of what he has lost? The economic analysis cuts through this. Hawkins expected a perfect hand. He received something worse than what he had. The difference between expectation and reality is calculable. Law should give him that difference, and the incentives thus established should discourage future physicians from making promises they cannot keep. The framework has genuine elegance. It replaces sympathy with arithmetic and the resulting clarity is, for a twenty-two-year-old first-year student, exhilarating.

The school rested on a genuine insight. In 1960, Ronald Coase published a paper demonstrating that in a world with zero transaction costs, parties will bargain to the efficient outcome regardless of how law assigns initial entitlements.² The insight was descriptive: legal rules matter in the real world because transaction costs are never zero, information is incomplete, and friction prevents parties from bargaining their way to welfare-maximizing arrangements. Coase wanted his followers to study that friction, to understand how institutions shape the way humans act in a world of imperfect information and incomplete contracting.

His followers did the opposite. They converted his description into a prescription. If transaction costs prevent efficient outcomes, then law must reduce transaction costs. If friction produces suboptimal results, then the goal of institutional design becomes the elimination of friction. The aspiration becomes a world without constraints on rational exchange. The logic has a remorseless simplicity. Constraints hurt people. Friction causes pain. The purpose of law is to prevent hurt. Therefore: eliminate friction.

I did not see the problem until I started watching what the approach built.

Haddaway’s Law

In 1993, a Trinidadian-German musician named Haddaway released a dance single that topped charts across Europe. The song poses one question, repeated with increasing desperation over a four-on-the-floor beat:

What is love?

And offers one answer, delivered in the chorus:

Baby, don’t hurt me.

The song is absurd. It is also, as an unintended description of modern legal thought, almost perfect:

What is law?

Baby, don’t hurt me.

Ask a contemporary legal scholar, a legislator, or a regulatory official: What is law for? You will receive sophisticated answers about social ordering, dispute resolution, behavioral modification, and public policy. Strip away the academic vocabulary, and you find the same structure as Haddaway’s chorus. Law exists to prevent harm. It protects consumers from dangerous products, workers from unsafe conditions, investors from fraud, the environment from pollution, citizens from violence. Law defines its purpose by what it prevents.

This is not wrong. Prevention matters. A legal system that cannot prevent murder, theft, and fraud is not a legal system. Haddaway was not wrong either; love that systematically harms is not love. The problem is not the negative definition. The problem is that the negative definition has become the only definition.

I call this Haddaway’s Law: the institutional imperative to prevent hurt, pursued without limit, without acknowledgment of what it costs, without any vocabulary for what law should enable rather than merely forbid, and without the epistemic capacity to know whether the hurt it prevents outweighs the judgment it destroys. Modern regulatory, compliance, and administrative systems operate under this imperative. They prevent investors from losing money. They prevent pilots from making errors. They prevent judges from exercising inconsistent discretion. They prevent the grandmother at the bank counter from wiring money to a hurricane relief fund without ninety minutes of interrogation about the source of her pension and the exact news broadcast that moved her to charity.

Three blocks from that grandmother, a Delaware shell company with nominee directors in Cyprus wires three million dollars through the same institution. No human examines the transaction. The algorithmic risk score registers as acceptable. The compliance boxes check out perfectly. The transaction clears in three seconds. The global financial system spends $274 billion annually on this architecture.³

The Safety Machine stops the grandmother and escorts the shell company to the vault. It cannot distinguish between them because distinction requires human discernment, and the machine is built specifically to replace human discernment with process. The machine produces two related conditions that together constitute the title of this book: institutions that are, in the precise legal and moral sense, judgment-proof.

Two Ways to Be Immune

In law, a judgment-proof defendant is one from whom no judgment can be collected. She has been sued; the plaintiff has won; the court has entered a verdict. None of this matters, because the defendant possesses no assets from which the judgment can be satisfied. She can be condemned in principle and remain immune in practice. The formal apparatus of accountability operates, but accountability does not.

Modern institutions have achieved a more ambitious version of this condition. They are judgment-proof not merely because they lack assets but because they have engineered out the faculty of professional discernment against which accountability would operate. When no one judges, no one can be said to have judged wrongly. Failure becomes a technical malfunction rather than a human error. The institution can point to the algorithm, the checklist, the compliance report, the peer-reviewed risk model. The paperwork is immaculate. No one is responsible. No one exercised discretion. No one can be held to account for the assessments that were never made.

Institutions become judgment-proof in the liability sense precisely because they have become judgment-proof in the epistemic sense. The two meanings are not a rhetorical coincidence. They describe a causal relationship. Eliminate judgment, and you eliminate accountability for wisdom that was never applied. No one exercised deliberation. No one can be assigned responsibility for outcomes that deliberation might have changed.

This double meaning is the subject of this book.

Planes without Pilots

The Boeing 737 MAX is the paradigm case.

In 2011, Airbus launched the A320neo, featuring new engines promising a fifteen percent reduction in operating costs.⁴ Boeing faced a competitive choice: design an entirely new aircraft, a decade’s work and tens of billions of dollars, or re-engine the existing 737 airframe. Boeing chose speed. Engineers mounted larger, more fuel-efficient LEAP-1B engines on the 737, moving them forward and slightly higher to fit beneath the low-slung fuselage. The repositioning altered the aircraft’s aerodynamic behavior. At high angles of attack, the engine housings generated additional lift, causing the nose to pitch upward, a characteristic that, if left unaddressed, would require pilots to receive extensive training to manage.⁵

Training, however, was the problem. Airlines would not purchase the 737 MAX if it required pilots to complete a new type rating. The business model demanded that the aircraft maintain its existing certification, keeping transition costs minimal. Traditional aviation philosophy would have addressed the aerodynamic change by developing the judgment of the pilots who would fly it: simulator time, procedural training, the development of what engineers call “stick and rudder” skills that allow pilots to manage an aircraft’s handling characteristics across its flight envelope. This approach required friction. It required time, money, and human adaptation. It treated the pilot as an agent who needed to develop capacity.

Boeing chose automation. Engineers designed the Maneuvering Characteristics Augmentation System, MCAS, to push the nose down automatically when sensors detected a steep pitch angle.⁶ Pilots did not activate the system. Pilots were not informed of its existence. The aircraft was redesigned to function without pilot authority in the flight regime where the aerodynamic problem lived. The friction of training was eliminated. The pilot’s capacity to compensate manually was never developed. The machine would handle it.

The FAA processed this design through an institutional architecture built on compliance delegation. Under the Organization Designation Authorization program, the FAA delegated certification activities to Boeing itself: the manufacturer authorizing its own aircraft.⁷ Boeing characterized MCAS as a minor modification of an existing speed trim system. The FAA, delegating up to eighty-seven percent of certification activities to Boeing, processed the characterization accordingly.⁸ The paperwork was impeccable. Every box was checked, every protocol satisfied, every required form filed.

On October 29, 2018, a faulty angle-of-attack sensor fed false data to MCAS on Lion Air Flight 610. The software executed its function flawlessly, repeatedly commanding the horizontal stabilizer to push the nose down. The pilots fought a system they did not understand, using skills they had not been trained to develop. The aircraft impacted the Java Sea at high speed, killing all 189 people aboard.⁹

Boeing issued an advisory. The global fleet continued to fly.

Four months and eleven days later, on March 10, 2019, Ethiopian Airlines Flight 302 encountered the same failure. The timeline bears reproduction in full, because the temporal precision reveals the mechanical inexorability of the architecture.

At 8:38 AM, the aircraft lifts off runway 07R at Addis Ababa Bole International Airport. Seconds into the climb, the left angle-of-attack sensor fails, feeding false data to the flight computer. At 8:39, the stick shaker activates, the control column vibrating violently in the captain’s hands, warning of a stall that is not occurring. The autopilot disengages. At 8:40, MCAS activates, commanding the horizontal stabilizer nose-down. The pilots pull back on the yoke with immense force, fighting the machine. MCAS activates again. And again. At 8:41, the pilots execute the runaway stabilizer trim cutout procedure, cutting electrical power to the trim motor. MCAS is neutralized, but the aircraft is now accelerating to well beyond its designed speed. At 8:42, the pilots attempt to turn the manual trim wheels on the center pedestal. Aerodynamic forces on the tail at this speed exceed what human arms can overcome. The wheels will not move. At 8:43, the pilots restore electrical power to the trim motor, hoping to regain control of the stabilizer. MCAS immediately commands another massive nose-down pitch. At 8:44, six minutes after takeoff, the aircraft enters an unrecoverable dive. Flight 302 strikes the ground near Bishoftu.¹⁰ All 157 aboard are killed.

The combined death toll is 346 people.¹¹

They died because the machine had eliminated the friction of judgment. Boeing designed an aircraft that did not require pilot judgment in the relevant flight regime. The pilots never practiced compensating for the pitch characteristics. They never developed the embodied, habituated skill — what Aristotle would have called hexis — that would have made manual recovery possible. The automation was not incidental. It was the architecture. MCAS was the Safety Machine built in aluminum and software. When MCAS failed, it failed in the direction of the very harm it was designed to prevent. The humans operating within the system lacked the capacity to intervene, because the system had been designed on the premise that no such capacity would ever be needed.

The compliance documentation was immaculate. The ODA delegation was legally sound. The aircraft was perfectly legal and fundamentally lethal. The institution was judgment-proof in both senses.

What We Forgot

The argument of this book is not that safety is bad. It is not that regulation is a mistake or that friction should be maximized for its own sake. The Boeing 737 MAX is not an argument against aviation regulation; it is an argument about what happens when the wrong kind of regulation is pursued with the wrong kind of logic.

The argument is this: we have forgotten that law has two purposes, not one. We have forgotten that law has limits built into its nature: limits on what it can legitimately specify, limits on what it can effectively accomplish, limits that arise not from external constraints but from what law is. And we have forgotten that these two amnesias are connected, that forgetting law’s second purpose and forgetting law’s inherent limits are two expressions of the same underlying error.

The first purpose of law is foundational. It suppresses violence, secures persons and property, establishes the basic ordering of social life that makes everything else possible. The Romans called this pax and ordo. Maimonides, the twelfth-century rabbi and physician of medieval Cairo, called it tikkun ha-guf, the perfection of the body, the physical infrastructure of social existence.¹² This purpose is genuine and essential. A legal system that cannot prevent murder, fraud, and theft is not a legal system. The foundation must be built before anything else is possible.

The second purpose of law is architectural. It cultivates the judgment, character, and practical wisdom that transform a collection of surviving individuals into a civilization capable of flourishing. Aquinas stated this with characteristic precision: “The purpose of human law is to lead men to virtue, not suddenly, but gradually.”¹³ This is not pious sentiment. It is a structural claim about the relationship between legal institutions and human development. Law develops the capacity for wisdom by creating conditions: structured encounters with difficulty, habitual patterns of action, the expectation of wise deliberation. Maimonides called this purpose tikkun ha-nefesh, the perfection of the soul, the cultivation of understanding. Law’s second purpose is not to replace judgment but to create the conditions in which judgment can grow.

These two purposes are sequential: you build the foundation before you raise the scaffolding. But they are not equal in rank. The foundation exists to make the scaffolding possible. The scaffolding is the point. You do not build a foundation so you can admire the foundation. You build it so the house can rise.

Modern law stopped building. We became obsessed with foundations. We built elaborate structures to prevent every conceivable harm: regulatory agencies to protect consumers, licensing boards to screen practitioners, permitting regimes to control innovation, compliance systems to monitor behavior. We measured success by harms prevented and never asked what we were preventing from existing.

Haddaway’s Law names this amnesia. Baby, don’t hurt me. Law defines itself by what it forbids, protects against, and prevents. It has no vocabulary for what it enables. It cannot articulate what human beings might become if law created space for them to grow rather than merely preventing them from falling.

Law Has Limits

We also forgot something deeper: law has limits. Not merely prudential limits of the kind that pragmatists invoke, but limits built into the nature of law itself.

Law speaks in general terms. But wise action requires judgment about particulars. Law can prohibit murder. It cannot specify in advance what courage looks like in every circumstance. Law can enforce contracts. It cannot tell the entrepreneur which risks are worth taking. Law can protect property. It cannot determine which innovations will serve human needs.

Aristotle drew a fundamental distinction between two modes of practical knowledge: technē, the knowledge of how to produce things, and phronēsis, the knowledge of how to act well.¹⁴ The craftsman who knows how to build a table exercises technē. His knowledge terminates in the table; the table is external to the craftsman; when the table is built, the knowledge has been fully expressed and the task is done. The judge who knows how to sentence wisely exercises phronēsis. Her knowledge does not terminate in the sentence; the sentence is the expression of a capacity cultivated through years of practice, a capacity that is inseparable from the character of the judge herself and that can only develop through the exercise of judgment in actual circumstances.

Technē can be systematized, encoded, and transmitted. A recipe is codified technē. An algorithm is codified technē. The ambition of modern regulatory design is always the ambition to replace slow, variable technē with faster, more consistent technē, to encode best practices and enforce them uniformly. This is sometimes legitimate and even admirable. A hospital’s surgical checklist really is better than idiosyncratic surgeon intuition about sterile technique, because sterile technique is technē: there is a best practice, and it is the same best practice every time.

Phronēsis cannot be systematized without ceasing to be phronēsis. The capacity for practical wisdom in particular circumstances develops only through the practice of discernment itself. It cannot be encoded, because encoding replaces the exercise with its output, and it is the exercise, not the output, that constitutes the capacity being developed. You can encode the conclusion of a phronetic judgment. You cannot encode the sagacity that produced it. The physician who recognizes pneumonia through years of auscultation has developed a perceptual capacity that cannot be transmitted to someone who has only read descriptions of what pneumonia sounds like. The practice is the knowledge, not merely the vehicle for it.

The Chicago School classified professional discretion as technē subject to optimization. But judgment in law’s second function — the Scaffolding function — is phronēsis constituted by exercise. The error is not one of efficiency. It is ontological. The consequences are not suboptimal institutions. They are institutions that systematically destroy the capacity they were built to depend upon.

Deskilling Loop

When an institution replaces judgment with rules, it does not merely eliminate a single discretionary decision. It eliminates the practice that would have developed the capacity for all future acts of discernment in that domain. The deficit, once created, justifies its own extension.

The logic runs as follows. Institutions replace judgment with rules to eliminate variance and to establish what lawyers call a compliance defense: documentary evidence that every decision was made in accordance with applicable protocols. The rules, by eliminating judgment, ensure that the humans operating within the institution do not develop the capacity to handle situations the rules do not address. When those situations arise (and they always do, because the world generates more situations than rules can enumerate), the institution lacks the practical wisdom to handle them. The institutional response is to write more rules. Each additional rule expands coverage at the cost of further judgment development. The coverage expands. The capacity contracts. The institution writes more rules.

I call this the Deskilling Loop, and its operation is visible at every scale of institutional life.

The pilots of Lion Air 610 and Ethiopian 302 were not bad pilots. They were trained pilots operating within an institutional architecture that had systematically removed from their practice the precise skill they needed to survive. MCAS was the Deskilling Loop implemented in software. The FAA’s delegation framework was the Deskilling Loop implemented in procedure. The airlines’ maintenance and training programs, calibrated to the approved type rating and nothing more, completed the circuit. When the sensor failed, the loop closed on 346 lives.

The compliance officer at the major bank is not a bad professional. She is a trained professional operating within an institutional architecture that has replaced the discretion of loan officers, risk managers, and relationship bankers with models, algorithms, and documented compliance processes. The mortgage origination crisis of 2008 was not a failure of fraudulent individuals, though there were fraudulent individuals. It was a systemic failure of institutions that had eliminated the considered assessments of underwriters and replaced them with quantitative models calibrated to a world of stable correlations, then discovered, when correlations dissolved in August 2008, that the practical wisdom they had replaced was the only thing that could have weathered the discontinuity.¹⁵

The legal brief filed by attorney Steven Schwartz in Mata v. Avianca is not the product of a negligent lawyer in the ordinary sense. Schwartz used an artificial intelligence tool to conduct legal research, and the tool produced citations to cases that did not exist.¹⁶ The court’s response (sanctions, public censure, a ruling that became a cautionary tale for the profession) focused, appropriately, on the failure to verify. But the deeper failure was architectural. Reading a case, evaluating its holding, tracing its application across a line of authority, and determining its relevance to particular facts is not a mechanical operation. It is an act of legal discernment, the practice through which legal phronēsis develops. When the frictionless interface of the AI replaces that practice with a polished output, it does not skip a step. It removes the curriculum. The attorney who accepts the output without verification is not simply lazy. The system was designed to invite precisely that acceptance, because the entire logic of the tool is the elimination of the friction that develops judgment.

Tikkun Ha-Guf, Tikkun Ha-Nefesh

The Deskilling Loop is not an accident of institutional design. It is the predictable consequence of a philosophical trajectory that Western law has been traveling since the seventeenth century.

The classical tradition understood law to serve both of its purposes. Aristotle held that legislators make citizens good by forming habits in them.¹⁷ Aquinas held that law must establish general principles while leaving space, determinatio, for those acting in particular circumstances to specify the concrete shape of right action.¹⁸ Maimonides understood that you cannot coerce wisdom: the perfection of the soul depends on the exercise of a faculty that disappears the moment it is replaced rather than cultivated.¹⁹

Then the Wars of Religion destroyed the consensus that had made the second purpose operationally possible. The catastrophe of confessional warfare, the Thirty Years’ War alone killed perhaps a third of the German-speaking population,²⁰ demonstrated with lethal clarity what happens when states optimized for the cultivation of the highest good encounter populations that violently disagree on what the highest good is. The Scaffolding function, in a world of fractured theological consensus, did not produce wisdom. It produced massacre.

The intellectual response was a retreat traced through Grotius, Hobbes, Locke, and Mill. Each thinker narrowed law’s legitimate scope in response to genuine historical catastrophe. Hugo Grotius, writing amid the Dutch Revolt, grounded natural law in sociability rather than theology, securing a minimum grammar of coexistence that Catholics and Protestants could share.²¹ Thomas Hobbes, writing while England was still cooling from its own civil war, eliminated the summum bonum entirely and replaced it with the universal aversion to violent death: law exists not to make men good but to keep them alive.²² John Locke drew a strict line between civil interests and the care of souls, insisting that the state had no legitimate authority over the latter and that coerced religious conformity was not merely ineffective but wrong.²³ John Stuart Mill articulated the principle that would become the operating rule of modern liberalism: the only legitimate basis for state coercion is the prevention of harm to others.²⁴

Each of these moves was responsive to genuine catastrophe. The retreat was not motivated by cynicism. It was motivated by the corpses of those who had died because states that claimed the authority to cultivate souls had exercised that authority with fire and sword.

The problem was not the retreat. The problem was what happened to the retreat once it succeeded. A modus vivendi designed for survival was mistaken for a permanent architecture of political life. The emergency scaffolding was taken as the permanent building. The minimum grammar of coexistence (don’t kill your neighbors, keep your promises, respect their property) became not the floor of legitimate law but its entire scope. Once the scope was fixed, the only way to demonstrate that law was doing its job was to demonstrate that harm was being prevented. Prevention became not merely law’s first function but its exclusive one. Haddaway’s Law followed with the logical inevitability of a corollary.

Architecture of Forgetting

But even the philosophical genealogy does not fully explain how we arrived at the world of the 737 MAX, the compliance industrial complex, the ninety-minute interrogation of the charity-motivated grandmother. The intellectual retreat from law’s second purpose was completed by an institutional architecture that converted that retreat into an enforcement mechanism.

Frank Knight, the Chicago economist whose 1921 monograph Risk, Uncertainty, and Profit drew what should have been an indelible distinction between calculable risk and genuine uncertainty, understood that some situations involve probability distributions you can estimate and some situations involve conditions so novel that no historical distribution offers reliable guidance.²⁵ Risk can be managed by algorithm. Uncertainty requires judgment. The distinction was precise, empirically grounded, and almost immediately ignored by Knight’s intellectual successors.

The financial innovations of the late twentieth century — Value-at-Risk models, the Black-Scholes framework, the collateralized debt obligations rated AAA by agencies that relied on historical correlation data — were built on the systematic conflation of risk with uncertainty.²⁶ Roger Lowenstein, writing about Long-Term Capital Management’s collapse in 1998, identified the core error: the partners had built an architecture for the domain of calculable risk and assumed it covered the domain of genuine uncertainty.²⁷ LTCM had fourteen principals with seventeen Nobel Prize invitations among them. The models were right until the world delivered a situation that had no historical precedent. The models had nothing to say about such situations, because models are calibrated to history.

The administrative state that grew alongside these financial innovations built its own version of the confusion. The Administrative Procedure Act of 1946 established a constitutional covenant for agency rulemaking: publish, invite comment, respond in writing, submit to judicial review.²⁸ The covenant held for roughly three decades. Then the courts, applying what they called the “Hard Look” doctrine, began requiring agencies to demonstrate exhaustive consideration of every relevant factor and every viable alternative.²⁹ Rational administrators responded by asking whether formal rulemaking could be avoided. The answer was yes: guidance documents, Dear Colleague letters, enforcement actions converted into consent decrees, compliance standards embedded in insurance requirements and accreditation conditions. All of this carries the practical weight of law while remaining, in the agency’s own characterization, technically non-binding. The shadow regulatory state is a system for achieving the effect of law without the procedural requirements that make law accountable.³⁰

Steven Teles called the result “kludgeocracy”: a system of accumulated complexity, each layer added to solve an immediate problem, whose aggregate effect is a legal environment that no single person can comprehend.³¹ The Code of Federal Regulations exceeds 185,000 pages. The compliance officer exists because the law has become unintelligible to non-initiates. The consequence is an industry whose product is friction management and whose continued existence depends on the friction remaining unresolved.

This is the architecture of the machine: the institutional form that Haddaway’s Law takes when implemented through the administrative state. The architecture is not the product of malice. It is the product of a logic that is internally coherent and historically intelligible. The tragedy is not that individual actors behaved badly. The tragedy is that the architecture produces bad outcomes reliably, predictably, and with the full cooperation of everyone operating in good faith within it.

What Maimonides Saw

A word on what this book is not. Barry Schwartz and Kenneth Sharpe’s Practical Wisdom and Michael Sandel’s What Money Can’t Buy occupy adjacent terrain, and both make the case for practical wisdom well. This book’s argument is different. It is about the institutional mechanism that destroys wisdom and the architectural alternatives that preserve it. The diagnosis is mechanistic. The prescription is structural. The question is not whether we should want wise physicians, wise judges, and wise bankers. The question is what happens to the institutional machinery when we build systems that depend on judgment and then engineer judgment out.

This book makes four connected claims.

The first is historical. The Western legal tradition understood, until approximately the seventeenth century, that law serves two purposes: foundation and scaffolding, tikkun ha-guf and tikkun ha-nefesh, and that it operates within inherent limits that prevent it from specifying what can only be judged. This understanding was not vague philosophy. It was the operational basis of actual legal systems built by men who also happened to be among the most penetrating thinkers their civilizations produced. We lost this understanding through a process of retreat that was initially intelligible and gradually catastrophic.

The second claim is epistemological. The ambition of modern regulatory design (to eliminate uncertainty through more complete specification, to substitute rule for judgment, to achieve through institutional architecture the safety that wisdom might have provided) is not merely inefficient. It is impossible in principle, because it requires a kind of knowledge that cannot be centralized. Frank Knight’s distinction between risk and uncertainty, Aristotle’s distinction between technē and phronēsis, Michael Polanyi’s account of tacit knowledge, all converge on the same conclusion: there is knowledge that can only be exercised, and institutions that eliminate its exercise also eliminate the capacity to acquire it.³²

The third claim is institutional. The Deskilling Loop is not an accident. It is the predictable consequence of institutional design optimized for Haddaway’s Law. When you build systems that replace judgment with rules, those systems will produce actors who cannot exercise judgment. When those actors encounter situations that rules did not anticipate, and they will always encounter such situations, the institutional response is more rules, which produces more atrophy, which produces more rules. This is not a spiral that eventually finds equilibrium. It is a spiral that terminates in the catastrophic failure of the system that generated it.

The fourth claim is prescriptive. Recovery is possible. Not by returning to the twelfth century (we cannot and should not) and not by eliminating regulation, because foundations matter and some modern problems require modern solutions. Recovery requires something more demanding: rebuilding the institutional conditions under which judgment can be developed and exercised. This means designing friction back into systems that have engineered it out. It means distinguishing between the friction that produces compliance and the friction that produces wisdom. It means recovering the vocabulary of law’s second purpose, not as an abstraction but as an operational commitment with structural implications for how institutions are designed, how professionals are trained, and how law defines what it owes to the people it governs.

The next four chapters trace the intellectual descent: the confusion of risk and uncertainty that produced the financial architecture of the past half-century; the great retreat of early modern philosophy that eliminated law’s second purpose in response to genuine catastrophe; the moment when prudence became a transaction cost and compliance replaced judgment as the professional ideal; and the administrative architecture that converted these philosophical commitments into the regulatory state as we currently inhabit it. The chapters that follow examine what the descent has cost us: in occupational licensing and the dignity of work; in criminal justice and the possibility of restoration; in drug regulation and the invisible graveyard of lives not saved. The book concludes with an argument for recovery, not a program but a framework, the terms on which institutions that have forgotten law’s second purpose might begin to remember it.

Haddaway Was Wrong

Return to that 1993 song. What is love? Baby, don’t hurt me.

The answer is insufficient. Love is not merely the absence of harm. It is presence, attention, commitment, sacrifice, and joy. You cannot define it by listing what it is not, any more than you can define light by cataloguing everything that is not light. Haddaway’s question deserved a better answer than Haddaway could give it, and the song’s pathos derives precisely from the gap between the magnitude of the question and the poverty of the response.

So with law. You cannot define law by what it prevents any more than you can define love by what it does not do. Law is not merely the absence of chaos. It is the presence of order, the structure that enables cooperation, the framework within which human beings can pursue their own flourishing in ways that serve the flourishing of others. It is, as Aristotle understood, the institution through which political communities work out, over time and through lived experience, what it means to live well together.

Haddaway’s answer was not wrong. It was incomplete. Love should not hurt. Law should prevent harm. But prevention is not purpose. It is precondition. You clear the ground so you can build. You establish foundations so the scaffolding can rise. You prevent the worst so the best becomes possible. The foundation is necessary. The scaffolding is the point.

Modern law has forgotten this. We built institutional architecture around Haddaway’s answer, and the architecture has become so elaborate, so pervasive, and so internally self-reinforcing that it has acquired the appearance of inevitability. It is not inevitable. It is a choice made by intelligible actors in intelligible circumstances for intelligible reasons that produced consequences none of those actors intended and that no single actor can reverse.

This book is an argument that the choice can be made differently. That the framework exists to make it differently. That the cost of not making it differently is measured in the 346 dead at Bishoftu and the Java Sea, in the grandmother at the bank counter, in the compliance officer who has never learned to lend and the attorney who has never learned to read a case and the administrator who has never learned to govern. The cost is measured in a civilization that has optimized for the prevention of hurt and has thereby prevented itself from building anything worth protecting.

The next chapter begins the account of how we arrived here.

Chapter 2: The Confusion

The ambition to prevent all harm has a financial corollary. Before anyone can build a machine to eliminate risk from institutions, someone has to believe risk can be eliminated from markets. In the summer of 1998, that belief was tested.

A Wager

The problem that destroyed Long-Term Capital Management was not a problem its founders could predict. That is, precisely, the problem.

In late August 1998, the partners of Long-Term Capital Management were watching something their models said could not happen. Dozens of positions (each calibrated to historical relationships between asset classes that had held for years, in some cases for decades) were moving against the fund simultaneously. The correlations the models treated as stable properties of the financial world were dissolving in real time, all at once. By the end of August, the fund had lost $1.9 billion. By mid-September, it had lost $4.4 billion more. On September 23, William McDonough, president of the Federal Reserve Bank of New York, convened a meeting of fourteen major financial institutions and told them that if LTCM failed in an uncontrolled unwinding, the consequences for global credit markets were unpredictable. The fourteen institutions contributed a combined $3.6 billion to take over the fund and wind it down in an orderly fashion.³³ The most sophisticated risk management apparatus in the history of finance had just required a federal rescue to prevent a cascade no one could fully anticipate.

To understand how this happened, understand what the fund was built to do.

In 1994, John Meriwether assembled what was arguably the most credentialed team in the history of American finance. The fund’s principals included Myron Scholes and Robert Merton, who would share the Nobel Memorial Prize in Economic Sciences three years later, awarded specifically for their work on the mathematical pricing of derivative securities. LTCM was financial theory made liquid: a demonstration that rigorous mathematics could identify and exploit mispricings in global fixed-income markets, converting the apparent randomness of bond spreads into a reliable, calculable stream of return. Where other funds employed financial theorists, LTCM was built to embody their conclusions. The fund also counted among its partners David Mullins, the former vice chairman of the Federal Reserve, who had helped design the regulatory architecture of global finance and now proposed to profit from its inefficiencies. The partners of Long-Term Capital Management had access to more sophisticated analytical machinery, more historical data, and more accumulated intellectual capital than any comparable enterprise had ever assembled.

For four years, it worked. By 1997, LTCM had generated annualized returns exceeding forty percent. Assets under management had grown to roughly $126 billion, supported by equity of approximately $5 billion: a leverage ratio approaching 25-to-1. The number expressed confidence as mathematics. The fund’s models indicated the risk was bounded. History supported this reading. The models had been right.

The models had been measuring the wrong thing.

In August 1998, Russia defaulted on its domestic sovereign debt. The catastrophe was not the Russian default itself but what happened in markets with no direct exposure to Russian bonds. Investors globally shed risk simultaneously, and the correlations between asset classes (the historical relationships that LTCM’s models had treated as permanent features of market structure) dissolved. Assets that were supposed to move independently moved together. Spreads that were supposed to converge diverged instead, and then kept diverging. The fund’s convergence trades, each individually grounded in observed historical relationships, collapsed in concert because the historical relationships had been a property of normal times, and these were no longer normal times.

LTCM lost $4.6 billion in less than four months. In a spectacular reversal, the most sophisticated risk management apparatus ever assembled, operated by men who had received the highest formal recognition the economics profession can bestow, failed in a way that threatened not merely the fund’s investors but the broader system in which the fund was embedded.

Roger Lowenstein, whose account of the collapse remains the essential text, identified the core error with surgical economy: the professors had confused the measurable with the manageable.³⁴ They had built an architecture for the first and assumed it covered the second. This confusion (between the domain of calculation and the domain of judgment) is the subject of this chapter, and the deeper subject of this book.

The Prophet Chicago Ignored

To understand what Merton and Scholes got wrong, it helps to understand what their predecessor got right.

Frank Knight arrived at the University of Chicago in 1917 and published, four years later, a doctoral dissertation refined into a monograph that would define the conceptual vocabulary of economic risk for the next century. Risk, Uncertainty, and Profit drew a distinction that was simultaneously simple in its formulation and radical in its implications.³⁵

Risk is a situation in which you face multiple possible outcomes but know, or can estimate, the probability distribution governing those outcomes. Rolling a die is risk. The faces are fixed; the probabilities are known. An insurance company facing a large portfolio of car accidents is in the domain of risk: individual outcomes are unpredictable, but aggregate outcomes are stable enough to price. The underlying structure is stable.

Uncertainty is something else entirely: the condition in which you face outcomes whose probability distribution you cannot specify because the future involves genuinely novel situations, structural breaks, innovations, and events that have no historical precedent offering reliable guidance. The entrepreneur deciding whether to build a factory for a product that does not yet exist faces uncertainty. The general anticipating a new form of warfare faces uncertainty. The regulator designing oversight for a technology she does not fully understand faces uncertainty. In Knight’s taxonomy, these actors cannot calculate their way to the right answer. They must exercise judgment: they must make decisions under conditions where no algorithm, however sophisticated, can substitute for the quality of the decision-maker’s mind.

The distinction mattered because it mapped directly onto the theory of profit. In a world of pure risk, Knight argued, competition would eventually eliminate excess returns: if an advantage is calculable, it is replicable, and replication drives prices to equilibrium. Genuine, sustained, above-normal profit requires something that cannot be calculated and replicated. It requires successful judgment under genuine uncertainty. Entrepreneurial judgment is the capacity that generates the returns pure theory cannot explain.³⁶

Knight’s framework was a claim about epistemic humility. Economists’ tools are powerful within their domain (risk, stable probability distributions, repeated games) and powerless outside it. To forget this boundary is to mistake the map for the territory and make large bets accordingly.

Frank Knight’s Chicago was a Chicago of disciplined limits. Mathematical machinery was sharp within its domain. To know where the domain ended was the task of serious thinking.

What Coase Actually Said

Knight’s Chicago respected the boundary between the calculable and the unknowable. The next generation erased it.

In 1970, Eugene Fama published a paper that conventionally dates the shift: “Efficient Capital Markets: A Review of Theory and Empirical Work,” in the Journal of Finance.³⁷ Fama’s Efficient Market Hypothesis argued that asset prices, in a well-functioning market, rapidly incorporate all available information. Prices are not perfect (they are, in Fama’s careful formulation, efficient with respect to available information, meaning that no investor can systematically earn above-normal returns by trading on public data). Markets are not prescient; they are, as economists say, informationally efficient.

The hypothesis was sophisticated, carefully hedged, empirically grounded, and enormously consequential in ways its author did not fully intend.

The problem was not the hypothesis but its migration. In Fama’s original formulation, EMH was descriptive: it claimed that prices in liquid markets rapidly incorporate available information. In the decades that followed, it became prescriptive: a framework for treating financial risk as calculable and manageable. The drift was gradual, understandable, and institutionally catastrophic.

If markets are efficient, price movements are essentially random, but in a specific technical sense: random like a fair coin, with a known distribution governing the noise. This allowed financial economists to apply probability theory to asset prices, generating option pricing models, portfolio theory, and risk management frameworks that became modern finance’s infrastructure. The mathematics succeeded in the domain it described: a world where outcome distributions were stable, where historical patterns reflected structural features that would persist, where the future resembled the past in the ways that mattered most.

Knight’s distinction was forgotten rather than refuted. Modern finance’s intellectual machinery was built for risk and then quietly deployed against uncertainty, without acknowledging the shift. When Merton and Scholes applied their tools to global bond markets in 1994, they were making a sophisticated mistake: the kind that generates four years of corroborating evidence before it destroys you.

In 2004, the Basel Committee on Banking Supervision institutionalized the mistake, incorporating Value-at-Risk models into the capital adequacy requirements that governed bank balance sheets globally.³⁸ Value-at-Risk, or VaR in the industry’s shorthand, is a specific answer to a specific question: what is the maximum loss, at a given confidence level, over a given holding period, within the distribution suggested by historical price data? It is a useful tool for measuring the risk that resides within a stable probability distribution. It is systematically silent about the risks that reside outside such distributions (the events that are too rare to appear in historical data, or that represent structural breaks in the patterns history records).

VaR told risk managers what the worst normal day looked like. It had nothing to say about abnormal days. The Basel frameworks, which governed capital requirements for the world’s major financial institutions, were calibrated to a measure incapable of detecting the risks most needing detection. Regulators built this framework not from ignorance but because VaR was mathematically tractable, auditable, and comparable—three properties institutional risk management requires and better tail-risk measures do not provide. The manageable drove out the accurate. Knight’s distinction between risk and uncertainty became an operational inconvenience.

Ludic Fallacy

Nassim Nicholas Taleb named the confusion.

The Ludic Fallacy, as Taleb defined it in The Black Swan (2007), is the error of treating the complex, non-repeating, structurally evolving world as a structured game with fixed rules and known probability distribution.³⁹ Ludus is Latin for game; the Ludic Fallacy is the mistake of building frameworks suited to games and applying them to situations that are emphatically not games.

A casino exemplifies the calculable world. The house sets the rules, payoffs are fixed, probabilities computed from first principles, and the law of large numbers converts individual randomness into predictable aggregates. In the long run, the casino wins by a known margin, and a competent statistician can state that margin precisely before play begins. Financial models were built for this world.

But the world is, in Taleb’s phrasing, more like a forest: structurally open, where novel organisms appear, where interaction rules change over time, where critical events have never happened before and appear nowhere in historical frequency tables. The normal distribution fails catastrophically in systems that generate large, discontinuous, structurally novel events. Taleb called these Black Swans: events of large magnitude, radical rarity, and retrospective inevitability (they seemed inevitable in hindsight but were invisible within probability frameworks in use before they occurred).

The mathematical error deserves scrutiny. Financial models assuming normally distributed returns are calibrated to a distribution whose tails fall off rapidly: extreme events get vanishingly small probabilities. But observed financial returns do not follow a normal distribution. Their tails are “fat”: extreme events occur far more frequently than the normal distribution predicts.⁴⁰ A model calibrated to normal distribution assigns correct probabilities to eighty or ninety percent of ordinary days and systematically underestimates ten or twenty percent of extraordinary days. This is backwards from what risk management needs.

As an institutional disposition, the Ludic Fallacy is as significant as a mathematical error: the preference for quantified, auditable risk estimates over acknowledgments of genuine uncertainty. The Ludic Fallacy persists not from ignorance but because acknowledging it produces no workable replacement. Knight’s uncertainty is real but institutionally unusable. An audit requires a number. A capital adequacy requirement requires a calculation. A board requires a report. Acknowledging that some risks cannot be quantified without systematic distortion satisfies none of these institutional requirements. The Ludic Fallacy is the product of infrastructure that rewards quantified answers and has no mechanism for receiving honest expressions of ignorance: a failure of design, not individuals.

This is why Taleb’s diagnosis cuts deeper than a methodological critique. Institutions built to govern by models cannot hear the argument against models. The argument arrives in a form the institution cannot process.

Turkey Problem

Taleb captured the problem most economically in a stark thought experiment.

Consider a turkey.⁴¹ For a thousand days, the turkey is fed and sheltered. Each day adds evidence to the turkey’s inductive case for the benevolence of the farming arrangement. By day 999, the turkey’s confidence in its security has never been higher. The historical record supports this confidence comprehensively. The pattern is unbroken. The data is unambiguous.

Day 1,001 is the day before Thanksgiving.

Inductive reasoning from stable historical records fails catastrophically when those records were generated under conditions about to change structurally. The Turkey Problem reveals why confidence peaks at maximum peril. The turkey’s confidence is highest when its circumstances are most dangerous: because the farm’s apparent beneficence was always a function of conditions (the turkey’s immaturity, the farmer’s timing, the approaching holiday) that the turkey could not observe and its data could not capture.

Translate this to institutions. A regulatory framework optimized for the historical distribution of observable risks will fail when the structural conditions generating that distribution change. Its confidence will be highest just before the break. Its capital buffers, calibrated to normal-distribution tail estimates, will be sized for the wrong catastrophe. Its models will generate reassuring outputs until the reassurance terminates abruptly.

In 2008, the financial crisis exemplified, among other things, a Turkey Problem at institutional scale. The banks, the rating agencies, and the regulatory bodies that governed them had accumulated years of evidence that the framework was working. Defaults were rare; correlations between housing markets in different regions were low; the models were performing. The Basel VaR requirements were producing the capital cushions they were designed to produce. The records were clean. And then the conditions that had generated the records changed: mortgage origination standards deteriorated across the industry simultaneously, regional housing markets began moving together, the instruments that had been calibrated against historical default rates encountered default rates with no historical precedent, and the entire architecture discovered, in short order, that it had been measuring the wrong thing with extraordinary precision.⁴²

Taleb’s broader framework explains what the Turkey Problem alone does not. In Antifragile (2012), Taleb distinguishes three responses to volatility.⁴³ Fragile systems are harmed by volatility (they lose more from disorder than they gain, and their ideal condition is stasis). Robust systems are neutral to volatility (they neither gain nor lose, merely surviving). Antifragile systems gain from volatility (they improve under stress, extract information from disorder, emerge from turbulence better calibrated than they entered it).

Most living systems are antifragile in some respects. Biology offers the mechanism: hormesis, the phenomenon by which organisms exposed to sub-lethal stressors become more resistant. The human immune system does not merely survive pathogens; it requires exposure to build competence. Bones stressed by weight-bearing grow denser. Muscles under load generate the micro-damage that triggers growth. The antifragile system must encounter controlled disorder or forfeit the capacity to handle uncontrolled disorder.

A system designed to eliminate all volatility does not become more robust. It becomes more fragile. The suppression of small, recoverable disturbances eliminates the feedback that allows adaptive systems to calibrate themselves and accumulates the invisible fragility that will discharge catastrophically when a disturbance arrives that is too large to suppress.

Safety machines are fragile in Taleb’s technical sense. By suppressing small failures that would reveal structural weaknesses, they produce a profile that is tranquil until it is catastrophic. The VaR framework did not eliminate unmeasurable risks; it concealed them until they had accumulated beyond hiding. The turkey is the canonical image, but the mechanism is general: the institution optimized for minimum observable failure is maximally vulnerable to failure it cannot observe.

This is a claim about institutional design, not individual malice or incompetence. The architects of the Basel frameworks were solving a real problem: how to make capital adequacy auditable, comparable, and enforceable across thousands of institutions in dozens of jurisdictions. They solved it with available tools—tools calibrated for risk and deployed against uncertainty. The distinction between these domains was not, in 2004, obscure. Knight had made it in 1921. But institutional infrastructure runs on tractable quantities, not philosophical points.

The Lindy Effect explains why the confusion self-perpetuates. for non-perishable things (ideas, institutions, laws, practices), expected remaining lifespan is roughly proportional to current age.⁴⁴ A tradition surviving a thousand years is likely to survive another thousand. A five-year-old technology may or may not persist. The Lindy Effect is a statistical claim about what long survival implies: not nostalgia dressed as probability, but that a thing persisting through centuries of turbulence carries embedded information about robustness that recent competitors, however theoretically sophisticated, do not yet possess.

Measured in decades, VaR, the Basel requirements, and quantitative risk management faced a common law tradition measured in centuries—the very tradition they were asked to complement and in some domains to replace. The Lindy Effect does not say common law is correct on any given question. It says an institution tested against centuries of novel conditions and still standing carries information about survival that a framework calibrated to twenty years of data cannot replicate. Discarding the older tradition for the newer model is a wager whose odds have not been carefully calculated.

Metastasis

The confusion between risk and uncertainty began in finance. It did not stay there.

The confusion entered legal and regulatory institutions through the conviction that measurable outcomes were the right outcomes to optimize, that unquantifiable judgment was a residue of pre-scientific thinking, and that the appropriate response to a governance problem was a well-specified model.

Before examining what the replacement produced, take seriously why it seemed necessary. Unstructured professional discretion had a documented record of failure that judgment’s defenders could not easily dismiss. Sentencing research in the 1970s and 1980s revealed enormous disparities: defendants convicted of identical offenses received sentences varying by years, with variance correlating to factors (appearance, judge’s fatigue, order of cases) no defensible theory of justice could sanction. Parole boards exercised unreviewable discretion that studies found barely superior to chance.⁴⁵ The case for structured instruments was initially as much a civil rights argument as an efficiency argument: an attempt to bring human decision-making’s arbitrariness under discipline of consistent, reviewable standards. The impulse was legitimate. The instrument it produced was not.

Algorithms arrived to reshape governance, presented as immune to human bias. Predictive models were offered as more reliable than individual officer assessments. Risk scores replaced parole board deliberations. Actuarial instruments displaced clinical judgments. The argument was consistent: the model is transparent, auditable, consistent; the human is opaque, unaccountable, variable.

The argument contained genuine insight and concealed catastrophic error.

Human judgment is susceptible to identifiable, systematic biases. Anchoring, availability heuristics, racial stereotyping, and dozens of cognitive distortions have been documented rigorously enough to indict unstructured discretion.⁴⁶ A sentencing judge imposing harsher sentences before lunch than after is a malfunctioning instrument by any reasonable standard, and a tool smoothing this variability serves a legitimate goal.⁴⁷

The model’s apparent objectivity is a property of outputs, not inputs. A model trained on historical data inherits embedded biases. A model calibrated to predict recidivism in a population where law enforcement has systematically over-surveilled certain communities encodes those surveillance patterns as predictive signals. The model observes not an objective world but a historical record itself the product of human decisions. It encodes human bias as mathematical fact. The algorithm launders the bias, transforming it from something a court can recognize and interrogate into something presenting as mathematics.

ProPublica’s 2016 investigation of COMPAS (Correctional Offender Management Profiling for Alternative Sanctions, deployed in sentencing across dozens of jurisdictions) documented this laundering with unambiguous specificity.⁴⁸ Among 7,000 Broward County defendants, COMPAS’s risk scores were racially asymmetric in error rates: Black defendants who did not reoffend were labeled high-risk at nearly twice the rate of white defendants with identical subsequent conduct. The inverse error was equally pronounced. The algorithm was differently inaccurate than a human judge. Its inaccuracy was encoded in a format substantially harder for the legal system to challenge than a human opinion.

Beyond epistemic concerns, the challenge is doctrinal. A defendant questioning a judge’s reasoning may, under most procedural frameworks, demand that reasoning be articulated. A defendant questioning an algorithm’s output faces a harder problem: the algorithm’s weights and training data are often proprietary, the mathematical relationship between inputs and outputs is technically opaque, and the system producing the score is in most jurisdictions not subject to the same scrutiny as human testimony. The model is, in the legal system’s own vocabulary, harder to cross-examine than a person.⁴⁹

The child welfare system followed. In 2016, Allegheny County, Pennsylvania deployed the Allegheny Family Screening Tool: a predictive instrument generating a 1-20 score based on families’ interactions with public service systems, where higher scores predict increased child abuse or neglect likelihood.⁵⁰ The algorithm synthesizes information about prior child welfare involvement, mental health treatment, housing assistance, and other public records. The output is a number influencing decisions about investigation, removal, or services.

Virginia Eubanks identified the structural problem that makes the Allegheny instrument a case study in confusion rather than progress.⁵¹ The algorithm draws on data about public service usage only. Administrative data reveals a fundamental bias. Families with no public service history (because they are affluent enough to purchase services privately) leave no data trail and score lower risk by default. Families relying on public services are more extensively documented and scored accordingly. The instrument is not measuring child welfare risk; it is measuring the density of administrative surveillance. The two are correlated, but the correlation reflects policy choices about who gets surveilled, not neutral discovery about danger to children.

A human caseworker can account for the difference between a family whose history appears in public records because they sought help and a family whose history does not appear because they never needed to. The algorithm cannot make this distinction; the distinction is not in its data. Replacing case-specific discernment with a procedure that is auditable, consistent, and systematically blind to the most important variable ensures the algorithm’s inadequacy.

This is the signature of the metastasis. Because it drove the Basel Committee to adopt VaR, the same institutional logic also drove the courts to adopt COMPAS and the child welfare system to adopt the Allegheny instrument. In each case, the legal system adopted algorithmic tools not because they were better but because they were measurable; they produced outputs that could be audited, defended in court, compared across jurisdictions, and reported to oversight bodies. The manageable drove out the accurate. The confusion between risk and uncertainty migrated from the bond trading floor to the sentencing hearing to the caseworker’s decision.

VaR, COMPAS, and the Allegheny instrument are all risk tools in the Knightian sense: they measure observable historical patterns and extrapolate. None handles novel situations: the defendant with no historical analog, the family whose risk profile matches no cluster in the training data, the market movement outside every calibrated distribution.

None of them knows what it does not know. This structural silence is constitutive of tools that measure risk in the Knightian sense. It cannot be engineered away, because it is what such tools are.

Knight Was Right

The failure was predictable from inception and haunts financial and legal history. Knight predicted it in 1921. Taleb diagnosed its mechanism precisely enough that readers of The Black Swan had the conceptual tools to anticipate 2008. ProPublica’s investigation of COMPAS documented what the framework produced in criminal sentencing, and that documentation has been in the scholarly record for a decade. Practitioners are largely aware of the critique. They have known it for some time.

The question that awareness cannot answer is: why do the instruments persist?

They persist because the alternative is not available.

The alternative to algorithmic sentencing tools is a judge’s discretion. The alternative to predictive risk scores is a caseworker’s assessment. The alternative to VaR is honest acknowledgment that certain risks cannot be quantified—an acknowledgment no regulatory framework, capital requirement, or board report can accommodate. The confusion persists because institutions cannot function on the honest answer. Staff are largely aware of the limitations. It makes no difference.

An institution saying “we do not know” performs no recognizable function. It protects no one, satisfies no accountability requirement, generates no auditable record. Deeper than any particular tool, the Safety Machine’s commitment to producing defensible outputs demands a number.

Models mistaking risk for uncertainty will always fail eventually. This is a theorem, not a finding. Far more interesting is why institutions remain committed, structurally and apparently without exit, to the proposition that governance problems can be solved by sufficient measurement. Why did replacing discretion with calculation seem (and to many thoughtful people still seems) not like a choice but like inevitability, the natural direction of progress?

This is a question about the history of an idea: how the function of law and governance came to be understood, over centuries of intellectual development, in a way that made the Safety Machine seem not merely desirable but necessary. It is a question about what institutions decided they were for—what goals were legitimate, what methods appropriate, what outcomes they could pursue—and how those decisions foreclosed alternatives.

Answering it requires going further back than LTCM, further back than the Chicago School, further back than the statistical tools developed by financial economists in the twentieth century. It requires returning to the moment when the foundational idea was decided: when architects of modern political and legal thought chose, under genuine pressure and with genuine reasons, to build a certain kind of institution rather than a different kind.

The choice felt like necessity. It always does.

Chapter 3: The Great Retreat

Every philosopher in this chapter was trying to stop people from killing each other. They succeeded. What their success cost was law’s second function: the cultivation of judgment itself.

Summum Bonum

Between 1524 and 1648, Europe tested what happens when rival communities each possess what they believe to be the summum bonum: the highest good, which each understands itself authorized and obligated to enforce. The test was not theoretical. The Thirty Years’ War killed approximately one-third of the German-speaking population.⁵² The French Wars of Religion, the English Civil War, the Dutch Revolt: each demonstrated a foundational legal idea in catastrophic failure mode.

For roughly two thousand years, the Western legal tradition understood law to serve two functions. The first was foundational: suppression of violence, security of persons and property, and basic social ordering that makes everything else possible. The second was architectural: cultivation of judgment, character, and practical wisdom that transform surviving individuals into a civilization capable of flourishing.

These two functions were not equal in rank, though sequential in time. The Foundation came first because it had to. You cannot teach prudence to someone being stabbed. Physical security is the precondition for everything else. But it was understood as a precondition, not an end. The Scaffolding (law’s capacity to form citizens capable of governing themselves wisely) was the point. The Foundation made the Scaffolding possible. The Scaffolding made the Foundation worth having.

Thomas Aquinas stated this with characteristic precision: “The purpose of human law is to lead men to virtue, not suddenly, but gradually.”⁵³ The claim carries structural weight: law exists to cultivate wisdom, not merely suppress violence. The political order, in the Thomistic-Aristotelian synthesis that dominated Western legal thought from the thirteenth century forward, exists to make wisdom possible—not replace it, simulate it, or insure against its absence, but cultivate it. Medieval jurists understood legislation as pedagogy at scale. Good law operates as a good teacher does: through structured friction, graduated difficulty, and the expectation that capacity will eventually be internalized.

Aristotle made the underlying argument in the Nicomachean Ethics. Legislators make citizens good by forming habits in them, he wrote, and this is every legislator’s wish.⁵⁴ The claim was not that law could manufacture virtue by fiat but that law could create conditions where virtue develops: habitual action patterns, structured encounters with difficulty. The state was a gymnasium: an institution designed to exercise human capacities by subjecting them to calibrated resistance.

The Thomistic synthesis connected individual virtue to political order to divine purpose. Temporal peace maintained by the king was not the final word but the platform on which wisdom’s cultivation could occur. The state exists to enable what Maimonides, writing from a different but closely related tradition in the same century, called tikkun ha-nefesh (the perfection of the soul) by first securing tikkun ha-guf (the perfection of the body).⁵⁵ Prior in time but subordinate in rank: law’s physical infrastructure precedes character cultivation.

This was the baseline. The question this chapter must answer is: what happened to it?

Catastrophe

The perfectionist state requires agreement about what flourishing means. When Catholics and Lutherans and Calvinists agreed on the basic content of the good, the Scaffolding function was merely pedagogically demanding. When they disagreed, it became a machine for civil war. Law that exists to promote the true religion, applied in a society that has radically fractured over what the true religion is, does not produce virtue. It produces massacre.

The Thomistic framework, constructed for a world in which theological consensus provided the shared vocabulary for defining human flourishing, proved vulnerable once that consensus fractured. The Reformation destroyed it with a thoroughness that no subsequent ecumenical movement has fully reversed. The tradition’s vulnerability was circumstantial, not philosophical: its operating conditions had been destroyed beneath it.

Early modern philosophers, responding to this collapse like physicians trying to stop the bleeding, produced an intellectual architecture that solved the immediate problem with a thoroughness that created a different, slower, less visible problem downstream. That architecture still operates. It is the Safety Machine’s foundation.

Grotius’s Modus Vivendi

Hugo Grotius was a Dutch jurist writing in the first decades of the seventeenth century, amid the Dutch Revolt and the Thirty Years’ War’s opening salvos. He understood what was at stake as well as any man alive. His task was to find a basis for binding legal obligations that could survive the fracture of Christian unity: something Catholics and Protestants, Dutch and Spanish, could acknowledge without first agreeing on which Pope was legitimate or which Scripture reading was authoritative.

His solution was methodological audacity. In the prolegomena to De Jure Belli ac Pacis, published in 1625, Grotius asserted that natural law’s foundational principles (keep your promises, do not take what is not yours) would retain their validity “even if we should concede that there is no God, or that the affairs of men are of no concern to Him.”⁵⁶ The etiamsi daremus principle, as it became known: law grounded not in theology but in human sociability’s nature. We are creatures designed for association. The norms making association possible are therefore binding on us whether or not God exists to enforce them.

Grotius was a devout Christian who spent much of his career working for Protestant-Catholic reunification.⁵⁷ The etiamsi daremus was a methodological stratagem: a way to construct a minimum legal grammar that conflicting parties could share without resolving theological differences. You can agree to keep your promises even if you disagree about why keeping promises is obligatory.

Grotius offered, in later philosophical language, a modus vivendi: a working arrangement, a way to stop the killing without requiring the restoration of unity the killing had shown was no longer available. He was saving what he could. The evidence that he regarded this as saving and not celebrating is unmistakable: he longed for Christian unity, wrote extensively toward its recovery, and viewed Christendom’s fracture as a tragedy to be healed.⁵⁸ The minimalist natural law was not his ideal. It was his emergency measure.

Grotius accepted the retreat from the Scaffolding function because he had to. He understood it as a retreat. He did not claim it as a victory.

Hobbes’s Inversion

Thomas Hobbes made the truly radical move.

Where Grotius sought a common denominator within the tradition (rescuing what was possible from the wreckage), Hobbes decided the tradition was the wreckage. The perfectionist state, in his analysis, was not an ideal corrupted by theological conflict but the cause of the conflict. Attempting to use state power to enforce the summum bonum was inherently, inevitably, a recipe for civil war, because there is no summum bonum. The idea is a philosophical error, and an expensive one.

Hobbes was explicit. In Leviathan, published in 1651 while England still recovered from its civil war, he attacked the “old moral philosophers” directly: “there is no such Finis Ultimus [utmost aim] nor Summum Bonum [greatest good] as is spoken of in the books of the old moral philosophers.”⁵⁹ Good and evil are not properties of the world. They are names we give to our appetites and aversions. Since appetites vary from person to person and change within the same person over time, no single account of the good can ground law. Felicity, Hobbes wrote, is “a continual progress of the desire, from one object to another.”⁶⁰ It does not terminate in beatitude. It terminates in death.

Hobbes offered not another conception of the good but a single, universal aversion: the fear of violent death. This, he argued, is the one thing all human beings share regardless of religion, philosophy, or class. Whatever else we disagree about, we agree we do not want to be killed. From this minimal shared premise, Hobbes constructed the entire architecture of the modern state.

From this premise, the political implications become stark. If the only state function is to prevent the summum malum—the greatest evil, which is the state of nature’s “war of all against all”—then the Scaffolding function disappears entirely. Law becomes the command of whoever holds sufficient force to enforce it. Auctoritas, non veritas facit legem: authority, not truth, makes law.⁶¹ The law is not a teacher. It is a guard. Its job is keeping people from killing each other. What they do with the resulting peace is their own affair.

This is a profound inversion of the original hierarchy. In Thomistic synthesis, the Foundation (physical security) was prior in time but subordinate in rank. The Scaffolding (cultivation of wisdom) was the point. In Hobbes, the hierarchy is reversed: the Foundation becomes the entire project. Safety is not the platform for wisdom. Safety is the goal. The scaffold is not what you build so you can build the building. The scaffold is the building.

He was satisfied with it. He did not view the neutral secular state as a second-best compromise with a fragmented world but as the only rational structure for any society in any world, because the perfectionist alternative was a conceptual error. Grotius regretted the loss of Christian unity. Hobbes regarded Christian unity as the source of the problem. The shift from the modus vivendi to the intrinsic ideal changed everything that followed.

Notice what has happened to the Scaffolding. In Grotius, it survives in attenuated form: natural law still cultivates something like civic sociability, even if it can no longer reach all the way to divine beatitude. In Hobbes, the Scaffolding has been taken down entirely. Law has no interest in what kind of people you are, only in whether you are killing your neighbors. The cultivation of discernment, character, and practical wisdom has been privatized (handed off to families, churches, and civil associations) and law has been relieved of any responsibility for it.

The logic holds. If you believe that attempting to use law to cultivate virtue caused the Wars of Religion, then removing that function looks like the rational response. And the relief is real: removing the Scaffolding function does prevent a particular catastrophe. The catastrophe it enables is different and slower to arrive.

What Locke Moralized Away

John Locke completed the work Grotius began and Hobbes radicalized, but in a different register. Where Hobbes was a rationalist rejecting the Scaffolding function because he thought it philosophically incoherent, Locke was a Christian rejecting it because he thought it theologically impermissible.

The distinction matters. Grotius’s retreat was a concession to necessity. Hobbes’s retreat was a philosophical conclusion. Locke’s retreat was a moral duty. By the time Locke finished, the question was no longer whether the state should refrain from cultivating souls. The question was whether a state that tried to cultivate souls was violating a sacred obligation.

The argument, laid out in the Letter Concerning Toleration of 1689, proceeds in two steps. First, Locke draws a strict line between civil interests and the care of souls. Civil interests (“life, liberty, health, and indolency of body; and the possession of outward things”) are the magistrate’s jurisdiction.⁶² The care of souls is not. It is not committed to the civil magistrate by God. It is not consented to by individuals entering civil society. It falls outside the legitimate scope of state power.

Second, and more importantly for what follows, Locke argues that even if the magistrate could coerce religious conformity, he would be causing harm rather than preventing it. “True and saving religion,” Locke writes, “consists in the inward persuasion of the mind.”⁶³ You cannot force someone to believe something. You can force them to perform belief’s outward motions, but coerced performance is not faith; it is hypocrisy, which is a sin. The magistrate trying to save your soul through compulsion is not being paternalistic. He is being cruel. He is leading you into damnation while believing he is leading you toward salvation.

The epistemological corollary follows naturally. If the magistrate cannot compel genuine faith, it is partly because faith resists compulsion and partly because the magistrate has no special access to the truth about salvation. “The one only narrow way which leads to heaven,” Locke observes, “is not better known to the magistrate than to private persons.”⁶⁴ The king has no theological authority that the ordinary believer lacks. His expertise is civil governance. He is not qualified to administer the Scaffolding function, and claiming to do so is an act of presumption that endangers rather than advances the souls in his care.

This is a crucial move. Grotius’s argument for neutrality was pragmatic: we cannot agree on the good, so law must be grounded elsewhere. Hobbes’s argument was metaphysical: there is no good to agree on. Locke’s argument is moral and theological: the state’s attempt to cultivate souls is an act of usurpation violating both faith’s nature and the limits of human epistemic authority. Where Grotius conceded, Locke corrected. The retreat had become an obligation.

By transforming the retreat into a moral imperative, Locke hardened it considerably. A modus vivendi can, in principle, be revisited when circumstances change. A philosophical conclusion can be challenged by better philosophy. But a theological duty (a claim that the state’s attempt to cultivate virtue is an act of impermissible overreach) resists revision in ways that a pragmatic arrangement or a philosophical argument does not.

Baby, Don’t Hurt Me (1859)

Haddaway’s four-word plea became a dance-floor anthem in 1993. John Stuart Mill had already given it the force of law in 1859. The harm principle is “don’t hurt me” rendered as political philosophy: the only legitimate reason for society to exercise power over an individual is to prevent harm to others. Everything else—your character, your choices, your soul—is your own affair.

If Locke supplied the theological argument for neutrality, Mill supplied its secular operationalization. Writing in 1859 in a world reorganized by the American and French revolutions and the early industrialism those revolutions had unleashed, Mill posed the governing question directly: when is the state entitled to coerce its citizens?

The answer he gave was crisp and became, in time, almost definitional for the liberal tradition: “the only purpose for which power can be rightfully exercised over any member of a civilized community, against his will, is to prevent harm to others.”⁶⁵ His own good, physical or moral, is not sufficient warrant. Over himself, over his own body and mind, the individual is sovereign.

This is the Harm Principle, and it completed the structural work that Grotius, Hobbes, and Locke had begun. Where earlier thinkers had argued that the state should not cultivate virtue, Mill argued that the state could not cultivate virtue without violating something sacred. The harm principle establishes a categorical limit on coercion: not a prudent guideline about when intervention tends to go well, but a boundary whose violation constitutes tyranny regardless of intent. The state’s entire authority rests on the mandate to prevent harm to others. Anything beyond that mandate is tyranny, however benevolent.

For the Scaffolding function, the consequence is complete. Aristotle’s legislator forms habits in citizens by requiring them to perform virtuous acts, on the theory that repeated practice of virtue gradually produces virtue as a settled disposition. This is precisely what the Harm Principle prohibits. The citizen who is harmless to others is beyond the state’s reach, even if she is developing no virtuous dispositions whatsoever, even if she is becoming progressively less capable of the practical wisdom that Aristotle considered the crown of human development. Her atrophy is her own concern.

Mill was aware that this position required a defense against the charge that it produced exactly the moral vacuum the pre-modern tradition had worried about. His response was that human excellence develops better through freedom than compulsion; that the free exercise of faculties, including discernment, is itself the gymnasium of human development.⁶⁶ This argument has genuine force: the history of compulsory virtue is not encouraging. But it rests on an assumption not guaranteed by the principle: that free societies would cultivate the institutions and practices that develop judgment. The Harm Principle tells us what the state cannot do. It says nothing about what civil society must do to fill the resulting space. When civil society fails to fill it, the principle offers no remedy.

Perfectionist State

The catastrophes of the twentieth century confirmed the liberal retreat with a force that no previous argument had been able to muster. Fascism and Soviet Communism were, among other things, experiments in what a perfectionist state looks like when it has access to modern administrative machinery, mass communication, and the technology of industrial killing. The result settled what earlier generations had argued about in theory.

Stalin’s collectivization of Soviet agriculture enacted this catastrophe at civilizational scale. The policy eliminated local agricultural judgment—peasant farmers’ generations of accumulated knowledge about soil conditions, planting schedules, crop rotation, and seasonal weather patterns specific to their regions—and replaced it with centralized planning directives issued from Moscow. Bureaucrats without knowledge of local conditions mandated grain quotas and planting schedules determined by theoretical models and political targets rather than by the distributed practical wisdom of the people who worked the land. The result was the Ukrainian famine of 1932–33, the Holodomor, in which millions died because centralized planners, lacking dispersed knowledge available to local farmers, imposed agricultural requirements catastrophically unsuited to local conditions.⁶⁷ This was not the unfortunate cost of a well-intentioned mistake. It was the logical outcome of eliminating situated judgment in favor of centralized specification: the book’s entire argument made visible in the deaths of millions.

Judith Shklar, writing in 1989 in the aftermath of those experiments, gave the liberal position its most honest self-description. She called it the “liberalism of fear”⁶⁸: a politics organized around the imperative to prevent cruelty rather than the aspiration to achieve any positive good. Not the good life, but the avoidance of the worst. Cruelty-first is the political philosophy of people who have witnessed what political philosophies produce.

This is the inheritance that shaped Rawls. The liberalism of fear was a civilization’s traumatized response to the demonstration that certain mistakes are irreversible (a settlement imposed by catastrophe rather than reasoned from first principles). The summum malum that Hobbes described as abstract fear of violent death had become, by 1945, a documented historical experience on a scale foreclosing easy argument. The case against the perfectionist state was no longer merely philosophical. It was empirical.

The result was a determination that the retreat had not gone far enough, that the liberalisms of Hobbes and Locke and Mill had still left too much room for the state to impose its vision of the good on citizens who disagreed. The solution was to make neutrality more robust, more systematic, and more immune to the kind of historical revision that had allowed perfectionist states to arise again and again despite the earlier lessons.

When Justice Became Procedure

By the time John Rawls published A Theory of Justice in 1971, the intellectual work had been prepared by three centuries of argument and confirmed by the most consequential political history in human experience. What Rawls contributed was systematic architecture: a framework rigorous enough to serve as the foundation for an entire tradition of jurisprudence, constitutional theory, and institutional design.

Rawls crystallized the structure into its final, most systematic form. His central claim was that justice is the “first virtue of social institutions” and that just institutions must be neutral among the “comprehensive doctrines” (the full conceptions of the good life that citizens hold).⁶⁹ The state cannot favor Christianity over atheism, classical virtue ethics over utilitarian hedonism, or any other substantive account of what makes a human life go well. Its job is to maintain the fair framework within which citizens can pursue their own conceptions of the good. The right is prior to the good.⁷⁰

The architecture of the argument is recognizable from the earlier tradition. Rawls, like Locke, grounds neutrality in an account of what reasonable people can agree to: his famous “veil of ignorance” thought experiment, in which parties choosing principles of justice do not know their own place in society, their natural talents, or their conception of the good.⁷¹ The principles chosen under this veil are ones that could be accepted by anyone, regardless of their particular commitments. This is public reason: the constraint that coercive laws must be justifiable in terms accessible to all reasonable citizens, not merely those who share a particular faith or philosophy.

The Scaffolding function, in this framework, is not merely unnecessary. It is unjust. A state that uses law to cultivate a particular conception of virtue (say, Aristotelian practical wisdom, or Maimonidean character formation) is imposing a comprehensive doctrine on citizens who may reasonably reject it. The state’s job is to secure the conditions under which citizens can cultivate whatever virtues they choose. It cannot tell them which virtues to choose without violating their equal standing as moral agents.

Rawls was a rigorous and careful philosopher, and he was aware of the tension at the heart of this position. He acknowledged, in Political Liberalism, that the neutral state is itself a substantive commitment, not a view from nowhere.⁷² The requirement that laws be justifiable through public reason is a principle that many reasonable citizens (including many religious citizens) might reject. He struggled, without full success, to explain why this particular form of neutrality was itself neutral rather than another comprehensive doctrine dressed in procedural clothing.

But these internal tensions, however real, did not prevent the framework from becoming dominant. By the late twentieth century, Rawlsian political liberalism was the operating system of the liberal legal academy. Its principles organized constitutional doctrine, administrative law, and institutional design. The state was a referee: it organized the terms of competition without prescribing what counted as winning. It provided the framework within which human flourishing could occur and was disqualified from defining flourishing itself.

Hollow State

The result of this four-hundred-year retreat is the peculiar institution that governs contemporary life: the Hollow State.

The Hollow State is, by historical standards, enormously active. It runs public schools, funds universities, licenses physicians and attorneys and pilots, mandates healthcare coverage, subsidizes artistic production, and employs hundreds of thousands of bureaucrats whose work involves, in some form, cultivating human capacities. In its actual operations, it scaffolds continuously.

But it can only speak of the Foundation.

Available to the modern liberal state for justifying its own actions are vocabularies of prevention, correction, and harm reduction. The school system exists because education corrects for market failures in human capital production. The healthcare mandate prevents the externalities of uninsured risk. The arts funding corrects for the public-goods underproduction of cultural markets. Every enabling activity must be translated into prevention’s grammar before it can be legitimized. We speak of “human capital” when we mean wisdom. We speak of “transaction costs” when we mean the bonds of community. We speak of “risk” when we mean the possibility of failure inseparable from the possibility of genuine achievement.

This translation is not merely imprecise. It is structurally distorting. Lacking any internal limiting principle, the vocabulary of harm prevention expands without natural boundaries. Every new harm justifies an extension of the prevention apparatus. Every extension generates new harm categories requiring further extension. The logic is self-reinforcing because it is one-directional: there is no point at which the vocabulary of harm prevention can say “enough; we have prevented sufficient harm, the Scaffolding is no longer needed.”

The Precautionary Principle illustrates this mechanism in its clearest form. In its canonical formulation, the Precautionary Principle holds that where there is a threat of serious or irreversible damage, lack of full scientific certainty shall not be used as a reason for postponing measures to prevent environmental degradation.⁷³ This seems reasonable. The problem is structural: the principle tells regulators when to act (whenever harm is possible) but provides no equivalent principle for when to stop. It has an accelerator but no brake. Every additional category of possible harm (environmental, psychological, reputational, dignitary) activates the apparatus. No category of achieved safety deactivates it. The machine cannot declare victory and stand down, because the vocabulary in which victory could be declared does not exist.⁷⁴

The vocabulary of the Thomistic tradition had a limiting principle: tikkun ha-guf is prior in time but subordinate in rank. Physical security makes wisdom possible; it does not replace it. The Foundation is the scaffolding for the Scaffolding. This hierarchy gives the political order something to be for beyond its own perpetuation. It makes the question “when have we done enough?” answerable: when citizens have developed the wisdom to govern themselves, the law’s pedagogical work is complete.

Unlike the Thomistic framework, the vocabulary of harm prevention has no equivalent limiting structure. Harm is open-ended. Its definition expands with the administrative apparatus that measures and addresses it. What begins as “harm” meaning physical injury gradually expands to include economic injury, psychological injury, dignitary injury, environmental injury, expressive injury. Each expansion is locally defensible (there are real people being hurt by things that earlier categories missed). But the cumulative result is a machine with no governor, no mechanism for asking whether the cost of prevention has begun to exceed the cost of the harm being prevented.

The Hollow State’s borrowed vocabulary is not its only problem. Its deeper problem is that it actively obscures what the state is actually doing and why. A government that justifies public education as “human capital formation” has hidden from itself the reason education was worth building. It cannot, within its own justificatory framework, explain why wisdom matters; only why productivity matters. When the two diverge, as they sometimes do, the framework has no resources for choosing wisdom. It can only choose productivity.

This is the intellectual condition of a civilization that has systematically stripped its governing institutions of the vocabulary they would need to ask whether any given prevention measure is worth its costs to human capacity. Not the costs in dollars; those can be quantified and analyzed. The costs to discernment, discretion, and the development of practical wisdom that occurs only through the exercise of practical wisdom. The Hollow State cannot see these costs because its vocabulary has no name for them. They do not show up in the cost-benefit ledger. They are the transaction costs that turn out not to be waste.

Chicago’s Inheritance

Chapter 1 described the intellectual appeal of the Chicago School of Law and Economics: the promise of mathematical precision applied to legal problems, the hope that sufficiently careful modeling could prevent the harm caused by arbitrary and inconsistent judgment. Chapter 2 described what that promise cost: the confusion of risk with uncertainty, the atrophy of capacities that develop only through confronting problems that resist modeling.

Neither chapter fully explained why this particular framework arrived when it did and achieved the dominance it achieved. The Chicago School’s analytical tools were powerful. But powerful tools appear in many eras and fail to take hold for want of a receptive environment. Law and economics conquered legal academia in the 1970s and 1980s because the analysis arrived in a society that had already, over four centuries, dismantled the vocabulary it would have needed to resist what the analysis implied. The quality of work mattered; the receptive environment mattered more.

The liberal state had already been persuaded, through four centuries of the process described in this chapter, that it had no authority to cultivate judgment. The Chicago School arrived with mathematical apparatus to measure friction and demonstrate its costs with a precision the neutral state’s existing vocabularies could not match. The framework fit the Hollow State’s pre-existing shape perfectly (because it had already been hollowed out by the project of extracting virtue from law’s legitimate purposes).

Reasonable in its time, the Grotian modus vivendi was a fitting response to an emergency. The Hobbesian inversion solved a real problem while creating a different one. The Lockean moralization was an internally coherent theological and philosophical argument. The Rawlsian crystallization addressed genuine injustices of the perfectionist tradition. None of these thinkers was wrong about what they were responding to. The wars were real. The gulags were real. The costs of a perfectionist state gone wrong are among the worst things a civilization can inflict on itself.

But the aggregate result of these reasonable responses to real problems is a governing framework that cannot tell the machine when to stop. Not because no one is paying attention. Because the vocabulary to articulate the stopping condition was abandoned four centuries ago, in the justified panic of a civilization watching itself tear itself apart over what it meant to be human.

Law’s Scaffolding function was abandoned: its capacity to cultivate judgment, character, and practical wisdom (the capacity to act under uncertainty, exercise discretion, and take responsibility for outcomes that rules cannot fully determine). The Foundation alone was installed in its place: harm prevention, physical security, procedural order, and the endless production of rules to govern the spaces where judgment used to live.

The Safety Machine is the Foundation without the Scaffolding. People of considerable intellectual sophistication built it deliberately, trying to solve real problems. They solved them. And in doing so they created the conditions for a different kind of problem: one that arrives not in the form of a catastrophe legible enough to demand a response, but in the form of a slow atrophy that looks, from inside the machine, like progress.

The Chicago School inherited this condition and equipped it with better math and sent it forward.

The next chapter traces what the inheritance built. Once the philosophical vocabulary for law’s second purpose was gone, judgment itself became a cost — and an entire industry arose to eliminate it.

Chapter 4: Prudence as Transaction Cost

Three developments during the 1970s through the 1990s share a common logic. Richard Posner published the framework that classified judgment as a transaction cost subject to optimization. American schools adopted zero-tolerance discipline policies that removed from principals the authority to distinguish a first-grader’s camping utensil from a weapon. And the compliance industry grew from an incidental back-office function into one of the country’s largest professional sectors.

Profession without Judgment

In 1970, the United States Bureau of Labor Statistics did not track compliance officers as a distinct occupational category. The profession was too small to merit a line in the federal employment surveys. By 2022, more than 353,000 Americans held the title: a figure that excludes the attorneys, auditors, risk analysts, and consultants whose primary function is identical but whose business cards say something else.⁷⁵ Major financial institutions now employ more compliance staff than trading staff. American hospitals dedicate entire administrative floors to documentation whose primary purpose is litigation defense rather than patient care. Universities maintain offices staffed by dozens of Title IX coordinators, accreditation specialists, and institutional review board administrators whose collective function is to ensure that every decision that could become the subject of a lawsuit is traceable to a rule rather than to a person.⁷⁶

The compliance industry generates an estimated $270 billion annually in the United States alone.⁷⁷ Most of this investment goes to documenting the absence of liability, not to preventing harm. The compliance officer’s core product is a defensible paper trail.

The compliance industry grew alongside the century’s most consequential institutional failures. Enron, the 2008 financial crisis, the opioid epidemic, the Boeing 737 MAX: each occurred inside institutions drowning in compliance documentation. Boeing held impeccable FAA certification records for MCAS. Enron had four sets of outside auditors. The nation’s largest opioid distributors filed hundreds of thousands of suspicious activity reports precisely as required. The machine ran exactly as designed. What the compliance architecture could not supply was the judgment to notice that following the procedure was producing the catastrophe. The procedure was the entire point. The procedure was the defense.

The growth of compliance as a profession maps almost precisely onto the penetration of the Chicago School’s framework into institutional life. Understanding one explains the other.

A Category Error

Chapter 1 described how Ronald Coase’s 1960 insight migrated from description to prescription. Coase showed that transaction costs, not legal rules, determine economic outcomes. The prescription followed directly: eliminate friction, replace judgment with rules, build institutions where optimal outcomes emerge automatically from correct rule structures.

Richard Posner’s Economic Analysis of Law, first published in 1973, supplied the operational framework.⁷⁸ Where Coase had described how transaction costs shape behavior, Posner prescribed how institutions should be designed to minimize them. Legal rules should be evaluated by whether they maximize social wealth. In this framework, judgment—the discretion exercised by judges, administrators, professionals, principals—becomes a source of uncertainty, inconsistency, and delay. Judgment is a transaction cost. It consumes time, produces variance, and cannot be audited for efficiency.

This classification was deliberate. The economic argument for eliminating judgment is, within its own terms, correct. A rule is cheaper to apply than a judgment. A bright-line standard is cheaper to apply than a contextual standard. An algorithm is cheaper to apply than a deliberation. The calculation is straightforward. The error is taxonomic.

Aristotle distinguished two modes of practical knowledge: techne, the knowledge of how to produce things, and phronesis, the knowledge of how to act well.⁷⁹ The craftsman who knows how to build a table is exercising techne. His knowledge terminates in the table; the table is external to the craftsman’s character. The craftsman can retire when the table is built. The judge who knows how to sentence wisely is exercising phronesis. Her knowledge does not terminate in the sentence; the sentence is the expression of a capacity that has been cultivated through the practice of judgment and that is inseparable from the character of the judge herself.

Techne can be systematized, encoded, and transmitted. A recipe is codified techne. An algorithm is codified techne. Optimization is always the replacement of slow, variable techne with faster, more consistent techne. This is legitimate and often admirable. A hospital’s surgical checklist is better than the idiosyncratic intuitions of individual surgeons about sterile technique. The checklist is more reliable because it encodes a best practice that is best: the same procedure every time really is the right answer.

Phronesis cannot be systematized without ceasing to be phronesis. The capacity for wise deliberation in particular circumstances develops only through the practice of judgment. Aristotle described this as the ability to perceive “the truth” of a situation in its irreducible particularity.⁸⁰ You can encode the conclusion of a phronetic judgment. You cannot encode the judgment itself, because the judgment includes the perceiving and deliberating that generated the conclusion, and precisely that perceiving and deliberating constitutes the capacity. A physician who has learned to recognize pneumonia through years of auscultation developed a perceptual capacity that cannot transfer to someone who has only read descriptions of what pneumonia sounds like. The practice is the knowledge.

The Chicago School’s framework classified judgment as techne subject to optimization when judgment in law’s Scaffolding function is phronesis constituted by exercise. The compliance industry is what institutions build when they accept that classification and act on it.

Rules That Breed Rules

The category error, once embedded in institutional logic, generates its own self-reinforcing dynamic. Each rule that displaces discretion does not merely eliminate a single exercise of it. It eliminates the practice that would have developed the capacity for all future discretionary decisions in that domain. The deficit, once created, justifies its own extension.

The logic runs as follows. Institutions replace judgment with rules to eliminate variance and establish compliance defenses. The rules ensure that humans operating within the institution do not develop the capacity to handle situations the rules do not address. When those situations arise (and they always do, because the world generates more situations than rules can enumerate), the institution lacks the discernment. The institutional response is to write more rules. Each additional rule expands coverage at the cost of further judgment development. Coverage expands. Capacity contracts. The institution writes more rules.

The paradox is visible in the history of financial regulation. The Securities and Exchange Commission’s initial regulations in the 1930s required disclosure and prohibited specific forms of fraud. Compliance was a matter of not lying and filing the required documents. By 2008, the regulations governing a major financial institution occupied thousands of pages of federal statute and tens of thousands of pages of agency rulemaking, supplemented by hundreds of guidance documents, interpretive letters, no-action letters, and enforcement actions that functioned as rules without going through the rulemaking process.⁸¹ The compliance staff of a large bank had grown commensurately. The professional discretion of loan officers, risk managers, and trading desks had been steadily replaced by models, algorithms, and documented compliance processes that generated the paper trail necessary to demonstrate that every decision had been made in accordance with applicable rules.

In 2008, the models failed. The failure violated no rule. The models had been built, validated, stress-tested, and approved according to correct procedures. Every compliance check was satisfied. The risk management frameworks were audited. The suspicious activity reports were filed. The discernment that might have perceived that AAA-rated securities backed by subprime mortgages were not actually AAA-quality risks had been replaced by the model that said they were, and the compliance architecture that documented the model’s use.

The institutional response to the 2008 crisis was to write more rules. The Dodd-Frank Act added more than 400 new regulatory requirements. Compliance staff grew by approximately 60 percent in the following decade.⁸² The judgment that the rules were replacing had already been substantially replaced before the crisis. The institutional lesson drawn from the crisis was that the rules were insufficient: more coverage was needed, rather than that the rules were themselves producing the atrophy that had made the crisis possible.

This is Haddaway’s Law at institutional scale. The attempt to eliminate the harm of unreliable judgment produced the atrophy of judgment, which produced the catastrophic failure of the model that had replaced it, which justified more rules to govern the domain from which judgment had been eliminated.

Sporking Schools

Downes Elementary School, Newark, Delaware. September 2009. The lunch tables were full of first-graders. Zachary Christie, six years old, had been in school for perhaps three weeks, long enough to have decided on some days to wear a suit and tie because he took school seriously. On this particular morning he was also excited about a recent acquisition: a camping utensil from his new Cub Scout kit, a compact combination of fork, spoon, and small folding knife, the kind sold in every outdoor supply store and distributed to millions of Scouts across the country every year.⁸³

He pulled it out at lunch to eat with it.

A lunchroom aide noticed the folding component. The aide reported it to the principal. The principal consulted the Christina School District’s Code of Conduct, which prohibited weapons on school property “regardless of possessor’s intent.”⁸⁴ A camping utensil with a folding blade is an object with a blade. An object with a blade is a weapon. A weapon triggers mandatory suspension. The principal applied the policy correctly and suspended Zachary Christie for forty-five days, to be served at the district’s alternative placement (the facility reserved for students whose infractions were too serious for a regular classroom setting).

His mother received the call that afternoon.

When the case came before the school board, President George Evans offered the rationale the policy required. Neither Evans, nor the principal, nor the aide who reported the incident believed the first-grader posed a genuine threat.⁸⁵ Evans said so directly. His point was different: “There is no parent who wants to get a phone call where they hear that their child no longer has two good seeing eyes because there was a scuffle and someone pulled out a knife.”⁸⁶ The board had no authority to grant an exception. The policy’s text admitted none. Evans was not explaining a decision the board had made. He was explaining a decision the policy had made for them, weeks before Zachary Christie unpacked his Scout kit.

The principal had no discretion to exercise: the institution was structured to make discretion impermissible rather than unnecessary.

The institution’s relationship to harm is this: the relevant harm is not Zachary Christie, suspended for a camping utensil, but a hypothetical future child injured by something that resembled a camping utensil in the hands of someone with violent intent. The zero-tolerance policy eliminates the second harm by eliminating the judgment that could distinguish between them. The first harm (the real, actual harm to a real, actual child) is classified as acceptable collateral damage in the prevention apparatus. The rule prevented the rule-avoidance that could have produced the second harm. The rule also produced the first harm. The rule could not see this, because rules cannot see.

Thomas Aquinas held that law requires two essential properties: ratio (reason) and telos (purpose).⁸⁷ A rule severed from its purpose produces not law but power wearing procedural clothing. The zero-tolerance policy’s purpose was student safety. Applied to Zachary Christie, the rule undermined student safety in the particular case while protecting the institution from liability in the general case. The rule devoured the purpose. The institution called it compliance.

Zero-tolerance policies spread across American schools in the 1990s through a specific mechanism: the Gun-Free Schools Act of 1994 conditioned federal funding on the adoption of mandatory expulsion policies for students who brought weapons to school.⁸⁸ The federal compliance requirement produced state implementing legislation. State legislation produced district policy. District policy produced the institutional architecture that removed from principals the authority (and over time the capacity) to distinguish a Cub Scout’s camping utensil from a weapon.

The parents lobbying for zero-tolerance policies after campus violence were responding to genuine harm. The legislators drafting the federal requirement were trying to address that harm. The district administrators writing the policy were trying to protect their institutions from litigation. Each step was locally rational. The aggregate result was an institution that could no longer tell the difference between Zachary Christie and a threat to public safety.

Not incompetence. Not malice. The systematic substitution of rules for judgment, producing, over time, the atrophy of the capacity to judge.

Moral Deskilling

The pathology Zachary Christie’s case illustrates operates with particular precision wherever automation replaces the exercise of skill. Rules eliminate judgment. Judgment elimination produces atrophy. Atrophy produces increased reliance on rules.

Aviation provides the most precisely documented example. In 2013, the Federal Aviation Administration released a study analyzing the correlation between cockpit automation and safety incidents. The conclusion was blunt: overuse of cockpit automation leads to pilot skill atrophy, manifesting as degraded manual flying performance. Sixty percent of the safety incidents reviewed were attributable to manual handling and flight control errors—errors made by pilots who had spent their careers supervising automation they rarely needed to override.⁸⁹

The mechanism operates in four stages. Automation handles tasks that pilots once performed manually, and safety improves: incident rates fall because automation is more consistent than human pilots under routine conditions. Because pilots no longer perform manual tasks regularly, the skills atrophy. Psychomotor and cognitive abilities that develop through practice—spatial awareness, proprioceptive feedback from the controls, pattern recognition that detects anomalies before instruments register them—diminish through disuse. When pilots encounter situations requiring manual flight, they perform worse than they would have with regular practice. Incidents increase in those specific situations. The institutional response is to extend automation further, reducing the situations requiring manual flying rather than increasing manual practice. With fewer opportunities to practice, skills deteriorate further. The loop tightens.⁹⁰

The same structure, documented with causal precision, governs spatial memory and GPS navigation. A longitudinal study published in Nature Communications in 2020 found that greater lifetime GPS use was associated with worse spatial memory during self-guided navigation; the relationship was causal, not merely correlational.⁹¹ Subjects who increased their GPS use between measurement points showed corresponding declines in their ability to construct cognitive maps of familiar environments. The dose-response relationship was clear: the more GPS was used, the worse the navigation without it. GPS users noticed fewer environmental landmarks, encoded fewer of them in memory, and produced less accurate representations of spaces they had traveled through repeatedly. The device replaced the spatial reasoning that would have developed through unassisted navigation. Without practice, capacity deteriorated. Without capacity, the device became more necessary. The loop ran.

The pattern extends to judicial sentencing. The COMPAS recidivism algorithm, used by courts in multiple states to inform parole and sentencing decisions, assigns defendants a numerical risk score based on demographic and criminal history data.⁹² Whether the algorithm is “fair” generated one of the more technically sophisticated legal-empirical debates of the past decade. ProPublica found that the algorithm incorrectly flagged Black defendants as high-risk at roughly twice the rate of white defendants; the algorithm’s defenders correctly noted that false positive rates were equivalent across races when fairness was defined differently. The technical disagreement was real. The mathematically incompatible definitions of fairness at stake cannot simultaneously be satisfied when base rates differ across demographic groups. The “fair” algorithm has encoded a choice about which injustice to accept.

Neither side of the fairness debate addressed prominently what happens to judges who regularly use COMPAS. A judge who consults a numerical risk score before sentencing (even to “inform” rather than determine the sentence) performs a different cognitive task than a judge who reaches the same conclusion through deliberation. The score anchors. Research on anchoring effects in judicial sentencing demonstrates that numerical inputs systematically shift sentencing outcomes toward the anchor even when judges explicitly try to reason independently from it.⁹³ Over time, the judge who regularly defers to a risk score develops less of the sentencing judgment the score was introduced to supplement. The algorithm occupies the cognitive space that deliberation would have occupied. The space for independent judgment contracts. The algorithm becomes more necessary precisely because it has been most used.

Shannon Vallor names what is being lost across all these cases: the “moral muscle,” developed through the exercise of judgment under conditions of genuine uncertainty, that constitutes practical competence.⁹⁴ The argument follows directly from Aristotle’s account of virtue as hexis—a stable disposition formed through habituation. Physical strength develops through resistance and atrophies without it. Moral and practical judgment develops through the exercise of judgment in difficult situations and atrophies when automation supplies the output in its place. The loan officer who defers to a credit score, the judge who defers to a risk algorithm, the principal who defers to a zero-tolerance policy—each is spared the struggle of deliberation. Each is, over time, less capable of the deliberation they have been spared.

Nicholas Carr, in The Glass Cage, documents the sensory and phenomenological dimension of this atrophy.⁹⁵ Automation changes the relationship between a human being and the domain being navigated. The pilot who flies manually inhabits the aircraft differently than the pilot who monitors automation. The navigator who reads terrain inhabits space differently than the navigator who follows turn-by-turn instructions. The intimacy of genuine engagement—the dwelling in a domain that develops through the friction of operating within it—requires something to push against. The frictionless interface produces an observer rather than an actor, a monitor rather than a practitioner, someone whose contact with the domain is mediated rather than direct.

Evan Selinger and Brett Frischmann extend this analysis to the design principle underlying most contemporary automation: the ideal of frictionlessness.⁹⁶ Silicon Valley’s animating aspiration—the seamless interface, the frictionless transaction, the decision without doubt—treats friction as the enemy of value. Selinger and Frischmann argue the reverse: friction is where agency lives. The “micro-struggles” of deliberation, the discomfort of genuine choice, the cognitive labor of contextual judgment are not inefficiencies to be engineered away. They are the gymnasium of practical wisdom. Remove them, and you produce a more manageable human, not a more capable one.

Judgment-Proof Humans

The Deskilling Loop’s endpoint is not a lazy person who has outsourced inconvenient thinking. It produces something structurally different: a person who has had no occasion to develop the capacity the compliance structure was built to replace.

The distinction matters. The pilot who trained before heavy automation and then lost manual flying skills through disuse is different from the pilot trained entirely within a high-automation environment who never acquired those skills. Both will fail under identical conditions. Only one experienced a loss. The other never had the capacity the situation demands.

Until automation fails, the pilot who has never flown manually under pressure cannot be distinguished from the pilot who has forgotten how. At that moment, the first pilot is revealed to have a structural gap where a capacity should be. The gap was produced by training within a system designed to make the development of that capacity unnecessary.

The compliance architecture in American schools performed the same substitution at the same professional inflection point. Principals who entered the profession after zero-tolerance policies were installed learned to exercise their judgment in every domain except the hard disciplinary cases—the ambiguous objects, the contextual situations, the moments that require weighing a child’s actual intent against an abstract category. Those cases were removed from the domain of professional discretion before the professional could develop the relevant capacity. The policy removed them from the curriculum of professional development entirely. The principal has no occasion to exercise the judgment the policy replaces. When the policy produces an absurdity, the principal lacks the professional vocabulary to recognize that following the policy was the error. Following the policy was the whole of what they were trained to do. The policy is their judgment’s entire horizon.

The disappearance of the professional is the complement of the emergence of the compliance officer. As compliance architectures replaced the discretionary judgment of professionals, the locus of institutional knowledge migrated from the practitioner to the system. The physician who once exercised clinical discernment now exercises clinical documentation—recording the compliance of their decisions with applicable protocols, billing codes, and treatment guidelines. The lawyer who once exercised legal judgment now primarily exercises compliance management—reviewing risk exposure across a matrix of regulatory requirements, documenting the client’s informed consent to the risk that the lawyer has identified and disclosed. The banker who once exercised credit judgment now exercises model documentation—recording the quantitative basis for a lending decision and preserving the paper trail that will defend the decision if challenged.

The professionals have not disappeared from their offices. They have been converted from agents who exercise practical wisdom into operators who document compliance. The knowledge that mattered for the exercise of discernment—the feel of a situation, the perception of what a particular context required, the practical wisdom that could not be fully articulated because it was constituted by the practice itself—is no longer the primary professional output. The primary professional output is the record that demonstrates adherence to applicable rules.

This is the Judgment-Proof Human: a person nested so thoroughly inside a compliance structure that the structure has pre-empted the development of every capacity the structure was built to replace. The institution is protected from liability when the structure fails. The individual is protected from the responsibility of judgment. Both forms of protection are real. Both are purchased at the same price: the deliberative capacity that could have prevented the failure in the first place.

The individual pathology is tragic enough. The institutional consequence is more severe. The Judgment-Proof Human occupying a professional role cannot see the structure that produced them as a structure rather than as the nature of professional life. The principal who has never had to distinguish a camping utensil from a weapon does not experience the zero-tolerance policy as a limitation on judgment. The policy is the entirety of their professional horizon for that class of decision. When it produces Zachary Christie, the response is not “the policy prevented a judgment I should have made.” The response is “the policy made a decision I was not permitted to make.” The institution has eliminated the perspective from which its own failure would be visible.

An Elegant Mistake

Chapter 3 traced the intellectual genealogy that produced the Hollow State: a governing order that acts to cultivate, educate, and form human beings but can only justify itself in the borrowed vocabulary of harm prevention and efficiency. The Chicago School did not create that condition. It inherited it and gave it better instruments.

The instruments were elegant. Posner’s framework brought analytical rigor to legal reasoning that had often proceeded through unstated assumptions and conclusory moral assertions. Cost-benefit analysis exposed rules that claimed to protect consumers while actually protecting incumbent businesses. The behavioral economics that grew from the law-and-economics tradition produced the most sophisticated account of human decision-making that legal scholarship has ever offered. The intellectual program was ambitious, disciplined, and largely internally consistent. Its best practitioners were clear-eyed about its limitations. Its influence on institutional design fell to practitioners less attentive to those limits.

The error was taxonomic.

Aristotle observed that each domain of knowledge has its appropriate method, and that intellectual error often consists of applying the method appropriate to one domain to problems belonging to another.⁹⁷ The mathematical methods of techne—the optimization of specifiable outputs from quantifiable inputs—were applied to the domain of phronesis: the cultivated capacity to perceive what a situation requires and to act accordingly. The outputs of practical wisdom can sometimes be measured. The cultivation of practical wisdom cannot be optimized, because optimizing it—substituting the output for the exercise—destroys the very thing being cultivated. A gymnasium does not build strength by lifting the weights for you.

The Safety Machine is what this error produces when it runs at institutional scale. Each individual design choice was defensible. Reducing variance in judicial sentencing reduces the risk that similarly situated defendants receive wildly different sentences—a genuine injustice the guidelines addressed. Mandating cockpit automation reduces the risk of the kind of manual flying error that caused most pre-automation aviation fatalities—a genuine improvement. Implementing zero-tolerance policies reduces the risk that a principal’s idiosyncratic judgment produces inconsistent discipline exposing the district to litigation—a genuine institutional benefit within the framework that classified the litigation risk as the primary harm to be addressed.

The aggregate result of individually defensible choices is an institutional environment in which practical wisdom has no occasion to develop, no space to operate, and no vocabulary in which its loss can be named. The Hollow State cannot explain why the principal should be permitted to see the camping utensil as what it is. The only vocabulary available is harm prevention, and in that vocabulary, the camping utensil is a knife-shaped object whose status cannot be assessed without a rule. The question of what kind of discernment the principal should be developing, and what institutional conditions would develop it, cannot be posed in the vocabulary of harm prevention. It requires the vocabulary of the Scaffolding—the vocabulary abandoned four centuries before the first compliance officer was hired.

The Chicago School solved the problem it set out to solve. It identified transaction costs and developed powerful tools to minimize them. It identified judgment itself as the transaction cost most available for elimination. The math was right. The problem was wrong.

Part Two examines what that solution built: the specific mechanical architecture of the Safety Machine, how it sustains and extends itself through bureaucratic logic that operates independently of any individual’s intentions, and what it produces when it encounters situations that require precisely the capacity it has systematically dismantled. The Boeing 737 MAX is the most precisely documented illustration of a machine that worked exactly as designed and killed 346 people because the design had eliminated the judgment that might have noticed the design was wrong.

Chapter 5: Harm-Prevention Architecture

In 1946, Congress built a brake into the administrative state. It took three decades for the machine to find a way around it.

A Fierce Compromise

In 1946, Congress struck what administrative law scholars call a “fierce compromise.” The Administrative Procedure Act established the terms under which federal agencies would exercise delegated power: agencies seeking to make binding rules must first publish proposed rules, accept written comments from interested parties, respond to significant objections in writing, and submit the resulting rule to judicial review for reasonableness. The process was slow and expensive. The friction was the point. Democratic law is not fast law. The APA’s rulemaking procedures were the machine’s constitutional covenant. The assurance was that administrative power, however vast, would remain accountable to something beyond the agency’s own judgment.⁹⁸

The covenant held for roughly three decades. Then the courts tried to strengthen it, and the machine found a door.

Beginning in the late 1970s, the D.C. Circuit Court of Appeals began applying with increasing rigor what it called the “Hard Look” doctrine. The doctrine required agencies to demonstrate that they had exhaustively considered every relevant factor, responded substantively to every significant comment, and examined every viable alternative to the rule they were proposing. Arbitrary or capricious rulemaking would be vacated. Hard Look review was judicial oversight doing exactly what it should.⁹⁹

Faced with the prospect of multi-year rulemaking efforts (agencies had watched colleagues spend a decade building a regulation, only to see a court vacate it for a minor analytical deficiency in the record), rational administrators began asking a question the APA had not anticipated: what if we do not formally rule at all?

That question’s answer is the subject of this chapter.

Administrative Dark Matter

The Administrative Procedure Act draws a categorical line between two kinds of agency action. “Legislative rules”—binding pronouncements that create new rights and obligations—must go through notice and comment. “Interpretive rules” and “general statements of policy”—documents that merely explain existing duties—are exempt from these requirements. The line was always somewhat porous in theory; in practice, it became the primary operating mechanism of the modern administrative state.

What scholar Clyde Wayne Crews terms “Regulatory Dark Matter” moved into the space below the line: agency memoranda, bulletins, circulars, guidance documents, blog posts, Dear Colleague letters, and press releases that carry the practical weight of law without its democratic pedigree.¹⁰⁰ These documents technically bind no one. They are, in the agency’s own characterization, merely clarifications. They offer helpful explanations of what the statute already requires. The regulated party remains free to ignore them.

Free, that is, in the same way a person is free to ignore a creditor who holds a lien on everything they own.

The mechanics worked as follows. In 2011, the Department of Education’s Office for Civil Rights issued a nineteen-page letter addressed to all recipients of federal education funding.¹⁰¹ The letter concerned sexual violence on campus. The statutory authority was Title IX of the Education Amendments of 1972—thirty-seven words long—prohibiting sex discrimination in education. The letter argued that a university’s failure to adequately investigate and adjudicate sexual violence constituted sex discrimination in violation of the statute. Specifically, the letter required universities to use the preponderance of the evidence standard in disciplinary proceedings (the lowest civil standard of proof) and prohibited the clear and convincing standard that many institutions had been using.

No vote in Congress. No notice-and-comment period. No judicial review of the letter’s legal theory. The document was technically non-binding. Its coercive force came from a single sentence toward the end. An institution that failed to comply risked losing all federal funding.

For a modern university, federal funding underwrites research, financial aid, facilities, and graduate programs. The threat of its loss is existential. Consequently, the Dear Colleague letter was not a suggestion. It was law: more comprehensive and more immediate in effect than most congressional statutes, issued by a regional administrator with no democratic accountability, in a form that the courts struggled to review because it was technically not a final agency action.

Universities across the country reformed their disciplinary processes within months. Legal scholars debated whether the letter’s requirements were constitutionally sound and whether the procedural model it mandated adequately protected the due process rights of accused students. The letter has since been rescinded and partially reinstated in modified form.¹⁰² The mechanism is not in serious dispute: a document that the agency said was not law functioned as law more swiftly and comprehensively than most laws do.

This mechanism is neither aberrant nor exclusive to the Department of Education. The Federal Trade Commission, operating without a comprehensive federal data privacy statute, built what two Columbia Law School scholars have called “the new common law of privacy” entirely through enforcement actions and consent decrees.¹⁰³ When the FTC settled data security cases against Wyndham Worldwide and LabMD, the settlement terms—the specific technical requirements, the audit obligations, the duration of oversight—became the de facto industry standard. Privacy lawyers and chief information security officers analyzed the decrees the way common law lawyers analyze precedent, because they were precedent. Never subjected to adversarial testing, the FTC’s theory of liability in these cases—that inadequate data security constituted an “unfair practice” under Section 5 of the FTC Act—was almost never tested in court. The cases settled because defendants could not afford to lose, and the FTC’s legal theory was never obliged to survive judicial scrutiny.¹⁰⁴

In LabMD v. FTC, the commission argued that a company had constructive notice of its data security obligations not from any published regulation but from “a tapestry of settlements, speeches, and guidance documents.”¹⁰⁵ The law was not written. It was embedded in agency action.

Hidden Regime

The Dear Colleague letter and the FTC consent decree share a structural feature: they require compliance from private actors who are not technically bound by them. The compliance is achieved through private intermediaries who do the agency’s enforcement work outside any administrative law framework. This is compliance isomorphism. It is the machine’s most elegant mechanism.

When a cyber-insurance carrier requires a corporate policyholder to implement multi-factor authentication as a condition of coverage, the carrier is acting as a private market participant, exercising no formal government authority. But the standard it is enforcing derives from the FTC’s settlement terms, which derive from the FTC’s interpretation of Section 5, which has never been subjected to notice-and-comment rulemaking or comprehensive judicial review. The insurer enforces the standard through premium differentials and policy exclusions. A company that does not comply does not receive a citation; it pays more for coverage, or cannot obtain it.

Consequently, a comprehensive regulatory regime emerges, enforced through market mechanisms, invisible to administrative law because no government actor is directly imposing the requirements.¹⁰⁶ The “Hidden Regime,” as the research literature calls it, is not less effective than formal regulation. In many respects it is more effective. Market enforcement is continuous and the consequences of noncompliance are immediate rather than delayed by litigation.

Beyond insurance, the shadow regulatory state extends widely. Credit rating agencies enforce Basel-adjacent capital standards. University accrediting bodies enforce Department of Education guidance on disability accommodation and Title IX adjudication. Accounting firms enforce SEC interpretive guidance on revenue recognition. The entities that actually govern day-to-day institutional behavior are private intermediaries—not the agencies that appear in the Federal Register—whose decisions are reviewed, if at all, by contract rather than administrative law.

The regulated entity faces a genuine navigational problem. Formal law—the statutes and regulations published in the United States Code and the Code of Federal Regulations—constitutes one layer of obligation. Binding agency rules constitute a second layer. Guidance documents constitute a third layer. Settlement-derived industry standards constitute a fourth. Insurance and accreditation requirements constitute a fifth. At any given moment, the binding legal obligations of a mid-sized financial institution, hospital, or university may be distributed across all five layers, partially inconsistent, and changing at different rates. No single document describes them. The compliance officer exists to maintain the map.¹⁰⁷

Kludgeocracy

Steven Teles introduced “kludgeocracy” in a 2013 essay that stands as one of the more precise diagnoses of the American regulatory condition.¹⁰⁸ A kludge, in engineering, is a rough-and-ready solution to an immediate problem. It is a patch applied without addressing the underlying architecture. The American legislative process is structurally biased toward kludge. Every veto point in the constitutional system (the committee chairman, the Senate filibuster, the conference committee, the president’s veto pen) functions as a toll booth. Paying the toll requires offering complexity: a carve-out for this constituency, a waiver for that state, an exemption for the industry whose lobbyist secured the last amendment. The resulting statute is not designed. It is accumulated. It does not solve the underlying problem. It patches around every objection until passage becomes mathematically possible.

Running to 906 pages, the Affordable Care Act exemplifies the pattern. The Dodd-Frank Wall Street Reform and Consumer Protection Act ran to 849 pages. These documents are long because every page represents a negotiation, a compromise, a carve-out that purchased a vote—not because the problems they address required that length, though the problems are complex. When the agencies responsible for implementing these statutes write the rules that give them effect, Dodd-Frank alone required hundreds of implementing rules. The Code of Federal Regulations as of 2023 contained over 185,000 pages of binding text.¹⁰⁹

The Mercatus Center’s RegData project attempted to quantify this complexity using methods borrowed from information theory.¹¹⁰ The research team applied Shannon entropy (a measure of informational unpredictability and cognitive load) to state regulatory codes and found that many of them are more cognitively difficult to process than the plays of Shakespeare. The regulatory code of Illinois, Minnesota, and California scored above 9.5 on the Shannon entropy scale; Shakespeare’s works typically score between 9.0 and 9.5. The law that governs daily commercial and personal conduct in these states is, by information-theoretic measurement, harder to read than King Lear.¹¹¹ Shannon entropy measures the unpredictability of each successive word in a text (the cognitive effort required to anticipate what comes next), and a regulatory code that scores higher than Shakespeare on this metric is not merely long but linguistically resistant to comprehension.

This is not a criticism of draftsmanship. The complexity is the content. A regulatory code that tried to be simpler would be a regulatory code that had failed to address all the edge cases, exceptions, and industry-specific provisions that the political process required. The complexity is what was purchased, at considerable expense, during the legislative and rulemaking processes. It is the mature expression of accumulated democratic compromise.

This barrier to entry operates in two dimensions. The first is economic: compliance costs fall disproportionately on small firms. The labor cost of regulatory compliance per employee is substantially higher for a ten-person company than for a ten-thousand-person company because the fixed costs of compliance (maintaining the legal expertise, the documentation systems, the audit readiness) do not scale linearly with firm size.¹¹² The sophisticated incumbent builds the compliance apparatus once and amortizes it across a large revenue base. The new entrant builds it from scratch against a smaller one, or cannot build it at all. This is regulatory debt as a barrier to competition. The accumulated complexity protects those who have already paid the price of mastering it.

The second dimension is cognitive and involves priesthood creation by the kludgeocracy. The compliance officer, the specialized regulatory attorney, the credentialed auditor exist because the law has become unintelligible to non-initiates. This phenomenon is a structural feature of any system complex enough to require specialization. It represents a failure of design, not of individual capability. The priest’s authority derives from the mystery, and there is nothing conspiratorial about this. Complexity generates its own advocates. The attorneys who have built their practices on the intricacies of Dodd-Frank are not lobbying for simplification. Their human capital is the complexity. The accrediting bodies, the compliance consulting firms, the regulatory affairs departments of major hospitals constitute what is appropriately called the Kludge Industry. It is an economic sector whose product is friction management, and whose continued existence depends on the friction remaining unresolved.¹¹³

The Number OSHA Chose

In 1980, the Supreme Court of the United States imposed a constraint on the machine that, had it survived, might have fundamentally altered its subsequent trajectory.

The case arose from an OSHA benzene standard. Benzene is a known human carcinogen. OSHA, invoking the Occupational Safety and Health Act’s mandate to protect workers from “significant risks of material health impairment,” sought to reduce permissible airborne benzene exposure to the lowest technologically feasible level. One part per million was the target. The agency’s position was essentially precautionary: since some level of benzene exposure causes some risk of leukemia, and since there is no identified threshold below which the risk becomes zero, the appropriate regulatory posture is to reduce exposure as far as technology permits.

The plurality opinion, written by Justice Stevens, rejected this reasoning and articulated what became the Significant Risk doctrine.¹¹⁴ Before OSHA could regulate any substance, the Court held, the agency was required to make a threshold finding that exposure to the substance at existing levels posed a significant risk of material health impairment. The significant risk finding was not a numerical threshold (Justice Stevens explicitly declined to quantify it), but it was a substantive requirement. The agency had to do more than establish that some risk exists at some level. It had to establish that the risk was meaningful enough to justify regulatory intervention.

To illustrate the principle, the Benzene plurality offered a figure: a one-in-one-thousand lifetime risk of developing a serious condition would generally qualify as significant, whereas a one-in-one-billion risk would not. Regulators could not invoke the mere existence of a risk to justify cost-regardless reduction. They had to quantify the risk, establish that it exceeded a threshold of materiality, and demonstrate that the proposed standard would meaningfully reduce it.¹¹⁵

This principle served as the machine’s one internal governor, allowing expansion only under constraints. It could expand, but it had to show its work. It had to show that the harm it was addressing was real, measurable, and substantial enough to justify the cost of addressing it. The principle acknowledged what the Benzene dissent did not want to acknowledge: that not every risk is worth regulating, that regulatory resources are finite, and that a rule requiring a cost-regardless response to any speculative harm was a rule without a limiting principle.

The Significant Risk doctrine survived. What dissolved around it was the evidentiary standard for establishing significance.

Vorsorgeprinzip

The Precautionary Principle entered American regulatory practice gradually rather than through any single decision or statute, shifting how agencies calculated the costs and benefits of regulatory intervention.

The principle originated in German environmental law (the Vorsorgeprinzip, or foresight principle) and was incorporated into international environmental instruments including the 1992 Rio Declaration.¹¹⁶ In its strong form, the principle holds that where an activity raises threats of harm to human health or the environment, precautionary measures shall be taken even if the causal relationships are not fully established scientifically. The burden of proof shifts: the proponent of the activity must demonstrate safety, rather than the regulator demonstrating harm.

American organic statutes, written mostly in the 1970s, do not formally adopt the Precautionary Principle. They retain the older risk-management framework, requiring agencies to establish significant risk before regulating. But the principle drifted into practice through mechanisms that did not require statutory revision.

The first mechanism, operating since the 1970s within EPA radiation and chemical risk assessment, was the Linear No-Threshold model, or LNT. The LNT model assumes that any exposure to a carcinogen, however small, carries some risk of harm, and that risk scales linearly with dose. There is no threshold below which exposure becomes safe. If the LNT assumption is accepted, then the Benzene significant risk doctrine can be satisfied through modeling rather than observation: construct a dose-response curve using LNT assumptions, and any level of exposure will generate a calculable risk estimate that can be characterized as significant. The Benzene constraint requires a significant risk finding. The LNT model generates a significant risk finding for any exposure. The constraint is formally satisfied while its substance is eliminated.¹¹⁷

The second mechanism was the expansion of “unquantifiable benefits.” The 2023 update to Office of Management and Budget Circular A-4 (the federal government’s governing document for regulatory cost-benefit analysis) explicitly authorized “distributional weighting.” This means assigning greater value to benefits accruing to underserved communities than to equivalent costs borne by others.¹¹⁸ This adjustment allows agencies to justify regulatory interventions whose costs exceed their quantifiable benefits by attributing sufficient weight to benefits that, by definition, cannot be quantified and therefore cannot be falsified. The mechanism is not irrational. It reflects a genuine and defensible judgment that cost-benefit analysis systematically undervalues harms to people with lower incomes. Yet it also removes a constraint: a cost-benefit requirement allowing costs to be outweighed by non-falsifiable distributional benefits is one that can always be satisfied.

The third mechanism was concept creep, documented by psychologist Nick Haslam. Examining the evolution of clinical and legal concepts of harm over several decades, Haslam identified what he termed “concept creep”: the expansion of psychological and social science concepts (“trauma,” “violence,” “harm,” “safety”) from narrower, more precise definitions toward broader, more inclusive ones.¹¹⁹ A concept that began anchored to a specific empirical phenomenon gradually extended to cover related but distinct phenomena, then to cover analogous phenomena in adjacent domains, then to become a framework applied to experiences that bore only metaphorical resemblance to the original referent.

In regulatory domains, “harm” (the jurisdictional trigger for nearly every health, safety, and civil rights regulation) quietly expanded its coverage. Physical injury was always harm. Physical injury plus psychological sequelae was harm. Psychological injury without physical contact was harm. Dignitary harm (the experience of being treated in a manner inconsistent with one’s dignity) became harm. Psychological unsafety (the subjective experience of an environment as threatening, regardless of objective threat) became harm. By 2020, OSHA had begun issuing guidance on “psychosocial hazards” (workplace factors such as stress, incivility, and the absence of “psychological safety”) as occupational health concerns within its regulatory jurisdiction.¹²⁰

Each expansion was individually defensible and justified on its merits. Psychological injury is real injury. Dignitary harm is real harm. Workplaces that systematically degrade employees produce measurable health consequences. But the expansion had a structural effect that exceeded the sum of its individual steps. It progressively eliminated the conceptual boundary between the machine’s jurisdiction and the totality of human experience. If psychological safety is a regulatory obligation, and if the definition of psychological safety is determined by the subjective experience of the individuals the regulation protects, then the machine’s jurisdiction is coextensive with every interpersonal interaction that occurs within an institution subject to its authority. Absent from the concept itself is any limiting principle.

Haddaway’s Administration

Each individual component of the machine’s expansion had defensible origins. The Dear Colleague letters were written by people concerned about campus sexual violence. The FTC’s data security enforcement was pursued by lawyers who believed, correctly, that inadequate corporate data security imposes real costs on real people. The LNT model is used by toxicologists who believe, with some scientific support, that it represents the most cautious available assumption in the face of genuine uncertainty. The concept creep of harm was driven by researchers and advocates who were observing real suffering that older frameworks had inadequately recognized.

The machine grew because it must. Not because anyone commanded it to grow, but because the structural features of the system reward expansion and cannot generate self-limitation.

From inside the agency, the logic emerges clearly. An agency facing a problem has two tools: formal rulemaking and informal guidance. Formal rulemaking takes years and may be vacated by a court for an analytical deficiency in a record that took years to build. Informal guidance can be issued in weeks and achieves immediate compliance through the coercive background of the agency’s enforcement authority. The rational administrator, evaluated on the number and speed of policy outcomes, uses the informal tool. The democratic constraint was real. The constraint on the constraint was equally real. The informal tool expanded to fill the space the formal tool vacated.

Consider the logic of expansion. An agency whose regulatory domain is well-defined faces budget constraints when it seeks new authority. It must demonstrate to Congress and the Office of Management and Budget that the new authority is necessary and the costs are justified. An agency whose regulatory domain is defined by a concept that can expand (harm, safety, unfair practice) faces no comparable constraint. Each new application of the expandable concept generates its own justification. The agency is not expanding its authority. It is clarifying what its existing authority already covers. The budget request is not for new jurisdiction. It is for resources to discharge existing obligations.

Consider the logic of failure. When a regulated activity produces a harmful outcome that the regulation was supposed to prevent, two responses are institutionally available. The first is to acknowledge that the regulation failed, examine why, and consider whether alternative approaches might perform better. The second is to identify the gap in the regulation’s coverage that allowed the harm to occur and propose additional rules to close the gap. The first response requires acknowledging institutional failure, which is institutionally costly. The second response converts institutional failure into institutional justification. The machine cannot fail; it can only be underfunded and underauthorized.

Built on the Foundation function of law and organized around the elimination of harm, the Safety Machine (the administrative apparatus operating through the mechanisms described in this chapter) is the logical output of a system that has every incentive to expand and no mechanism for self-limitation. The Benzene significant risk doctrine was the closest the system came to building such a mechanism into its own operating logic. Its effective neutralization by the tools described above is therefore not incidental. A system with no internal limiting principle will, over time, dissolve any limiting principle that is introduced into it. It does not happen through deliberate sabotage but through the accumulated weight of individually rational responses to individually reasonable institutional incentives.

At the institutional scale, this is what the Paradox of Expansion, described in Chapter 4, looks like. Each rule eliminated some human discretion. Each elimination produced a gap that required more rules. Each layer of rules produced more complexity. The complexity produced more demand for specialized compliance infrastructure. The compliance infrastructure produced more institutional actors whose interests were served by the complexity’s maintenance and growth. The machine is doing exactly what a system without a limiting principle does.

This is Haddaway’s Law made administrative. The institutional imperative to prevent hurt has found its permanent operating system.

Part Two asks a prior question. Before examining the machine in specific domains (Boeing’s certification apparatus, the financial compliance regime), it asks whether the machine’s governing ambition is coherent: whether the knowledge the Safety Machine requires to do what it claims to do can, even in principle, be obtained. The answer depends on two epistemic constraints the preceding chapters have only implied.

Rather than safety, the machine produces the documentation of safety. These are different products, and the difference matters.

The compliance officer who fills out the incident report, the administrator who circulates the policy that no one reads, and the board that adopts the risk framework calibrated to last year’s data each serve this function. The documentation is necessary. Without it, the institution is exposed to liability, to regulatory sanction, to the kind of public failure that ends careers. The documentation is valuable on its own terms. But documentation of safety and safety are related by correlation, not identity. The correlation is considerably weaker than most institutional actors would acknowledge.

What the Turkey knows, on the morning before Thanksgiving, is that the documentation is perfect. The record is clean. The models are current. The compliance committee has met. The policy manual was revised six months ago. Uniformly favorable, every metric the machine uses to measure its own performance signals success.

The machine is safest on the day before it fails.

Chapter 6: Wisdom of Not Knowing

Preceding chapters asked whether the Safety Machine works. This chapter asks whether it could ever work — whether the ambition is coherent.

Nine Days

On December 8, 1903, Samuel Pierpont Langley watched his Aerodrome plunge into the Potomac River for the second time. He was the Secretary of the Smithsonian Institution, a professor of astronomy, the former director of the Allegheny Observatory, the recipient of $50,000 in War Department appropriations (more than a million dollars in current value), and the most credentialed authority on the problem of heavier-than-air flight in the United States. His calculations were meticulous. His laboratory was well-equipped. His institutional backing was secure. He had already achieved a successful unmanned flight in 1896 and believed that a manned version of the same design would follow directly. The Aerodrome entered the Potomac at an angle that made the failure unmistakable. Langley’s biographers record that he never attempted another flight.¹²¹

Nine days later, Orville and Wilbur Wright flew at Kitty Hawk.

The Wright brothers had no advanced degrees, no government funding, no laboratory, and no institutional sponsor. Their professional background was bicycles. They had corresponded with Langley’s colleague, Octave Chanute, but their primary method was systematic iteration: build, test, fail, adjust, repeat. They were willing to try configurations that the credentialed experts believed would not work, because the credentialed experts were working from theory and the Wright brothers were working from friction. Their knowledge was acquired through repeated physical confrontation with the actual behavior of actual aircraft in actual air (knowledge that could not be derived from Langley’s calculations and could not be transmitted to Langley through any document Langley would have had reason to credit).¹²²

Here is the question this juxtaposition raises, and it is not a hypothetical: if there had been a Federal Aviation Administration in 1903, what rules would it have written?

You already know the answer. It would have written rules based on the best available knowledge (which, in 1903, was Langley’s knowledge). It would have consulted the leading aeronautical scientists. It would have required aspiring aviators to demonstrate that their designs met performance standards established by credentialed experts. It would have prohibited configurations that the scientific community deemed structurally unsound. It would have issued guidance specifying that aircraft for manned flight must meet the criteria established by the Langley and Chanute research programs (the criteria that the Wright brothers’ design violated in multiple respects).

And we would have lost the airplane.

Modern aviation is extraordinarily safe, and much of that safety is attributable to regulatory standards that have driven out dangerous practices and established engineering disciplines that have collectively saved tens of thousands of lives over a century.¹²³ The observation here is different and more fundamental: regulatory systems built on the best available knowledge systematically foreclose the discoveries that cannot be predicted by the best available knowledge, because those discoveries are precisely the ones that contradict what the best available knowledge says is possible.

The structural problem is what happens when law meets genuine uncertainty: not ordinary uncertainty (where you know the categories of outcome and must estimate which will occur), but the deeper uncertainty that Frank Knight identified. This is the uncertainty in which the relevant categories do not yet exist, in which no probability distribution is available because the situation is novel.¹²⁴ The Safety Machine’s institutional architecture was documented in the preceding chapter. The question this chapter asks is whether its ambitions are coherent (whether, even in principle, regulation calibrated to prevent known harms can claim to produce good outcomes in a world where the most consequential goods are not yet known).

The ancient tradition this book has been recovering understood why not and built its jurisprudence on that constraint.

Distinction That Mattered

Frank Knight published Risk, Uncertainty, and Profit in 1921 (the same year commercial radio began broadcasting in the United States, a technology that Knight’s framework, applied by regulators, could easily have suppressed before it existed). Knight distinguishes two conditions that are routinely confused.¹²⁵

Risk is the condition in which the future is unknown but the space of possible futures is known and its probabilities are estimable. The actuary calculating automobile insurance rates faces risk. Individual accidents are unpredictable; aggregate accident rates are stable enough, across a large portfolio, to price reliably. The casino operator faces risk: no individual bet outcome is certain, but the house’s long-run margin is calculable from the rules of the games and the laws of probability. In the domain of risk, mathematical tools are appropriate and often powerful. Probability, expectation, variance (these are the right instruments) because the underlying structure of outcomes is stable enough to support statistical inference.

Uncertainty is the condition in which this structure is absent. Knight used a specific technical sense: genuine uncertainty exists when outcomes involve “unique” situations (situations so novel that no frequency distribution over past occurrences offers reliable guidance about future probabilities). The entrepreneur deciding whether to build a factory for a product that does not yet exist faces uncertainty. The physician prescribing a treatment whose long-term interactions with a particular patient’s specific genetic profile are unstudied faces uncertainty. The regulator designing oversight for a technology she does not yet understand faces uncertainty. In each case, the question “what is the probability of harm?” cannot be answered rigorously because the probability space has not been determined by experience (it is in the process of being constituted by the decisions being made).¹²⁶

Knight’s distinction had direct implications for the theory of profit. In a market of pure risk, competition eliminates excess returns: calculable advantages are replicable, and replication drives prices toward equilibrium. Sustained entrepreneurial profit therefore requires something that cannot be calculated and copied: the judgment to act under genuine uncertainty in ways that turn out to be correct. Profit is the reward for successful decision-making under conditions where no algorithm could have predicted success.¹²⁷

Knight understood something that economists have been slow to extend to law: the instruments appropriate to the domain of risk fail systematically in the domain of uncertainty. Applying probability tools to situations of genuine uncertainty does not produce imprecise answers. It produces answers that are structurally misleading (answers that have the form of probability estimates but are not grounded in the frequency distributions that make such estimates valid). A risk model applied to a novel situation is not a rough approximation. It is a category error wearing the costume of rigor.

Much of what the administrative state governs sits closer to Knight’s uncertainty than to his risk. New technologies, new industries, new business models, new chemicals, new pharmaceuticals, new financial instruments (each involves combinations of factors for which historical frequency data is sparse, the range of relevant interactions is poorly understood, and the long-term consequences cannot be derived from available evidence with any defensible confidence). The regulatory response to this epistemic situation has been to treat uncertainty as risk (to demand probability estimates, to conduct cost-benefit analyses that assign expected values to outcomes that are not amenable to probabilistic treatment, and to apply precautionary standards calibrated to preventing the harms that current knowledge identifies as possible).

The result is not sophisticated risk management. It is the systematic suppression of possibilities that current knowledge cannot assess (which is to say, the suppression of the discoveries that would have extended what current knowledge can assess). Langley’s regulatory successors would have been optimizing for the known category of outcomes and would have eliminated the unanticipated category in which the Wright brothers were working.

Negative Theology

Eight centuries before Knight, Maimonides had identified the same problem in a different domain. Writing his Moreh Nevukhim (the Guide for the Perplexed) in Cairo in the 1190s, Maimonides was not addressing aviation regulation or pharmaceutical approval. He was addressing the most difficult epistemological problem in medieval theology: what can human beings reliably know about God?¹²⁸

His answer was radical in its form and has direct structural relevance to the argument of this chapter. We can know what God is not more reliably than what God is. We can say that God is not corporeal, not ignorant, not limited in power (and these negations are defensible because they rest on what we know about the conditions under which corporeality, ignorance, and limitation arise, conditions that our theological commitments require us to exclude from the divine). Positive characterizations of the divine essence (claims about what God is) exceed human epistemic capacity, because our concepts derive from creaturely experience and cannot be extended to characterize the uncreated without systematic distortion. This is the doctrine of negative theology, via negativa. In Maimonides’s formulation it is not a counsel of despair but a precise mapping of the boundary between what we can and cannot reliably know.¹²⁹

The structural parallel to regulatory epistemology is not accidental. Negative knowledge (knowledge of what not to do, what to prohibit, what constitutes clear harm) is more accessible than positive knowledge of what will produce good outcomes in situations that have not yet occurred. We can identify fraud with some confidence because fraud involves deliberate deception about existing facts (facts we can assess). We can identify certain toxicological hazards with some confidence because laboratory evidence of mechanism is available and the causal pathway from exposure to harm is established. We can prohibit these things on the basis of knowledge we actually possess.

We cannot specify in advance with confidence what the beneficial innovations of the next decade will look like, which experiments should be tried, which risk-tolerant commitments will turn out to have been wise, which apparently dangerous departures from current practice will prove to be the discoveries that advance human welfare. This positive knowledge is unavailable to regulators for the same reason that positive theology exceeded medieval human capacity: the relevant domain of inquiry has not yet been constituted by experience, and the concepts available to us derive from a world that is categorically different from the one we are trying to characterize.

Law must be built, according to Maimonides, on what human beings can reliably know, and must respect the limits of that knowledge rather than pretending to transcend them. The legal system he designed was built around the logic of tikkun ha-guf (the establishment of the conditions under which harm is prevented and order maintained) precisely because tikkun ha-guf rests on negative knowledge (the identification of clear harms) rather than on positive knowledge of what human flourishing requires in its full complexity. The scaffolding function of law (the cultivation of wisdom toward positive goods) he understood as dependent on practices that law can encourage but not specify, because the positive content of wisdom cannot be fully articulated in advance.¹³⁰

The regulatory state has inverted this structure. It has taken the positive ambition (the specification of what beneficial outcomes look like) and attempted to institutionalize it through the precautionary principle: the rule that action should be prohibited until its safety and benefit have been demonstrated. The precautionary principle is negative theology’s inverse, and like negative theology inverted, it is self-defeating.

Thalidomide

No account of pharmaceutical regulation can proceed without acknowledging thalidomide.

In the late 1950s and early 1960s, thalidomide was widely prescribed in Europe as a sedative and anti-nausea treatment for pregnant women. It was effective at what it did, and its short-term safety profile was reassuring. What manufacturers and prescribers did not know (could not have known from the available evidence) was that it was a potent teratogen: a substance that disrupts fetal development in ways that, in retrospect, the pharmacological literature could not have predicted from the compound’s molecular structure and its observed interactions in adults. Approximately 10,000 children were born with severe limb malformations before the connection was recognized and the drug withdrawn.¹³¹

The United States largely avoided this catastrophe not through sophisticated regulatory analysis but through the institutional fortitude of a single FDA medical officer, Frances Kelsey, who declined to approve thalidomide’s American distribution because she found the sponsor’s evidence of safety insufficient (specifically, her concern about a potential link to peripheral neuritis that the evidence did not rule out).¹³² Kelsey’s caution was not a probability estimate. It was a judgment that the evidence was inadequate, a negative knowledge claim: we do not know enough to approve this. She was vindicated in a way that could not have been predicted.

The vindication settled into institutional memory as the story the system tells about itself. The story is true. It is also partial. Kelsey’s caution was wise because the drug was dangerous. The institutional system has no mechanism for distinguishing that caution from the caution applied by the next official (applied to a drug that works, by an equally well-intentioned reviewer, equally unaware that the invisible graveyard is filling behind her). The system remembers thalidomide. It does not count its delayed approvals. The asymmetry is not incidental. It is the mechanism.

The thalidomide catastrophe established a vivid and lasting lesson in the political and public consciousness: the approval of an inadequately tested drug can produce catastrophic and irreversible harm. The lesson is true. It was encoded into the 1962 Kefauver-Harris Amendments to the Food, Drug, and Cosmetic Act, which for the first time required manufacturers to demonstrate not merely the safety but the efficacy of new drugs before approval (and gave the FDA broad discretion to define what demonstration was sufficient).¹³³

Thalidomide proved one proposition and has been taken to establish a broader one. The drug proved that an inadequately tested compound can cause catastrophic harm. A lesson was encoded in law. The broader inference, that cautious pre-market approval requirements consistently produce better health outcomes than faster approval processes, does not follow from that proof. Delayed approvals impose costs as real as the harms they prevent, falling on the people who needed the withheld drug during the years it was absent from pharmacies. The drugs that are delayed are not the drugs that are approved; they are drugs that might have been approved and were not. Their victims are not visible in the way that thalidomide’s victims were visible. They die from diseases that existing approved treatments failed to control, and their deaths are recorded as disease fatalities rather than as regulatory consequences.

Invisible Graveyard

The economists who have studied FDA approval times and their consequences have given the phenomenon a name: the invisible graveyard.

The most extensively documented deaths in the invisible graveyard are from heart attacks during the years when beta-adrenergic blocking agents (beta-blockers) were available in Europe but not yet approved in the United States. Beta-blockers reduce post-infarction mortality substantially; their efficacy was established in European clinical trials by the mid-1970s. FDA approval of propranolol for post-infarction use did not follow until 1978. Academic analyses of the approval delay estimated that the lag cost between 10,000 and 30,000 American lives that might have been saved had the drug been available earlier.¹³⁴ The estimates are contested; all estimates of counterfactual mortality are methodologically uncertain. But “contested” and “zero” are not the same. The invisible graveyard has occupants whose number is disputed, not occupants who do not exist.

Misoprostol, a prostaglandin analogue that reduces the risk of gastric ulceration in patients taking nonsteroidal anti-inflammatory drugs, was approved in several European countries in the early 1980s. FDA approval in the United States did not come until 1988. Patients taking NSAIDs in the interim had no access to the prophylaxis that would have prevented a portion of the serious gastrointestinal complications (including deaths) that NSAID therapy produces in a predictable proportion of long-term users.¹³⁵ Misoprostol is a drug that reduces a known and quantifiable risk; its delay did not cost lives hypothetically but statistically, in the sense that the expected value of gastrointestinal morbidity averted by earlier approval was calculable from the NSAID prescribing rates and the known incidence of serious complications.

Interleukin-2, approved in Europe for metastatic renal cell carcinoma in the early 1990s, was not approved in the United States until 1992, after a lengthy and contentious review process. The disease is aggressive; five-year survival rates without effective treatment are low. Patients who could access interleukin-2 therapy through European clinical trials had access to a treatment that produced durable complete responses in a meaningful minority. American patients without the resources to participate in overseas trials did not.¹³⁶

The structural imbalance is built into the regulatory incentive system. When an FDA official approves a drug that subsequently causes harm, that official faces the wrath of the press, Congress, and plaintiff attorneys. The approval is documented, the harm is traced to the approval, and the decision-maker is identifiable and accountable. But the FDA official who declines to approve a drug that subsequently proves beneficial has prevented a visible harm and imposed an invisible cost. No plaintiff sues for not having had access to a drug that was not approved. No congressional hearing scrutinizes the official who was too slow to approve a life-saving treatment. No journalist writes the story of the drug that could have been.

This asymmetry was identified by Sam Peltzman in his landmark 1973 analysis of the 1962 Kefauver-Harris Amendments.¹³⁷ Peltzman compared drug introduction rates before and after the 1962 amendments and found that the amendments had substantially reduced new drug introductions and had not produced commensurate improvements in the quality of approved drugs. His conclusion was that the amendments, on net, had delayed beneficial treatments by more than they had prevented harmful ones. The analysis has been contested, refined, and partially superseded by later work on drug safety, but the underlying structural observation has survived: a regulatory system that makes visible harms painful and invisible costs costless will systematically err in the direction of excessive caution.

The invisible graveyard has real occupants (the predictable consequence of applying the precautionary principle in a domain of genuine uncertainty where the costs of caution are borne by identifiable people and distributed over time in ways that make attribution politically impossible).

Precaution’s Price

The precautionary principle, in its standard formulations, holds that when there is uncertainty about whether a course of action will cause harm, the burden of proof falls on those who wish to act, not on those who wish to prevent action.¹³⁸ Where serious or irreversible harm is possible, lack of scientific certainty is not a sufficient reason for postponing preventive measures. Act when uncertain; do not wait for proof.

Cass Sunstein has made the definitive case that this principle is internally incoherent (not merely impractical but logically self-defeating).¹³⁹ The argument is precise and has not been successfully answered. For any proposed action about which there is uncertainty, there are potential harms associated with acting and potential harms associated with not acting. The precautionary principle addresses only the first set. It treats action as requiring justification and inaction as the safe default. But inaction is also an action, in any situation where something is happening in the absence of the contemplated intervention, and the harms associated with inaction are also uncertain.

Consider pharmaceutical approval. A drug whose safety profile is uncertain carries potential harm if approved and potential benefit if approved. The precautionary principle, applied to approval, counsels delay: do not approve until uncertainty is reduced. But delay also carries potential harm (the harm to patients who might benefit and cannot access the drug while it awaits approval) and potential benefit (the harm prevented to patients who might be hurt). The precautionary principle is silent about the harm from delay. Applied consistently, it would counsel delay of the decision to delay since the decision to delay is also an action with uncertain consequences. The principle dissolves into a regress.

The precautionary principle, applied to regulatory decision-making, does not suspend action pending greater certainty. It directs attention to a specific subset of potential harms (those associated with approving, licensing, or permitting) and treats the mirror-image potential harms as background noise. This is a value judgment masquerading as a neutral principle. It systematically favors the interests of people who might be harmed by action over the interests of people who might be harmed by inaction. Thalidomide’s victims were the first group. The invisible graveyard’s occupants are the second.

There is no epistemological basis for this asymmetry. From the standpoint of genuine uncertainty, the harms from acting and the harms from not acting are equally real as possibilities and equally difficult to quantify. The precautionary principle’s differential treatment of them reflects not superior epistemic access to the domain of risk but a prior political commitment (the commitment to treating visible, traceable harms as more important than invisible, diffuse ones).

Concept Creep

The invisible graveyard has expanded beyond pharmaceuticals as the precautionary principle has migrated into regulatory domains where its application is less evidently defensible.

Nick Haslam introduced the concept of “concept creep” to describe the progressive expansion of psychological harm concepts.¹⁴⁰ The same dynamics operate in regulatory contexts. Once a category of harm achieves regulatory salience, the institutional incentives favor expansion rather than contraction. Agencies derive their budgets, staffing, prestige, and career prospects from the scope of harms they address. The agency that identifies a new regulatory concern receives a new mandate; the agency that completes its mandate dissolves.

The expansion of the precautionary principle from its origin in chemical and environmental regulation to occupational licensing, securities regulation, financial services, and communications has replicated the invisible graveyard effect across multiple domains. Occupational licensing boards established to prevent identifiable harms from unqualified practitioners have expanded to license practitioners in fields where the harm from unlicensed practice is speculative and the harm from preventing entry is concrete: higher prices, reduced access, suppressed economic mobility for entrants the licensing requirements exclude.¹⁴¹ The precautionary logic produces an invisible graveyard of services not rendered, economic opportunities not realized, and social mobility not achieved.

Consider one concrete case. As of 2015, thirty-six states required cosmetologists to complete an average of 1,284 hours of training before licensure (roughly eight months of full-time instruction, more than the training required to become an emergency medical technician in most jurisdictions). A 2015 White House report, prepared under the Obama administration and drawing on research by Morris Kleiner and others, found that occupational licensing requirements reduced employment in licensed occupations by an estimated 2.85 million jobs nationally, with the costs falling disproportionately on lower-income workers, immigrants, and people with criminal records who could not meet the entry requirements.¹⁴² The licensing boards justified the requirements as consumer protection. Kleiner’s careful assessment finds negligible evidence that consumers of unlicensed cosmetology services face measurable health risks. The graveyard is populated by the aspiring practitioners who never entered the occupation, the consumers who paid higher prices, and the communities that went without services. None of these harms appears in the licensing board’s annual report. The board measures violations prevented. It does not measure livelihoods foreclosed.

The analogy to the pharmaceutical graveyard is not perfect (the stakes are different, and the degree of uncertainty varies). But the structural dynamic is the same. Visible harm (the possibility that an unlicensed practitioner will cause identifiable injury) generates political salience. By contrast, invisible harm (the services not provided, the entrants blocked, the consumers who paid more or did without) is diffuse, distributed, and traceable to no specific regulatory decision. The board that prevents entry faces no accountability for the invisible harm. The board that fails to prevent a licensing violation faces accountability that is immediate and concrete.

Before the Harm

There is a way of stating the epistemological problem at the core of this chapter that makes its legal-theoretical implications precise.

Pre-market regulation (the requirement to demonstrate safety, efficacy, or suitability before action is permitted) is epistemically ex ante regulation. It requires an assessment of what will happen before it happens. Under conditions of genuine Knightian uncertainty, this assessment cannot be grounded in the frequency data that would make it reliable. It is necessarily an assessment under genuine uncertainty (a judgment about a situation whose characteristics are not yet known).

Post-market regulation (the imposition of liability for harm after harm has been caused, or the monitoring and response to observed outcomes) is epistemically ex post. It requires an assessment of what has happened after it has happened. The epistemic situation is categorically better: the relevant facts have been observed rather than predicted, and the assessment can draw on actual outcomes rather than modeled ones. Post-market regulation can be wrong (the harm may be difficult to attribute, the liability rules may be poorly designed) but it is not wrong in the specific way that ex ante regulation under genuine uncertainty is wrong.

The legal-theoretical point is not that ex ante regulation should be abolished. Some hazards are catastrophic and irreversible in ways that justify accepting very large costs in foregone activity to prevent them: nuclear weapons development, certain classes of biological agents, pharmaceutical compounds with established catastrophic teratogenic profiles. For these, the asymmetry between harm and precaution is defensible on grounds that are independent of the epistemic argument.

The point is that ex ante regulation should not be the default for all novel activities in the name of precaution, because the epistemic conditions that would justify it (a reliable estimate of the probability and severity of potential harms) are precisely the conditions that genuine uncertainty forecloses. Where uncertainty is genuine, the costs of ex ante prohibition are more reliably estimated than the benefits: the foregone activities are known in outline, the prevented harms are not. Ex post regulation, supplemented by targeted ex ante requirements for catastrophic risks, is the epistemically appropriate default under genuine uncertainty.¹⁴³

The FDA’s evolution toward Accelerated Approval, Fast Track designation, and the Breakthrough Therapy pathway represents an institutional acknowledgment of this logic, however incomplete.¹⁴⁴ Each pathway involves trading some degree of ex ante certainty for earlier access, accepting that the remaining uncertainty will be resolved through post-market observation. The pathways are imperfect and their implementation is contested. But they represent a movement toward the epistemically appropriate structure: calibrated ex ante requirements, with continued monitoring and accountability after approval. They preserve the possibility of correction rather than demanding complete knowledge before any action is permitted.

Humble Regulator

The argument of this chapter should not be read as a counsel of regulatory passivity. Langley’s failure was not caused by excessive caution; it was caused by overconfidence in theory that contradicted accumulating experimental evidence. The problem with Haddaway’s Law is not that it tries to prevent harm (harm should be prevented where it can be identified and prevented) but that it treats prevention as the only criterion and treats uncertainty as a reason to prevent rather than as a reason for epistemic humility about what should be prevented.

Well-designed law can exhibit three structural features under genuine uncertainty.

The first is reversibility. Where possible, regulatory interventions should be designed to preserve the capacity for correction. Ex ante prohibitions that are difficult to lift are epistemically worse than ex post requirements that can be adjusted in light of evidence. Regulatory sandbox programs allow limited, supervised experimentation before permanent authorization decisions are made. They are institutionally appropriate responses to genuine uncertainty because they generate the evidence that reduces uncertainty without requiring the complete uncertainty-resolution that ex ante precaution demands.¹⁴⁵ One implementation appears in the UK Financial Conduct Authority’s regulatory sandbox, launched in 2015. Whether its design adequately protects against serious harm while permitting experimentation is a legitimate empirical question; the principle of preserving reversibility while generating evidence is sound.

Outcome monitoring with credible response constitutes the second. If ex ante assessment of novel activities is epistemically limited, the value of the regulatory system depends on the quality of the feedback loop from observed outcomes to regulatory response. This requires that the regulatory system be designed to observe outcomes that matter (not merely compliance with process requirements, which generates the illusion of knowledge without its substance) and to respond to adverse observations with specificity and proportionality. A drug approval system that monitors post-market adverse events and has the institutional capacity to act on them quickly is epistemically superior to one that demands complete pre-market certainty, not merely because it approves drugs faster but because it generates and uses better information.¹⁴⁶

The third is liability alignment. Post-market liability, designed to impose costs on actors whose activities cause identifiable harm and to internalize those costs into the ex ante decisions of those actors, is an epistemically sound instrument under genuine uncertainty because it does not require the regulatory system to know in advance what the harms will be. It requires only that the system be able to attribute observed harms to identifiable actors and impose costs accordingly. The design of liability rules under genuine uncertainty is a sophisticated undertaking (questions of causation, apportionment, and the definition of compensable harm are genuinely difficult) but the epistemological structure is favorable compared to ex ante regulation calibrated to harms that have not yet occurred.¹⁴⁷

The Loop That Learns

The argument of this chapter opens onto a further problem that the next chapter must address.

The future is genuinely uncertain in Knight’s technical sense. The precautionary principle, applied as a general default under genuine uncertainty, is internally incoherent and systematically produces invisible costs. Ex ante prohibition is the epistemically wrong default under genuine uncertainty. And law can be designed to preserve reversibility, generate better information through feedback, and align incentives through ex post liability.

The argument has also assumed, throughout, something that will not survive closer examination: that a sufficiently well-designed regulatory system could know enough to govern well. It has treated the problem as one of calibration (that with better feedback mechanisms, better evidence collection, and better liability design, regulatory systems could possess the information their functions require).

Chapter 7 rejects this assumption from the ground up. The deeper epistemic constraint is not about the future at all. Significant portions of the knowledge required to govern well are inaccessible to any central authority by nature because that knowledge is dispersed, tacit, and irreducibly local (held across millions of situated, particular, and partially inarticulate minds, existing only in the practices of people working within the situations regulation is supposed to govern). This knowledge resists aggregation into a central authority’s possession because the form in which it exists does not survive the aggregation process, and more resources cannot change the form.

The wisdom of not knowing runs deeper than the argument about time. Nobody knows what will happen next. As Chapter 7 will establish, nobody (in the relevant epistemic sense) knows fully what is happening now.

Chapter 7: Nobody Knows Everything

Chapter 6 established that the future cannot be known in advance. The present cannot be fully known from any central authority either. Knowledge required to govern well is dispersed across minds that no regulatory process can reach.

Spontaneous Order

In 1945, Friedrich Hayek published a nine-page article in the American Economic Review that is routinely described as the definitive case against socialist central planning.¹⁴⁸ Accurate as a summary, this description is repeated with confidence by people who have primarily encountered the paper as a citation. That group is large; the paper is widely assigned and less widely read. What the citation-level summary misses is what matters most for the argument of this book.

Titled “The Use of Knowledge in Society,” the paper’s target is not primarily socialism. Its target is a particular theory of knowledge: the assumption that the information required to govern complex social arrangements could in principle be collected, organized, and processed by a central authority. Hayek believed this assumption was distorting not merely economic policy but the entire intellectual project of modern social science. His argument was that centralized planning was impossible in principle (because the knowledge required does not exist in a form that centralization can reach). He believed it would also be inefficient, but impossibility was his claim.

Hayek opens with this question: “What is the problem we wish to solve when we try to construct a rational economic order?” He thereby sets the terms.¹⁴⁹ His answer was that the question had been posed incorrectly for a century. The planning debate assumed that the information required for rational coordination existed somewhere in dispersed form, and that the task of institutional design was to aggregate it into a central repository from which rational decisions could flow. Hayek argued that this assumption was false. The relevant knowledge “never exists in concentrated or integrated form, but solely as the dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess.”¹⁵⁰ Coordination’s fundamental challenge is not aggregating information that exists but using information that, by its nature, cannot be fully aggregated.

This is a claim about the structure of knowledge, not merely about the practical difficulties of data collection. If the knowledge required to coordinate a modern economy is inherently dispersed, the same is true of the knowledge required to govern a modern society. No regulatory agency, however well-staffed and well-funded, can possess the information it would need to specify in advance what every regulated actor should do in every situation. Embedded in the minds, bodies, and social relationships of the people being regulated, this knowledge cannot, in significant portions, be communicated to an outside observer at all.

Chapter 6 established the first dimension of the epistemological problem: the future is uncertain, and institutions that assume otherwise will systematically foreclose the discoveries that could only have been made by trying. The present is also, in large part, unknowable from the center. Regulators cannot know, in any complete sense, what is already happening. The knowledge required for that knowing is distributed across millions of situated, particular, and partially inarticulate minds.

Prices

Hayek’s argument for markets is, at its foundation, an argument about the processing of dispersed information. Conventionally, defenders of markets emphasize their efficiency in allocating resources toward their highest-valued uses. Hayek was making a more fundamental claim: the price system is valuable not because it is efficient but because it is epistemically possible in a way that central planning is not.

Consider what happens when a coffee crop in Ethiopia fails. Drought reduces the harvest. Farmers who sell coffee receive less for the same labor. Merchants who buy coffee pay more. Traders who ship coffee face higher costs. Café owners in New York see their wholesale prices rise. They raise their retail prices. Customers order fewer lattes. Some substitute tea. An entrepreneur detects the price signal and begins investigating alternative sourcing. A speculator buys coffee futures, betting the shortage will persist. A researcher applies for a grant to develop drought-resistant coffee plants.

None of these actors need to know that there was a drought in Ethiopia. None of them need to know anything about Ethiopian agriculture, the global coffee market, consumer preferences in New York, or the research pipeline for agricultural genetics. What they need to know, each of them, is only what is immediately relevant to their own situation: the price of the thing they buy or sell, and how that price compares to what they knew before. In compressed form, the price carries all the information each actor needs to respond appropriately to a change in conditions on the other side of the world.

This is what Hayek meant by the price system as a “mechanism for communicating information.”¹⁵¹ Rather than approximating some underlying truth a sophisticated planner could calculate independently, the price is the information, in the only form in which it can be simultaneously available to millions of actors who know nothing of each other’s circumstances. No central authority could replicate what the price system accomplishes. The information the price encodes exists only as a disposition to act, a response to local circumstances expressed in market behavior and nowhere else. No computing power reaches it.

The planning authority that tries to replace the price system with centrally determined allocations operates in a state of systematic ignorance that it cannot, by the nature of its position, remedy. The information it lacks is not sitting in a database somewhere, waiting to be retrieved. It is embedded in the situations, relationships, and tacit dispositions of the people it is trying to plan for.

Knowledge Problem

The knowledge problem manifests in multiple regulatory contexts: the regulation of professional practice, the design of administrative procedures, the setting of environmental standards, the governance of financial institutions, and the whole range of activities that constitute the modern administrative state.¹⁵²

Consider what a regulatory agency must know in order to write a good rule. It must know the range of situations in which the rule will be applied (not the average situation, but the full distribution of actual situations, which will differ in ways no drafter can fully anticipate). It must know how regulated actors will respond to the rule (not merely whether they will comply, but how compliance will interact with the other constraints and incentives they face). It must know what the rule will prevent (not merely the harms that are visible now, but the beneficial adaptations that the rule will make unavailable). It must know, at some level, what a wise actor would do in each of the situations the rule governs.

None of this knowledge is possessed by the drafting agency. Some of it does not exist in any unified form; it is distributed across the actors whose behavior the rule will affect. Some of it is tacit, accessible only to those with direct experience of the relevant situations. Some of it does not yet exist because it will only be generated by attempts to work through the situations the rule addresses. The rulemaking process (notice, comment, response, review) is an attempt to aggregate some of this knowledge from the regulated community. It is better than nothing, and it is substantially short of what the task demands: specifying, in binding text, what practical wisdom looks like in the full range of situations the rule will govern.

Rules remain fundamentally constrained in scope. At best, they are calibrated to the situations the drafters anticipated (which is a subset). At worst, they are calibrated to the situations that generated politically salient complaints (a further, and potentially quite unrepresentative, subset). A rule designed to govern a distribution of situations is calibrated away from the tails of that distribution, the situations where it will matter most.

Drafted by lawyers with expertise in financial regulation, the Dodd-Frank Act of 2010 runs to 849 pages and required hundreds of implementing rules.¹⁵³ The Affordable Care Act, which runs to 906 pages, was the product of years of legislative effort and extensive expert consultation.¹⁵⁴ Both statutes are limited in the ways they are limited because no drafting process could fully incorporate the knowledge of what is actually happening in the situations they regulate. The knowledge problem is inherent to the regulatory task itself rather than a failure of execution. It cannot be overcome through better drafting.

What We Know but Cannot Say

Hayek’s argument depends on a claim about the nature of knowledge that he stated clearly but did not develop fully. He distinguished between “the knowledge of the circumstances of which we must make use” and the “scientific knowledge” that can be systematically organized and transmitted.¹⁵⁵ Academic economics and expert planning agencies function on the assumption that understanding can be codified. The former category (knowledge of particular circumstances, of local conditions, of the specific situation at hand) is what dispersed actors possess and central authorities cannot acquire.

Two decades after Hayek’s paper, Michael Polanyi gave this distinction a name and phenomenological force.¹⁵⁶ The tacit dimension of knowledge—knowledge that cannot be fully articulated—is not a residual category or supplement to stated knowledge but the foundation itself. “We can know more than we can tell.”¹⁵⁷

Polanyi’s paradigm case is the physician who recognizes disease through years of clinical observation. Ask the experienced diagnostician how she recognizes pneumonia and she lists symptoms: diminished breath sounds, dullness on percussion, crepitation in the lung fields. But the diagnostician does more than apply a symptom list. Through years of listening to healthy and sick lungs—the lungs of smokers and nonsmokers, the fit and sedentary, the early-caught and advanced cases—she has developed a perceptual capacity that cannot be transmitted by description. She recognizes something that she cannot fully explain, in ways that someone who has only read the symptom list cannot match.¹⁵⁸

Skills are acquired through practice. The knowledge that practice generates is embodied in the practitioner (in her perceptual systems, her motor patterns, her judgment) in ways that cannot be fully extracted and transmitted propositionally. The recipe is not the cooking. The description of how to ride a bicycle does not teach you to ride one. The symptom list does not develop the diagnostic eye.

Hubert Dreyfus spent three decades investigating this phenomenon in artificial intelligence.¹⁵⁹ His argument—extravagant when first advanced in the 1970s, increasingly compelling since—was that expert performance in complex domains cannot be reduced to rule-following. The expert does not apply rules to situations. The expert perceives the situation in a way that makes the appropriate response evident. This perceptual capacity develops through stages: from the novice’s explicit rule-application to the competent performer’s pattern recognition to the expert’s immediate situational grasp. At no stage can the expert’s performance be fully specified as rules that a non-expert could follow.¹⁶⁰

Law is articulated text, rules are propositions, and whatever law can specify is necessarily limited to what can be stated. But the knowledge required to govern well in complex situations is not fully stateable. It is tacit. It exists in the situated, embodied, relational knowledge of the professionals, communities, and individuals who are actually confronting those situations. The tacit dimension cannot be rendered as rule. Any rule attempting to fully specify wise behavior in a particular domain will, in the very act of specification, replace the judgment it is specifying (and necessarily fail to capture the tacit dimension that made that judgment wise).

High Modernism

In 1998, James Scott examined what he called “high modernist” institutional projects.¹⁶¹ These were large-scale attempts to reorganize complex social and natural arrangements according to rational plans designed by central authorities. His examples include scientific forestry in nineteenth-century Prussia, Soviet collectivization of agriculture, urban planning in Brasília and Chandigarh, and compulsory villagization in Tanzania. Each project was conceived by planners with genuine commitment to improving conditions. Each produced outcomes ranging from disappointing to catastrophic.

Scott’s diagnosis explains why these projects failed: they substituted legible, abstract, formal knowledge for what Scott called mētis—practical, local, contextual knowledge that cannot be reduced to explicit propositions or transferred to a planning authority without fundamental distortion.

Nineteenth-century German foresters developed what they believed was a rational approach to timber management. The forest was surveyed, mapped, and reduced to a monoculture of the most commercially valuable trees, planted in neat rows that could be inventoried, harvested, and replanted on a predictable schedule. For a generation, the plan appeared to work. Timber yields were predictable. Management was efficient. The forest was, from the perspective of the planning authority, legible.

Then the forest began to die. Waldsterben, the German foresters called it—forest death.¹⁶² The planners had eliminated more than biodiversity. The original forest’s ecology performed essential functions that the planners did not understand because those functions had never needed articulation. They were embedded in the forest itself, in the accumulated result of evolutionary processes no planner had designed. The monoculture was efficient by the metrics the planners could measure. It was fragile against disturbances those metrics could not see.

This analogy to regulatory systems is deliberate. Scott drew it explicitly: “The simplified, legible, centralized, high-modernist order, which had worked, when it worked at all, only when backed by coercive power, was always only a partial view.”¹⁶³ Regulatory codes perform the same operation. They impose legible, abstract, general rules on situations whose relevant characteristics are particular, local, and tacit. Simple, homogeneous situations make the abstraction close enough to reality to guide action. Complex, heterogeneous, dynamically changing situations reveal the divergence. Only practitioners embedded in those situations can detect it.

Mētis is not merely uncollected knowledge. It is knowledge that can only be exercised—knowledge that exists in the practice of situational judgment and that disappears when that practice is replaced by rule-following. Rules do not merely eliminate judgment in the moment. Their elimination of practice eliminates the capacity for judgment altogether. Scott’s foresters did not merely fail to collect knowledge. By planting monocultures, they destroyed the ecosystem that embodied it.

When Rules Run Out

Wittgenstein asked what it means to follow a rule. This is a question legal theorists have yet to fully engage.¹⁶⁴

According to a naive picture, rules are instructions that determine their own application. If you know the rule, you know what to do in any situation the rule covers. The rule “drive on the right” tells you what to do on any road in any car in any condition. There is no judgment required beyond understanding the rule. This is the picture that drives the aspiration of regulatory completeness (the belief that with enough rules, enough specificity, enough explicit guidance, the need for judgment can be eliminated).

Wittgenstein showed that this picture is incoherent. Rules cannot interpret themselves. Any rule requires a background of shared practices, common understandings, and contextual knowledge to apply correctly. The rule “drive on the right” does not tell you what to do when the road is blocked, when you encounter a one-way street, when you are on a shared path with pedestrians, or when you are driving in a country where the rule is reversed. More fundamentally, it does not tell you how to recognize a situation as a “driving” situation, what counts as a “road,” or what to do when following the rule would produce an outcome the rule was never designed to produce.

Every rule presupposes a vast background of tacit understanding that the rule cannot supply. Adding more rules does not solve this problem. The new rules also require tacit understanding to apply. There is no level of specification at which judgment disappears. Judgment is not supplementary to rule-following. It is constitutive of it.¹⁶⁵

This directly applies to the compliance industrial complex documented in Chapter 4. The aspiration behind compliance regimes is the aspiration of regulatory completeness: replace the need for judgment with rules precise enough to determine their own application. As Wittgenstein’s analysis predicts, this aspiration cannot be satisfied. The more rules a compliance regime contains, the more judgment is required to apply them. A sophisticated financial services firm managing the interaction of Dodd-Frank requirements, Basel III capital standards, state banking regulations, insurance carrier requirements, and SEC guidance documents does not face less judgment than a smaller firm working under fewer rules. It faces more (because the interactions among the rules create interpretive problems that no single rule can resolve). The compliance attorney is equipped to interpret the rules’ interactions. The compliance attorney’s fee is not contingent on the quality of the underlying conduct. She bills the same rate to document the institution that managed its risks responsibly and the institution that is paying her to construct a record that will survive scrutiny of conduct it would be indelicate to describe as responsible. The priest does not evaluate the sinner. The priest validates the ritual. The fee is the same.

Compliance displaces the situated discernment of the practitioner closest to the situation. It displaces the loan officer who knows the community, the physician who knows the patient, the teacher who knows the child. It requires instead the abstract expertise of the legal specialist who knows the rules. This is backwards from wisdom: particular knowledge displaced by general knowledge, the practitioner who knows the situation replaced by the specialist who knows the law about the situation.

What Ostrom Found

The knowledge problem manifests most concretely in the destruction of relational knowledge, which develops within sustained, particular relationships between practitioners and those they serve.

Consider what a community banker knows. She knows the small business owner who has banked with the institution for twenty years—his character, work ethic, decision history, market circumstances, and the reasons his business performed poorly in 2009 and how he recovered. She knows his balance sheet understates his creditworthiness because he has run his business conservatively. She knows, in the particular, embodied, relational way that years of situated knowledge produce, whether his loan application represents a reasonable risk.

The banker’s knowledge resists quantification. She cannot reduce it to a credit score, a debt-to-income ratio, or any of the quantitative inputs that an algorithmic model will use. Her knowledge is tacit—knowledge of a particular person in a particular situation, existing only in the relationship that produced it.

The Dodd-Frank Act increased documentation and verification requirements for mortgage lending and limited lender discretion.¹⁶⁶ These requirements responded to documented abuses: lenders making loans they knew would fail, concealing this from investors. Correctly diagnosed was a failure of professional integrity in certain large institutions. But its implementation destroyed something valuable in a different category of institution: the community bank whose competitive advantage was precisely the relational knowledge that the new regime made irrelevant.

Between 2010 and 2020, the number of community banks in the United States declined by more than thirty percent.¹⁶⁷ The causes were multiple, but regulatory compliance burden was consistently identified as a primary factor. A community bank with fifty employees faces compliance costs that are, on a per-employee basis, many times higher than those of a large institution, because the fixed costs of maintaining compliance infrastructure do not scale proportionally with the size of the institution.¹⁶⁸ The rule imposed its greatest burden on institutions whose competitive advantage was professional judgment, even though it was designed to govern banks that had abandoned it.

In medicine, this dynamic operates with equal force. The physician who knows her patient—who has seen this person across multiple visits, understands the context of her symptoms, has developed the clinical relationship that enables communication—possesses exactly the tacit, relational, situational knowledge that good medical judgment requires. But modern medical administration’s documentation requirements are not designed to capture this knowledge. Instead, they are designed to document compliance with insurance requirements and regulatory mandates, and to produce defensible records in litigation. They consume physician time in inverse proportion to their relevance to the knowledge that actually determines medical outcomes.¹⁶⁹

A 2019 study found physicians spend, on average, 4.5 hours per day on electronic health record tasks, nearly as much as direct patient care.¹⁷⁰ These hours represent displaced cognitive resources: the attention, observation, and relational presence that good diagnosis requires. The physician composing documentation while the patient speaks loses the attentive, contextual exchange that phronēsis demands. She performs technē when the situation requires phronēsis.

In Determinatio

Hayek’s dispersal argument, Polanyi’s tacit dimension, Scott’s mētis, and Wittgenstein’s rule-following analysis converge on a single structural claim about the limits of regulatory knowledge.

The knowledge required to govern well in complex situations differs fundamentally from a single aggregate body of information merely distributed across inconvenient locations. Significant portions cannot, in principle, be fully centralized because they are tacit (exercised rather than stated), situational (meaningful only in particular relationships and circumstances), and performative (generated by the practice of judgment, destroyed when that practice is replaced by rule-following).

Regulatory systems that attempt to eliminate local, situated judgment are not merely inefficient. They are epistemically overreaching in a way that systematically produces the opposite of their intended effect. A rule designed to ensure fair lending treatment cannot incorporate the banker’s knowledge of the particular borrower. A standard designed to ensure adequate care cannot incorporate the physician’s knowledge of the particular patient. An algorithm designed to ensure consistent sentencing cannot incorporate the judge’s knowledge of the particular defendant.

The rule, standard, and algorithm are correct in a general sense. Calibrated to a statistical representation of situations that may not reflect any actual situation, they manage the average. And the average is a fiction: a mathematical construction that corresponds to no actual person, no actual loan application, no actual medical presentation, no actual sentencing context. The wisdom the rule is designed to ensure is the wisdom of attending to particulars, which is precisely what the rule, as a generalization, cannot do.

Aquinas understood this. Law must establish general principles while leaving space for determinatio by those acting in particular circumstances.¹⁷¹ His claim was not that determination of particulars is more efficiently performed locally. His claim was that it cannot be performed otherwise. The particular is not fully captured by the general. No formulation of general principle, however careful, can close the gap between principle and particular case. That gap is where judgment lives, and it is ineliminable.

The Price of Legibility

The regulatory state rests on an implicit premise: the gap between principle and particular can be closed. With enough rulemaking, guidance, oversight, and compliance infrastructure, the need for situated judgment can be progressively eliminated. This premise is false, and its falseness has consequences.

The first consequence is the displacement of practical wisdom. When rules require actors to behave in ways that their situated knowledge would lead them to reject, the rules do not transmit the regulator’s superior knowledge. They transmit the regulator’s ignorance of the particular situation at the expense of the practitioner’s knowledge. The result is not better outcomes but outcomes calibrated to the planner’s model rather than to the actual situation.

Equally consequential is the destruction of the epistemic ecosystem that generates the knowledge the regulatory system claims to draw on. Regulatory drafters consult experts—credentialed specialists, academic researchers, senior practitioners whose knowledge has been abstracted into teachable form. Excluded from the regulatory process, the tacit knowledge of the community banker, the situational knowledge of the experienced physician, and the mētis of the skilled practitioner never reach rule-makers. When the resulting rules displace the practices that generated the tacit knowledge, they destroy the epistemic foundation they rest on.

Third is the illusion of governance. Compliance systems produce documentation that looks like evidence of governance. A financial institution filing four million Suspicious Activity Reports annually, maintaining a compliance staff larger than its trading staff, submitting to regular examination by multiple agencies, has all the appearances of a well-governed institution. Whether those compliance activities correspond to actual knowledge about actual transactions is a different question. It is one the compliance apparatus is specifically designed not to ask, because asking it would require exactly the situated, relational, tacit judgment the apparatus was designed to replace.

The community banker who knew her customers would have flagged the shell company and approved the grandmother. The compliance algorithm does the reverse. It is functioning exactly as designed. In the design itself lies the problem. Its designers could not know what the banker knows, because what the banker knows cannot be stated, can only be practiced, and has been progressively eliminated by the regulatory apparatus that replaced it.

Where Jurisdiction Belongs

Chapter 6 established that the future is uncertain. Equally, the present cannot be known from the center because the relevant knowledge is dispersed across situated minds in a form that centralization cannot reach.

Together they describe the epistemic conditions under which the Safety Machine operates. The knowledge problem is a structural feature of governing complex human activities from a position that cannot possess the knowledge governing well requires. Better regulation, more rigorous certification, and more thorough documentation cannot overcome it.

Hayek’s insight, extended through Polanyi and Scott into legal theory, is not an argument for abolishing law or regulation. Hayek understood that markets require a legal framework. Scott found that some state planning succeeds. Polanyi treated tacit knowledge as a subject of examination. These thinkers converge on a principle: law must be designed for practitioners who possess knowledge that law cannot fully state. Law’s role in complex domains is not to specify wise behavior but to create the conditions under which practitioners who possess situated, tacit, relational knowledge can exercise it.

The classical tradition’s concept of determinatio—the specification of particulars by those acting in particular circumstances—was an architectural principle. General principles can be stated by law. Particular applications require the judgment of actors who know the particular situation. The boundary between the two is not a matter of regulatory ambition. It is a structural feature of the knowledge problem.

Modern law has largely refused this boundary. The consequence is regulatory expansion that progressively displaces the situated, relational, tacit knowledge on which good outcomes depend. The failure then justifies more rules to govern the situations the existing rules have made ungovernable. This is the Deskilling Loop at the level of the knowledge system itself: the elimination of situated judgment produces worse outcomes, which justify more rules, which eliminate more situated judgment, which produce worse outcomes still.

The next chapter puts these constraints to work on an actual case. On two mornings (October 29, 2018, and March 10, 2019) the Boeing 737 MAX encountered conditions its safety architecture had not modeled and its pilots had not been trained to handle. The machine responded exactly as designed. Three hundred and forty-six people did not survive the response.

Chapter 8: Boeing

Part One traced the philosophical genealogy of the Safety Machine. Chapters 6 and 7 established its epistemic limits: the future is uncertain, the present is partly inaccessible from the center, and the administrative structure has no internal mechanism for closing either gap. Between October 29, 2018, and March 10, 2019, both constraints operated simultaneously inside a single institutional system. The Boeing 737 MAX certification is what the theoretical argument looks like when the stakes are 346 lives.

Lion Air Flight JT610

On October 28, 2018, a Boeing 737 MAX 8 registered as PK-LQP departed Denpasar, Bali, on Lion Air flight JT043, bound for Jakarta. Thirteen minutes after takeoff, the aircraft’s left angle-of-attack sensor began transmitting a false reading. The sensor measures the angle between the oncoming air and the aircraft’s nose. It reported that the nose was pitched steeply upward, the precursor to an aerodynamic stall, when in fact the aircraft was climbing normally at a reasonable attitude.

The flight management computer received this false signal and activated the Maneuvering Characteristics Augmentation System. MCAS commanded the horizontal stabilizer, the movable surface at the tail that controls pitch, to rotate nose-down. The control column in the cockpit pushed forward against the pilots. The aircraft’s nose began to drop.

The crew responded with the runaway stabilizer memory item, a procedure pilots are required to know without reference to a checklist. It involves identifying uncommanded stabilizer movement, disconnecting the autopilot if engaged, counteracting the movement with control column force, and throwing two switches on the center pedestal, the stabilizer trim cutout switches, which kill power to the electric trim motor. The crew executed this procedure. The stabilizer stopped moving. The crew hand-flew the aircraft to Jakarta and landed without further incident.

The aircraft was released from maintenance the following morning with a replacement angle-of-attack sensor. Maintenance records would later indicate that the replacement sensor had not been calibrated correctly, though the full significance of this was not apparent until after the accident investigation.

On October 29, 2018, PK-LQP departed Jakarta as Lion Air flight JT610, bound for Pangkal Pinang. The crew that morning had not been on the previous day’s flight. The crew had no direct knowledge of what the prior crew had encountered. Thirteen minutes after takeoff, the same faulty sensor transmitted the same false signal. MCAS activated. The stabilizer drove nose-down. The control column pushed forward.

The crew of JT610 fought the aircraft for the eleven remaining minutes of the flight. At 6:31 local time, the aircraft struck the Java Sea at high speed. All 189 people aboard were killed.¹⁷²

The question the accident investigation would eventually answer is not why PK-LQP crashed on October 29 but why it did not crash on October 28. The answer is that the crew of JT043 had a third pilot occupying the jump seat, an off-duty pilot riding to Jakarta as a passenger. He had recently reviewed the runaway stabilizer procedure and prompted the crew when the anomaly began. His presence on that flight, and his absence the following morning, accounts for the difference in outcome.

The safety architecture of the Boeing 737 MAX produced a situation in which 189 lives depended on the seating assignment of a passenger. The certification process, the software design, the system safety analysis, and the operator notification protocols—each element of the system—had contributed to this outcome.

For thirteen days after the crash, the global 737 MAX fleet continued scheduled operations. The Indonesian air accident investigation was underway, but its findings had not yet been made public. The aircraft continued to be delivered to airlines, accumulated more flight hours, and carried more passengers. Both Boeing and the FAA treated the Lion Air crash as a localized incident: an aircraft that had experienced a sensor malfunction and inadequate maintenance rather than a systemic design vulnerability. Neither Boeing nor the FAA issued a grounding order. The aircraft type continued to be viewed as fundamentally sound, the failure as operator error or maintenance failure rather than as a warning about the system’s design.

On November 6, 2018, eight days after the crash, Boeing issued an Operations Manual Bulletin to 737 MAX operators addressing uncommanded nose-down stabilizer trim. The bulletin did not name MCAS. The bulletin reiterated the runaway stabilizer memory item, the procedure that had enabled the crew of JT043 to recover when the system first malfunctioned. Between the moment the aircraft struck the Java Sea and the moment Boeing notified operators, no formal mechanism in the regulatory system had asked whether this crash represented a singular failure or a window into a design vulnerability that could recur. No airworthiness directive grounded the fleet. The system had received its first warning and responded with instruction: follow the procedure you already know.

MCAS

The 737 MAX entered development in 2011 in response to a competitive threat that Boeing had not anticipated. Airbus had committed to re-engining its A320 family with CFM LEAP high-bypass turbofans, offering airlines fuel efficiency improvements of roughly fifteen percent over previous generation aircraft. The A320neo would be certifiable under existing pilot type ratings, meaning airlines could transition their pilots without simulator retraining costs. It entered service in January 2016 and accumulated substantial order backlogs quickly.¹⁷³

Boeing faced a choice: develop a new narrowbody aircraft, which would require the better part of a decade and investments in the tens of billions of dollars, or re-engine the existing 737 to match the A320neo’s economics. The 737 had been in continuous production since 1968 and was the most commercially successful aircraft in history. Boeing chose to re-engine it.

The choice created an engineering constraint. The 737’s landing gear is shorter than the A320’s, a legacy of the original design’s specification for ground-level engine servicing. The new LEAP-1B engines are substantially larger in diameter than the CFM56 engines on the previous 737 generation. To achieve adequate ground clearance, Boeing’s engineers moved the engines forward and upward on the wing, further from the fuselage centerline and higher relative to the wing’s leading edge than on previous 737 variants.¹⁷⁴

This repositioning altered the aircraft’s aerodynamic behavior. At high angles of attack, when the nose is pitched steeply upward and the aircraft is flying slowly relative to its stall speed, the engine nacelles generate additional lift. This lift acts forward of the aircraft’s center of gravity, creating a pitch-up moment: the nose tends to rise further as angle of attack increases, rather than returning to a lower attitude as the pilot would expect. The effect is most pronounced at low speed with flaps retracted, the configuration in which a pilot reducing speed during a flap-up maneuver would experience handling characteristics different from previous 737 variants.¹⁷⁵

Boeing’s certification path depended on maintaining the MAX’s status as a variant of the existing 737 type certificate rather than a new aircraft type. A new type certificate would have required airlines to train their MAX pilots on a new type rating, incurring simulator costs and transition time. The commercial value of the MAX to Boeing’s airline customers rested substantially on the representation that a pilot current on previous 737 variants could fly the MAX without a new type rating, with differences training accomplishable on a tablet computer rather than in a simulator.¹⁷⁶

The aerodynamic handling difference created by the engine repositioning was incompatible with this commercial requirement. If pilots noticed that the MAX handled differently than previous 737s at high angles of attack, they might push back on the no-simulator-training representation, and regulators might agree.

MCAS was the solution. The Maneuvering Characteristics Augmentation System would monitor the aircraft’s angle of attack through its sensor inputs and, if the angle of attack exceeded a threshold with flaps retracted, automatically command the horizontal stabilizer to pitch the nose down. MCAS would activate without pilot input and without appearing on any pilot display. It would mask the handling difference. From the pilot’s perspective, the MAX would feel like previous 737 variants.¹⁷⁷

Flight management software that augments handling characteristics is standard. What made MCAS distinctive was the layering of subsequent decisions around it.

Boeing Certifies Boeing

The Federal Aviation Administration certifies commercial aircraft through delegation. Beginning in the 1990s and accelerating through the 2000s, the FAA developed the Organization Designation Authorization program. Under the ODA, Boeing employees designated as authorized representatives perform certification work on the FAA’s behalf—reviewing engineering analyses, witnessing tests, and making compliance determinations, subject to FAA oversight.¹⁷⁸

The rationale is straightforward. The FAA cannot employ engineers with enough current expertise in every technical specialty to independently review a modern aircraft certification. Boeing employs thousands of engineers with deep specialization in its aircraft systems; the FAA employs hundreds. Using Boeing’s engineers for detailed technical review is more efficient than hiring additional FAA staff, and Boeing’s engineers know more about Boeing aircraft than the FAA’s generalists can know independently.

The Senate Committee on Commerce, Science, and Transportation’s 2020 investigation found that the FAA had delegated up to eighty-seven percent of the MAX’s certification work to Boeing.¹⁷⁹ Boeing employees performing ODA functions reported pressure from management to limit their time on specific review items and to avoid raising issues that might delay the certification schedule.¹⁸⁰

Within this architecture, Boeing’s engineers made a series of decisions about how to characterize MCAS to the FAA. Each decision reflected defensible judgment individually. Collectively, they produced a safety analysis that described a different aircraft than the one being certified.

The first decision was classification. Boeing characterized MCAS as a modification to the existing 737 speed trim system, a system that adjusts the stabilizer trim in response to changes in speed, rather than as a new flight control function. This characterization placed MCAS below the threshold for the most rigorous categories of FAA review. It allowed MCAS to proceed through the certification process without the level of scrutiny a novel flight control system would have required.¹⁸¹

The second decision was authority. In the original MCAS design, the system could move the horizontal stabilizer a maximum of 0.6 degrees per activation. During development, Boeing engineers expanded MCAS authority to allow larger nose-down inputs. The system ultimately could move the stabilizer significantly more than in the original specification, with repeated activations possible.¹⁸² The System Safety Analysis—the document classifying the severity of potential system failures and analyzing the adequacy of safeguards—was not updated to reflect this expansion.

The gap was determinate. The System Safety Analysis evaluated runaway MCAS activation based on the original, limited-authority design. A runaway activation of the original MCAS would produce a small, potentially correctable pitch disturbance. The expanded MCAS could produce an uncontrollable pitch-down exceeding the pilot’s physical ability to counter. The stabilizer could move significantly more per activation. The system could re-activate after the pilot countered its first input. The analysis evaluated the first scenario. The aircraft contained the second.¹⁸³

The third decision was sensor redundancy. The final MCAS design relied on a single angle-of-attack sensor. Modern commercial aircraft have two or more AOA sensors. Systems using AOA inputs typically compare them for consistency before acting. MCAS did not. A failed or miscalibrated AOA sensor transmitting false high-angle-of-attack data would produce uncommanded nose-down stabilizer movement with no internal check. This is the scenario that killed 346 people.¹⁸⁴

The fourth decision was disclosure. MCAS was not described by name or in detail in the 737 MAX Flight Crew Operations Manual. Boeing maintained that MCAS was a background system operating outside the normal flight envelope, and that pilots did not need to know about it in detail to fly safely. The runaway stabilizer memory item—already in the manual—would trigger on MCAS activation. Pilots experiencing the malfunction were expected to recognize it as a runaway stabilizer event and respond with the existing procedure.¹⁸⁵

The flaw the Ethiopian Airlines accident exposed was this: the existing procedure assumed that once the electric trim motor was cut out, the stabilizer would stop moving and pilots could manually trim using the trim wheels. MCAS, at the high airspeed the Ethiopian crew reached by the time they performed the cutout, had moved the stabilizer to a position at which the aerodynamic loads on the jackscrew made manual trimming physically impossible. The procedure was designed for a different scenario than MCAS created.

Not Fully Informed

The internal communications that the House Committee on Transportation and Infrastructure investigation recovered and made public in 2019 and 2020 document the state of knowledge within Boeing about MCAS during the period preceding the Lion Air crash.

In January 2016, a Boeing test pilot evaluated the 737 MAX with Japan Civil Aviation Bureau representatives. In a subsequent internal message, he wrote that he had understated MCAS’s authority. He was not fully informed about the expanded design. “I basically lied to the regulators (unknowingly),” he wrote. “I have been flying with MCAS engaging and I’ve been flying a sim that doesn’t have it, and when we were at [JCAB], I didn’t know enough about the pitch augmentation law.”¹⁸⁶ The message was sent in 2016, two years before the Lion Air crash.

In September 2018, a month before the Lion Air crash, a Boeing employee wrote in an internal message: “This airplane is designed by clowns who are in turn supervised by monkeys.”¹⁸⁷ The context reflects frustration with the development process in response to a question about simulator training effectiveness. The message was sent in the same month that the first operational crash would occur, by someone involved in the program. It does not reflect technical knowledge of the MCAS failure mode.

In a separate message, a Boeing test pilot wrote about the 737 MAX simulator training that was being prepared for airline customers: “Would you put your family on a MAX simulator trained aircraft? I wouldn’t.”¹⁸⁸ The message was sent in 2018.

These messages do not establish that Boeing engineers knew MCAS would cause the specific failure modes that killed 346 people. They establish that people positioned to know the aircraft’s systems and training expressed serious concerns about its safety margins immediately before and during the first flights. The institutional processes that should have surfaced these concerns, evaluated them, and either resolved or escalated them did not do so effectively. The aircraft’s design and training requirements did not change.

Uncommanded Nose-Down Trim

On November 6, 2018, eight days after the crash, Boeing issued an Operations Manual Bulletin describing uncommanded nose-down stabilizer trim and reiterating the runaway stabilizer memory item.¹⁸⁹ The bulletin did not name MCAS or identify the system that would produce it, or explain why a single failed AOA sensor would trigger repeated activations.

The FAA issued an Emergency Airworthiness Directive three days later that referenced the Boeing bulletin and added a requirement that operators update their quick reference handbooks. The directive did not ground the fleet.¹⁹⁰

The 737 MAX continued scheduled operations. At the time of the Ethiopian Airlines crash, more than 370 aircraft were in revenue service with thirty-six operators worldwide.

The accident investigation reports found that the operators who received the bulletin and whose pilots trained on the updated runaway stabilizer procedure had not been told that the scenario MCAS created differed materially from the runaway stabilizer scenarios the procedure was designed for. MCAS could activate repeatedly. The stabilizer could move significantly after the electric cutout. The procedure had not been designed for these conditions.¹⁹¹

Ethiopian Airlines Flight 302

Ethiopian Airlines Flight 302 pushed back from gate D5 at Addis Ababa Bole International Airport at 8:38 AM on March 10, 2019. The aircraft was a 737 MAX 8, registration ET-AVJ, delivered four months earlier. Captain Yared Getachew, twenty-nine years old with approximately eight thousand total flight hours, and First Officer Ahmednur Mohammed, twenty-five years old with approximately two hundred hours, occupied the flight deck.¹⁹²

Flight 302 was cleared for departure on runway 07R. The crew advanced the throttles to the takeoff power setting and the aircraft accelerated normally. At 8:38:44, ET-AVJ lifted off and began climbing.

Four seconds after liftoff, the left angle-of-attack sensor began transmitting data that diverged from the right sensor’s reading. The left sensor’s reading was 74.5 degrees higher than the right sensor’s reading. This disparity indicates a sensor fault rather than an actual flight condition. An angle-of-attack of 74.5 degrees above the actual value represents a physical impossibility in normal flight.¹⁹³

At 8:38:44, the stick shaker on the captain’s side activated. The stick shaker is a mechanical device that vibrates the control column to warn of an imminent stall. It was responding to the left sensor’s false reading. The right seat’s stick shaker did not activate; the right sensor was reading correctly.

The captain called for the first officer to engage the autopilot. The autopilot disconnected almost immediately, refusing to engage because the sensor disagreement exceeded its logic thresholds.

At 8:40:22, MCAS activated. The system received the false high-AOA signal from the left sensor. MCAS used only that single input; no disagreement check constrained it. The system commanded the stabilizer to move nose-down. The stabilizer moved from 2.3 degrees to 1.0 degree of nose-up trim. The nose dropped. The captain applied back pressure to the control column to hold the aircraft’s attitude.

Fifteen seconds later, MCAS activated again.

The crew were applying maximum back column force. The aircraft was pitching down. The airspeed was increasing rapidly. The aircraft was accelerating rather than climbing, as the nose-down attitude and near-maximum thrust were combining to trade altitude for speed.

At 8:40:41, the crew performed the stabilizer trim cutout. They moved both stabilizer trim cutout switches on the center pedestal to the cutout position, killing power to the electric stabilizer trim motor. MCAS could no longer command stabilizer movement. The stabilizer was frozen at its current position.

The current position was 2.1 degrees nose-down.

With the stabilizer trimmed nose-down and the aircraft accelerating through high airspeed, the crew were carrying significant back column force to maintain a climbing attitude. They attempted to manually trim the stabilizer using the trim wheels on the center pedestal. The trim wheels were turning, but the aerodynamic loads on the jackscrew, the mechanism that translates the wheel’s rotation into stabilizer movement, were too great for manual override. The stabilizer did not move.

At 8:43:11, the crew restored power to the electric trim motor by returning the stabilizer trim cutout switches to their normal position. The documentation available from the cockpit voice recorder and flight data recorder does not resolve whether this decision reflected a specific plan or a response to the physical impossibility of manual trimming. Both pilots applied nose-up electric trim inputs in the seconds after restoring power. MCAS reactivated and commanded nose-down stabilizer movement that exceeded the crew’s trim inputs.

At 8:43:57, the flight data recorder recorded its last data point. ET-AVJ had reached a speed of approximately 500 knots indicated airspeed at an altitude of approximately 8,000 feet, descending at a rate consistent with an aerodynamically uncontrolled dive. At 8:44 AM, the aircraft struck a field near the town of Bishoftu, approximately 62 kilometers southeast of Addis Ababa.¹⁹⁴

The impact excavated a crater ten meters deep. All 157 passengers and crew were killed.

346 Dead

The combined death toll from Lion Air 610 and Ethiopian Airlines 302 was 346 people from thirty-five countries. ET302’s manifest included aid workers, academics heading to a UN Environment Assembly meeting in Nairobi, families, a Canadian author, a Kenyan bishop, and staff of the UN Food and Agriculture Organization. Lion Air 610 carried Indonesian business travelers, civil servants, and holidaymakers returning from Bali.

The causes became clear within weeks of the Ethiopian crash. MCAS, relying on a single angle-of-attack sensor, repeatedly commanded nose-down movement in response to false readings. Both aircraft entered unrecoverable pitch-down conditions. Both crews could not identify the system that caused the crash from the aircraft documentation. Both had training consistent with Boeing’s claim that MAX pilots required only tablet-based differences training, not simulator time. Neither was prepared for the failure mode MCAS created.

The FAA grounded the 737 MAX fleet on March 13, 2019, three days after the Ethiopian crash. Aviation authorities in China, Ethiopia, the European Union, and Canada had grounded it first.¹⁹⁵ The aircraft did not return to commercial service until November 2020, following extensive redesign of MCAS, new certification requirements, and mandatory simulator training. Simulator training was the training Boeing had represented was unnecessary.

Boeing entered a deferred prosecution agreement with the U.S. Department of Justice in January 2021, admitting that two employees had misled the FAA about MCAS. The agreement required $2.5 billion in total obligations: $243.6 million in criminal penalties, $1.77 billion to airline customers, and $500 million to crash victim beneficiaries.¹⁹⁶ In 2024, the Department of Justice moved to require Boeing to plead guilty to conspiracy to defraud the United States, finding that Boeing had not complied with the agreement’s remediation requirements.¹⁹⁷ The violation: Boeing failed to implement a genuine corporate compliance program. It produced documentation of a compliance program without the program itself. The machine, corrected, had produced the same output.

Oops

Chapters 6 and 7 identified two limits on what regulatory systems can know. Boeing and the FAA, between the aircraft’s first flight and its grounding, encountered both simultaneously.

The first is Knightian uncertainty about the future. The System Safety Analysis had evaluated the MCAS failure mode that killed 346 people and then misclassified it. The analysis modeled a runaway MCAS malfunction based on the system’s original, limited authority and classified it as non-catastrophic. It did not model the expanded authority that engineers had built into production. The analysis considered risk and classified it wrong—a risk model applied to a system the model had never actually evaluated. The safety determination had the form of rigor without its substance.

The Benzene doctrine required agencies to establish that a risk was significant before regulating. Boeing’s System Safety Analysis established which risks were significant for FAA purposes. The analysis was wrong about MCAS authority. Everything downstream rested on a foundation that did not correspond to the aircraft being certified: the certification, the training requirements, the pilot manual, the absence of redundancy requirements.

The second limit is the Hayekian dispersal of present knowledge. The engineers who expanded MCAS authority knew the system’s design. The engineers who performed the System Safety Analysis knew the analytical framework for safety classification. The FAA personnel overseeing ODA had neither the specialization in MCAS nor the capacity to independently verify the analysis. The ODA delegation—eighty-seven percent of certification work delegated to Boeing—was not a corruption of the safety system. It was the safety system, designed on the accurate premise that manufacturers’ engineers know more about their systems than the FAA’s generalists.

Boeing’s engineers did know more about MCAS than the FAA’s reviewers. What the delegation could not accommodate was people who knew the most about the system’s design being embedded in an organization that made full disclosure of safety-relevant changes institutionally costly. The people responsible for verifying the analysis could not independently do so.

The pilot who wrote in 2016 that he had misled a foreign regulator about MCAS did not do so from dishonesty. He did not fully know what MCAS was. The tacit knowledge of what MCAS actually did in the expanded, production configuration was dispersed across the engineers who made the authority expansion, the engineers who updated the software, and the engineers who reviewed the change. No single person held it in integrated form. It did not reach the pilots who flew the aircraft, the pilots who evaluated it, or the FAA personnel who reviewed the certification package. The ODA structure had no mechanism to aggregate and verify it.

The relational knowledge held by the crew of JT043—who had encountered MCAS malfunction and survived—did not reach the crew of JT610. Maintenance records were reviewed and the sensor was replaced. That the prior crew had needed a third pilot in the jump seat to execute the procedure in time was not a finding any formal system recorded.

Jackscrewed

The 737 MAX certification illustrates the Safety Machine at its most complete. Every feature the preceding chapters describe appears in this one story.

The machine produced thorough documentation. The certification record was comprehensive. The System Safety Analysis was formally complete. The ODA review followed established procedures. Boeing’s engineers submitted what the process required. The FAA’s authorized representatives reviewed according to protocol. The aircraft received its type certificate through a rigorous process by the machine’s own standards.

The documentation was wrong about the aircraft. The System Safety Analysis evaluated a less powerful version of MCAS. The pilot manual described symptoms without identifying cause. The training standard required no simulator time in scenarios the aircraft’s own engineers knew could occur. The aircraft that received a clean certification was not the aircraft described in the documents.

MCAS, in production configuration, was the documentation’s fiction made operational. The organization told itself through its safety analysis what it had built. It then built something different through iterative engineering decisions that each seemed routine, without updating the story it had told itself. This is institutional self-deception rendered in software: not a conspiracy but a drift, each step defensible in isolation, the aggregate lethal. The aircraft flew 346 people on the basis of what the paperwork said it was.

The Senate investigation, the House investigation, and the accident reports identify specific decisions by specific people that contributed to specific failures. What made those decisions dangerous was not their individual character but the architecture that made their consequences invisible until 346 people were dead.

The ODA delegation was rational. The FAA cannot employ enough aerospace engineers to independently review a modern aircraft certification. Delegation to manufacturer engineers is the only practical way to manage the scale. But the delegation created an arrangement in which the people responsible for identifying safety-relevant design changes were also subject to schedule and commercial pressures that made disclosure costly. The system had no mechanism for resolving this conflict. It had a mechanism for documenting that the conflict had been managed.

The MCAS authority expansion was a technical decision made during development, when design changes are routine and formal updating of every dependent analysis is not standard. Engineers logged the expansion in engineering change orders and did not communicate it to the System Safety Analysis team in a way that triggered re-evaluation of hazard classification. The gap between the design analyzed and the design built was not concealed. It was an artifact of how complex systems develop: iteratively, across teams, with formal documentation that lags engineering reality during active development. A formal safety review gate before certification would have closed the gap. The gate that existed allowed the MAX to proceed without re-evaluating a hazard classification based on an earlier design.

The training standard was commercially rational. Simulator training for the MAX would have required airlines to pay for simulator time and absorb crew transition costs. The aircraft would be less competitive. Boeing’s representation that the MAX required no new type rating was a business decision embedded in a technical claim: that handling characteristics were close enough to previous 737s that experienced 737 pilots could fly it safely without simulator orientation. MCAS existed specifically to make that claim defensible. When MCAS malfunctioned, it produced a situation that experienced 737 pilots, with no simulator training on MCAS, could not manage with the procedures available to them.

The machine described in Chapter 5 has no internal limiting principle. Its expansion is the rational response of each institutional actor to the incentives they face. Boeing’s engineers solved engineering problems within commercial constraints. The FAA’s ODA program accommodated the knowledge asymmetry that makes comprehensive independent review impossible. The training standard was the minimum consistent with the commercial promise Boeing had made to its customers. Each decision was locally rational. The system they composed was lethal.

The machine produced what the preceding chapters predict: documentation of safety without its substance. The 737 MAX certification dossier established to the machine’s satisfaction that the aircraft was airworthy. The aircraft killed 346 people before that satisfaction was revised.

Nineteen minutes after takeoff, the knowledge that mattered was in the cockpit of ET-AVJ: the captain’s trained response, the first officer’s trained response, the procedures in the memory items, the position of the stabilizer trim cutout switches. The aerodynamic loads on the jackscrew were approximately 4,500 pounds. Two people pulling the manual trim wheels with their full strength could generate approximately 200 pounds of torque. The gap between those numbers is the endpoint of a system that had removed, step by step, every check that might have caught the decisions leading to that moment.

This is Haddaway’s Law carried to altitude. The institutional imperative to prevent hurt produced a certification architecture that satisfied itself at every stage that the aircraft was airworthy. Documentation was thorough, processes were followed, delegations were made according to procedure. The aircraft killed 346 people in the gap between what the machine had certified and what the aircraft did.

Part Three will ask what institutions would look like if they were designed to acknowledge this gap rather than paper over it. The answer requires examining what the traditions this book has been recovering actually built, not as theological curiosities, but as specifications for institutions that understood, long before Boeing, that the machine cannot know everything, that knowledge disperses, and that the gap between what the documentation says and what the aircraft does is precisely where people die.

Chapter 9: The Safety Machine

Boeing was the acute failure: two crashes, 346 dead, a grounding. The financial compliance apparatus is the chronic one. It distributes its costs across millions of transactions daily in ways that generate no catastrophe and therefore no investigation. The system spends $274 billion annually, files 4.6 million reports into an unprocessable database, and performs precisely the function it was designed to perform. The problem is what that function is.

Chronic Conditions

U.S. and Canadian financial institutions spend $61 billion annually monitoring transactions for financial crime. Globally the figure is $274 billion. The monitoring generates 4.6 million Suspicious Activity Reports filed annually with the Financial Crimes Enforcement Network. Law enforcement agencies query the database when investigating specific actors. Academic analysis of effectiveness (what fraction of those filings contributes to prosecution) places the number well below one percent.

The gap between expenditure and outcome is the system’s designed output.

The 737 MAX crashes concentrated their costs: 346 dead in two mornings, two aircraft, two investigations, a grounding order, a deferred prosecution agreement. The financial compliance system distributes its costs across millions of transactions daily in ways that produce no individual catastrophe attributable to any specific failure. The grandmother subjected to a ninety-minute compliance interview does not die. The shell company routing three million dollars through a correspondent bank to a favorable jurisdiction does not appear in an accident report. The failures are real and the costs are substantial, but they surface as a diffuse tax on legitimate economic activity and a diffuse immunity for sophisticated illegitimate activity, not as an event that triggers a Senate investigation.

The mechanism is the same. Documentation is produced that corresponds imperfectly to the underlying reality it purports to represent. The system generating the documentation has no internal mechanism for closing that gap. This chapter examines why.¹⁹⁸

Bank Secrecy

The Bank Secrecy Act, enacted in 1970, and its amendments established the modern financial monitoring framework. Financial institutions must identify customers, monitor transactions for suspicious patterns, and file reports with the Financial Crimes Enforcement Network when detecting activity that might indicate money laundering, terrorist financing, or other financial crimes.¹⁹⁹ The architecture is rational. If law enforcement cannot monitor every transaction, requiring financial institutions to do so and report suspicious findings creates a distributed network extending the state’s reach into the financial system.

The design assumes two things: financial institutions can distinguish suspicious from legitimate transactions with enough accuracy to make reporting useful, and law enforcement will review and act upon reports in proportion to their evidentiary value.

Both assumptions have collapsed.²⁰⁰

The first collapses when a compliance officer reviews transaction patterns. Armed with general training in indicators, the officer faces a transaction containing amount, counterparty, timing, and account history. But this transaction record lacks the community banker’s knowledge of a customer’s twenty-year relationship, the character of that business, the reason patterns have changed, or the contextual knowledge a situated human being would need to distinguish legitimate from suspicious.

The second fails: FinCEN received 4.6 million Suspicious Activity Reports in fiscal year 2023.²⁰¹ The volume reflects expanded reporting scope and more reporting institutions. Law enforcement review capacity lags far behind filing volume. Academic analyses estimate that SAR filings contributing to successful prosecution represent somewhere below one percent, possibly well below.²⁰² The reports are filed, stored, and theoretically available. The bottleneck is not the reporting requirement but the ratio of filed reports to investigative capacity, which the volume has made unmanageable.

The machine requires not the identification of financial crime but the production of documentation indicating that financial crime monitoring has occurred. These are different activities, performed by different processes, and optimized by different incentives.

Lemon Problem

In 1970, George Akerlof published “The Market for ‘Lemons’: Quality Uncertainty and the Market Mechanism.”²⁰³ The paper, which would earn its author a Nobel Prize, analyzed a puzzle standard economic theory could not explain: why do used-car markets behave so badly?

Information asymmetry. Sellers know more about their cars than buyers. A buyer cannot readily distinguish a reliable vehicle from one poorly maintained or mechanically flawed. Rational buyers offer a price reflecting the average quality available: adequate for poor-quality cars (“lemons”) but inadequate for high-quality cars. High-quality sellers withdraw. The market fills with lemons, driving average quality and price down, driving out more sellers, until the market collapses or settles populated almost entirely by lemons.

Any market in which one party has systematically better information about quality than the other is susceptible to adverse selection. The party with inferior information cannot distinguish good from bad and prices accordingly. The good drives out the bad at the offered price. The market fills with the bad.

The Akerlof inversion applies to the financial compliance apparatus as follows:

The monitoring system flags transactions for indicators of financial crime. These indicators are proxies: observable characteristics correlating imperfectly with the underlying phenomenon. Large cash transactions, structuring to avoid reporting thresholds, geographic anomalies, volume inconsistent with account history, and counterparties in high-risk jurisdictions—these are the signals. They are observable, automatable, checkable against lists, comparable against benchmarks.

Criminal enterprises have the same access to the indicator list. Criminal enterprises operating through the financial system employ compliance counsel.²⁰⁴ The attorney advising the criminal enterprise engages in the same profession as one advising the legitimate institution. The indicators are the same. The guidance documents are the same. The clients differ. The fee is the same. Criminal enterprises structure transactions to produce scores below monitoring thresholds. They use shell companies that appear legitimate. They establish account histories that create the baseline against which deviations are measured. They keep activity within normal variation so no deviation triggers a report. They produce documentation of legitimate financial activity.

Financial sophistication correlates with resources, legal counsel, and institutional access—characteristics of both large-scale criminal operations and large legitimate actors. The monitoring system, calibrated to detect indicators that unsophisticated actors exhibit, is better positioned to generate reports on the unsophisticated.

The actors whose activity produces anomalous signals are not, in the main, sophisticated criminals. Consider an elderly person wiring money to a foreign country for the first time in a forty-year relationship—they produce anomalous data. Immigrants sending remittances to family in a high-risk jurisdiction produce patterns matching monitoring indicators. Small businesses receiving several large cash payments in cash-intensive industries produce structuring-adjacent patterns. These are legitimate transactions that read to the compliance algorithm as suspicious.

The compliance apparatus cannot distinguish between them. This is not a design flaw. Rather, the machine produces reports about transactions matching indicators. Both illegitimate and legitimate activity match the indicators in proportions the designers cannot fully specify. What is observable is whether the indicator was present. The machine monitors for indicators. It reports on indicators. It cannot do more without the situated knowledge the monitoring regime has systematically displaced.

Flagging Grandma

Consider two transactions at the same financial institution’s compliance system on the same morning.

The first is a $500 wire from the personal account of a woman in her eighties to an organization in the Caribbean not in her compliance file. She has banked here for four decades. Her account activity has been stable: Social Security deposits, small grocery and pharmacy charges, periodic transfers to a nearby account. But the wire represents a significant deviation from history. The recipient is in a high-risk jurisdiction. This is her first international wire.

When the algorithm flags the transaction, a customer service representative calls and asks the purpose. She says it is a donation to a hurricane relief organization she heard about on the news. The representative routes to the compliance department. A compliance officer reviews the account, notes the deviation, and schedules an in-branch meeting. The woman comes in with a news printout and identification, waiting ninety minutes for an interview. She answers questions about the source of her retirement savings, the basis for her charitable intent, and her relationship with the recipient. The institution files the wire and, depending on threshold criteria, may or may not file a Suspicious Activity Report.

The second is a $3 million wire from a Delaware LLC with nominee directors at a registered agent in Wilmington. Eight months earlier, the LLC was established and made several transfers to the same institution, establishing a baseline pattern. The wire routes to a correspondent bank in Luxembourg, then to an entity in a jurisdiction the beneficial owner selected for favorable reporting requirements. The transaction is within normal variation for the pattern. The beneficial owner’s compliance attorney reviewed the structure before execution.

The algorithm does not flag it. It clears in three seconds.²⁰⁵

This is the asymmetry the monitoring system produces. The system detects deviations from pattern and advantages actors who establish the right patterns. The woman deviates from her pattern through a legitimate transaction. The LLC does not deviate because its pattern was established correctly. The system cannot distinguish between them. The interview falls on the person least equipped to handle it.

The interview’s function is to gather information the algorithm cannot observe: the customer’s explanation for the anomaly. For legitimate anomalous transactions, the interview is an imposition. For illegitimate transactions that are algorithmically unremarkable, no interview is triggered.

The Machine’s Price Tag

The financial crime compliance industry generated $274 billion in global expenditure in 2022, according to LexisNexis Risk Solutions.²⁰⁶ U.S. and Canadian institutions accounted for $61 billion. These figures represent direct costs (personnel, technology, training, reporting infrastructure) and exclude indirect costs: transactions not completed due to compliance friction, customers denied banking access because flag rates make servicing uneconomical, time costs borne by customers subjected to compliance procedures.

This $274 billion expenditure purchases 4.6 million Suspicious Activity Reports filed with FinCEN in fiscal year 2023, roughly ten percent above the prior baseline of 4.2 million.²⁰⁷ Once filed, the reports sit in a database. Law enforcement agencies request specific reports when investigating specific actors already identified. The reports provide supporting documentation for investigations already underway.

A database accumulating 4.6 million filings annually over decades is not a tool for identifying unknown criminal activity but for documenting known activity identified through other means. The database establishes that the institution flagged relevant transactions, protecting it from regulatory liability for failure to report.

Peter Reuter and Edwin Truman, in the most rigorous independent analysis of anti-money-laundering effectiveness, concluded that evidence the global AML framework deters or disrupts significant money laundering is thin relative to resources consumed.²⁰⁸ The framework generates documented compliance activity at enormous scale. The production of that documentation is independently valuable to institutions producing it and regulators demonstrating reporting requirements are met. The relationship between documented compliance activity and reduction of financial crime is not established by filing volume.

This is the machine’s output in the financial domain: an industry of unprecedented scale producing monitoring documentation at a rate no human review can process, generating costs falling most heavily on actors least equipped to absorb them, demonstrating regulatory compliance with monitoring requirements.

Suspicious Activity Reports

The critique of the financial compliance system meets an immediate objection: if it is not working, why has it not been reformed? The institutional logic of Chapter 5 answers this question directly.

The institution that reduces SAR filing rates by exercising more judgment faces two risks. First, unreported transactions may later be identified as money laundering, generating regulatory sanction for failure to file. Second, regulator scrutiny—SAR filing volume is an indicator of monitoring robustness, so a declining rate looks like declining monitoring.

A SAR filed for a legitimate transaction costs the institution little: administrative processing and a potential customer call. A SAR not filed for a criminal transaction imposes potentially catastrophic costs: regulatory enforcement, reputational damage, civil liability. The rational response is to file at the high end of the plausible range.

This is the precautionary principle operating in the compliance domain, carrying the same incoherence Chapter 6 identified in the pharmaceutical context. The institution that files a SAR for a legitimate transaction bears essentially no cost for the false positive. The institution that fails to file for a transaction that turns out to be criminal bears large costs. The error function is one-sided. Large penalty for under-reporting, minimal penalty for over-reporting. Systematic over-reporting is the predictable output.

The regulator, facing tens of millions of filed reports and finite resources, adapts. For the regulator, the reports function as documentation of institutional monitoring rather than investigative leads. Such documentation has regulatory value independent of investigative value. The reporting requirement generates compliance documentation. Compliance documentation demonstrates regulatory compliance. Regulatory compliance is the machine’s product.

This is Haddaway’s Law in the compliance domain. The institutional imperative to prevent hurt has organized itself into a $274 billion industry producing monitoring documentation no review capacity can process: the grandmother interviewed, the shell company cleared, and the machine satisfied.

Geneivat Da’at

Maimonides identified a moral failure that the compliance apparatus systematizes. Geneivat da’at (the theft of the mind) is the prohibition against creating a false impression in another’s mind—not through explicit lying but through structuring an information environment that leads to mistaken belief.²⁰⁹ It extends the prohibition on deception: it is not sufficient to avoid false statements. It is prohibited to arrange circumstances so the other person draws a false conclusion, even if nothing stated is literally untrue.

The compliance system commits institutional geneivat da’at by presenting its documentation volume as evidence of the underlying function it represents. When an institution files 4.6 million Suspicious Activity Reports, it creates the impression it is monitoring financial transactions at a rate consistent with 4.6 million flagged events. It has actually produced 4.6 million reports about transactions that triggered indicator algorithms—a different activity, performed by a different process, with a different relationship to financial crime prevention.

The impression is not deliberately constructed. No compliance officer chose to deceive the regulator about the relationship between SAR volume and actual monitoring quality. Rather, it is the natural output of an oversight architecture that measures what can be measured—filing volume, training completion rates, examination findings, the percentage of transactions reviewed—and treats the measured as equivalent to the real. When the institution meets the measured standard, it appears compliant. The measured standard creates the impression of meeting the underlying objective. The underlying objective is not what the measured standard measures.

In aviation, the gap closed in eleven minutes. In financial services, the gap is chronic, distributed, and non-fatal in any individual transaction. The cumulative cost—financial crimes not prevented, legitimate customers deterred by compliance friction, resources that could address actual investigation—is substantial but diffuse, attributable to no specific filing or decision. The machine cannot fail. It can only be underfunded.

The Space Between Rules

The monitoring regime cannot know what it needs to know. Looking ahead, the future is uncertain, not merely risky, and regulatory instruments designed to manage risk apply imperfectly to genuine novelty. The present is also partly unknowable from the center. Knowledge required to govern complex situations is distributed across situated, tacit, relational forms that resist extraction. Both constraints are permanent. Better analysts and improved data collection cannot overcome them.

The Safety Machine cannot produce what it needs to produce. Even if we aggregate the knowledge it requires, the behavioral outcomes it seeks—judgment exercised well under uncertainty by people who have internalized institutional purposes—cannot be produced by specification. Compliance produces documentation of judgment’s outputs. It does not produce the disposition from which those outputs come.

The machine reports what it can report. Consider the financial compliance system’s 4.6 million annual filings, its $274 billion expenditure, its adverse selection flagging the grandmother and clearing the shell company—these are not aberrations. They are the rational output of the incentive structure built into the reporting requirement. The system optimizes for the measurable. The measurable is not the same as the real. The gap between what the documentation says and what the institution actually produces is what geneivat da’at names.

The machine’s failures are its predictable outputs. They are what the machine is, not what happens when it malfunctions. Better rules and more rigorous enforcement cannot correct them. A system without limiting principle absorbs any principle introduced into it, as the Benzene doctrine’s neutralization demonstrated. The question is not whether the Safety Machine can be reformed but whether institutions can be built differently from their foundations: designed for beings with the epistemic constraints these chapters describe, oriented toward cultivating judgment rather than documenting it, built to preserve the practices where wisdom develops rather than to produce records asserting its existence.

The traditions this book has been recovering built something less like the Safety Machine and more like what the Safety Machine has been trying, and failing, to become. Understanding what they built and why it held requires going back further than the Administrative Procedure Act, further than Benzene, further even than the Nicomachean Ethics. It requires asking what a legal institution designed for genuine human limitation actually looks like and what it takes to build one that works.

Chapter 10: The Gymnasium

The answer already exists. A tribunal of farmers meets on Thursday mornings at the door of the Valencia Cathedral, and has done so for over a thousand years. A twelfth-century rabbi wrote objections into the margins of the Mishneh Torah. An economist discovered the same structure operating in a Turkish fishing village. What these institutions share is a form that preserves disagreement, and with it, the conditions for practical wisdom.

Dissent

In approximately 1180 CE, Moses Maimonides completed the Mishneh Torah, the most ambitious work of Jewish legal scholarship in centuries. He had spent a decade writing it. His organizing principle was deliberate: clarity over argument, ruling over deliberation, and the law stated without rehearsal of the debates that had established it. Prior legal codes had preserved the dispute alongside the conclusion and the rejected opinion alongside the accepted one. Maimonides chose otherwise. The Mishneh Torah would give you the law.

Rabbi Abraham ben David of Posquières (the Ravad) received the text in Provence and disagreed with this principle before he disagreed with any ruling. He wrote his objections into the text.²¹⁰ The glosses appear between passages and in the margins, brief annotations positioned precisely where the ruling they challenge appears. Some correct an error of fact. Some dispute a conclusion. Some are sharp. In one gloss on Maimonides’ ruling that a person who attributes a physical body to God is a heretic, the Ravad writes: “Why does he call such a person a heretic? Greater and better men than he have been of this opinion, relying on Biblical texts and even more so on aggadic literature, which cause the mind to go astray.”

The Ravad is not wrong. He is restoring the dignity of people Maimonides’ ruling condemns by noting that their error, if it is one, was shared by scholars Maimonides has no standing to dismiss. The gloss is part of the argument. It is addressed to every reader of the ruling, at the moment of reading it, in a form that cannot be skipped.

Once written, the Ravad’s glosses could not be separated from the text they annotated. Every manuscript and printing of the Mishneh Torah carried both the ruling and the challenge on the same page. A student of Jewish law who opened the code to understand what it said was required by the physical form of the text to encounter what someone argued was wrong with it. The gloss is part of the argument.

The Gymnasium preserves this principle: embedding the challenge in the authoritative text, inseparable from it.

Naked

The Greek gumnazō (to exercise or to train) takes its root from gumnos: naked, unprotected, exposed. The gymnasium was not primarily an athletic venue. It was the institution where training occurred in conditions of genuine exposure, where the work was done in a form that made failure visible, consequential, and correctable. The gymnasium was where one learned by being wrong in front of people who would remember it.

In the sense this chapter uses, the Gymnasium is an institution designed for practice under conditions of genuine accountability: to the subject matter, to the community of practitioners, to the record of what was argued and what was found to be wrong. The Ravad’s gloss is a gymnastic act. It inserts the objection into the authoritative text in the only form that cannot be later dissolved by editorial revision or selective citation. Maimonides cannot respond across eight centuries and have the response filed. The objection stands with the ruling, requiring each new reader to hold both.

The compliance apparatus the preceding chapters described is the gymnasium’s inverse: designed so the officer cannot fail for having filed. The compliance officer who files a Suspicious Activity Report has performed an institutional act that cannot fail in any way the institution will later be asked to account for. The filing is the performance, and the performance succeeds by existing. The System Safety Analysis that classified MCAS as non-catastrophic was reviewed against its own analytical framework, not against the aircraft it purported to describe. The documentation of safety could not fail on documentation grounds. It failed on different grounds, which the documentation process was not designed to check.

The Gymnasium checks the documentation against the reality, in conditions where the checker is accountable for the check.

Tribunal

On Thursday mornings, at the door of the Valencia Cathedral, eight elected farmers convene the Tribunal de les Aigües (the Water Tribunal of Valencia).²¹¹ The tribunal has met there on Thursdays for over a thousand years. The eight syndics, one from each of the eight huertas that draw from the Turia River’s main canals, hear disputes about water allocation: unauthorized diversions, failures to maintain one’s section of the ditch, damage to shared infrastructure. They deliberate. They rule. The proceedings are oral; no written record of individual cases is kept. The decision issues the same morning.

The huerta system the tribunal governs (the irrigated agricultural plain surrounding Valencia) operates in a semi-arid Mediterranean climate where water is scarce, seasonal, and collectively managed. The canals that deliver it have been maintained by the farming communities that depend on them since at least the tenth century. Some elements of the system predate Arab rule in Spain and were preserved through successive changes of sovereignty. The tribunal manages a resource that, in the standard economic analysis of commons governance, should have been exhausted long ago.

Garrett Hardin’s 1968 essay “The Tragedy of the Commons” argued that common-pool resources governed by shared users without private property rights or state regulation would be exploited to ruin: each user, acting individually, would take more than the commons could sustain, and the commons would collapse under the aggregate weight of individually rational decisions. The Valencia tribunal had been preventing this for a millennium before Hardin published. The theory was wrong. The tribunal was not impossible but real, meeting every Thursday, so the theory required revision.²¹²

Elinor Ostrom spent her career supplying that revision. She studied the Valencia huerta. She studied Swiss alpine meadows that have governed timber and grazing rights among farming communities for eight hundred years. She studied Japanese mountain commons (iriaichi) where villages have managed shared forests and pastures through governance arrangements dating to the Tokugawa period. She studied groundwater basins in California, fishing grounds in Maine, irrigation systems in Bali where water temples have coordinated rice cultivation across thousands of terraced paddies for centuries. In each case the economic theory predicted failure. In each case the institution had been managing the commons for generations.

What Ostrom found, across the full range of these cases and published in her 1990 work Governing the Commons, was that the institutions that work share a form. Not the same rules: the rules of the Valencia tribunal fit the hydrology of the Turia River and the patterns of demand in a Mediterranean agricultural system; they would not transfer intact to a Japanese mountain commons or a Maine fishing ground. But the same design features, arrived at independently, across cultures and centuries and legal traditions with nothing in common except the problem they were solving.²¹³

Local Knowledge

Boundaries are defined. Those who hold rights to the resource are known, as are those who do not. Ambiguity erodes the basis for obligation and enforcement.

Rules fit the resource. The regulations governing the Valencia huerta are specific to the Turia’s hydrology, the soil, and the cropping patterns of Mediterranean agriculture. The knowledge required to write them is local.

Those governed by the rules modify them. The syndics are elected farmers accountable to their communities, with the power to adjust rules when they produce outcomes the community rejects.

Monitoring is performed by those subject to the rules. The farmers who depend on the canal monitor it. They bear the cost of monitoring failure: the diverted water is their water, the damaged ditch is their ditch.

Sanctions are graduated. A first infraction brings a small fine; repeat violations escalate. This allows correction before exclusion, makes enforcement credible, and shows that severity matters rather than applying a flat rate that functions as a tax.

Resolution is accessible and fast. The oral proceeding at the cathedral door costs nothing and produces a decision the same morning. It eliminates the procedural overhead that would make dispute resolution a last resort.

Where the governance system covers a large area, it is organized in layers. The huerta syndic handles disputes within the ditch. Problems at the level of the canal are handled at the canal level. Problems involving the river are handled at a scale that encompasses the river. Each layer addresses the problems its knowledge reaches, accountable to the layer below it and coordinating with the layers above.

These features are empirical findings, not theoretical prescriptions. Ostrom identified the pattern and explained why it recurs: communities discover this form because it matches what the problem requires.²¹⁴ The knowledge needed to govern is distributed across the people in the situation. The form that works uses the knowledge where it exists.

The Safety Machine cannot accommodate this insight. The machine rests on the premise that knowledge can be extracted from the situation, codified, and applied from the center. The commons governance arrangements that work rest on the opposite premise: the knowledge stays where it is, and the institution goes to it.

Alanya’s Coffeehouse

Each September, in the fishing town of Alanya on Turkey’s Mediterranean coast, the fishers draw lots. The lottery assigns each fisher to a numbered site for the season. Since the early 1970s, they have gathered in the coffeehouse to do this, after a period of conflict over the productive locations forced them to invent a method to allocate access fairly.

The coastline is mapped into named fishing spots: locations worked for generations, each distinctive. Some sites run rich in late autumn; some are productive only in particular conditions; some consistently outperform others across the season. The fishers know their coast the way farmers know their fields. The knowledge exists in practice and cannot be readily codified.

The lottery distributes this advantage. Beginning in mid-September, each fisher occupies his assigned location and then rotates daily: east to west from September through January, then west to east from January through May. By season’s end, each fisher has worked every site once. No fisher accumulates the advantage of consistently superior locations. No fisher bears the cost of consistently inferior ones year after year.

After the drawing, the list is posted on the coffeehouse wall. Any fisher can verify where any other fisher should be on any morning. There is no ambiguity about who belongs where. A copy goes to the local gendarme.

That copy records the state’s relationship to the system. The state does not set the schedule, conduct the lottery, or resolve disputes. The copy documents a deliberate choice: the state present as witness, not manager. It acknowledges that the knowledge needed to govern the resource is in the room where the lottery occurs.

Disputes occur. A fisher finds someone at his assigned site who should not be there. The matter is brought to the coffeehouse. The posted list resolves it. The assignment is unambiguous; the violation is specific. The community adjudicates and reaches resolution quickly because the record is local and present.

Ostrom studied Alanya in the 1980s as one case among many. What she found was not exotic but recurring: the same architecture from Valencia, Swiss alpine meadows, Japanese mountain commons, and Balinese irrigation temples. Different rules fitted to different resources, but the same structure arrived at independently. Knowledge where it exists. Governance accountable to those it governs. The state present but not in charge.

The opposite arrangement governed the Grand Banks cod fishery. At the national level, government scientists in Ottawa set catch quotas. Fishers working those grounds across generations knew stocks were declining, visible in falling catch rates and changed fish behavior. The decision-making process could not receive this knowledge. The scientists’ models showed sustainable extraction levels. The fishers’ experience showed something else. In 1992, Canada imposed a moratorium after the population fell below two percent of its historical abundance. Thirty thousand people lost their livelihoods. The fishery has not recovered.

Remove the coffeehouse and keep the gendarme, and Alanya becomes the Grand Banks.²¹⁵

Friction

The Safety Machine produces friction. The Bank Secrecy Act’s reporting requirements impose costs on every transaction. The ODA review required Boeing engineers to work through certification protocol. The grandmother in a ninety-minute compliance interview experienced friction. The crews of JT610 and ET302 experienced the wrong kind.

The Gymnasium produces a different friction.

The grandmother’s ninety-minute interview improved nothing: not her judgment about giving, not the compliance officer’s judgment about suspicious transactions, not FinCEN’s capacity to investigate crime. It produced a record that monitoring had occurred: a document satisfying institutional requirements without advancing institutional purpose. This is sludge—friction that costs without improving, delays without discerning, imposes without teaching.²¹⁶

Constitutive friction differs. The Ravad’s gloss improved the Mishneh Torah by identifying where the ruling required modification, where a conclusion overreached, where authority should be more tentative. The gloss is not an obstacle but part of the process by which the law reaches a form that can survive challenge. Without it, only Maimonides’ ruling stands: a better start than nothing, a worse end than both together.

The Tribunal de les Aigües runs on the same principle. The hearing is constitutively costly: it requires time, presence, and public accounting. The cost is the content. The farmer who appears and loses knows more afterward than a mailed fine could convey. The neighbors who witnessed the proceeding know the monitoring works, the sanction is calibrated to the violation, and the institution is present in actual canal life.

The schools of Hillel and Shammai disputed every significant question of Jewish law. A divine voice declared: “Both are the words of the living God.”²¹⁷ The Talmud preserves both positions. Shammai is overruled on most questions and remembered on all of them. Any student wanting to know the law must encounter the rejected argument in the same text, in the same breath. The rejected opinion is preserved because the conditions that would make it correct may recur.

The Safety Machine discards the Shammai position. Failed comment letters are summarized in the preamble. Dissents from System Safety Analyses do not appear in the certification record. The process documents consensus because consensus is what it produces. The Gymnasium preserves the objection because you need it when consensus fails.

Havruta

The Valencia tribunal, the rabbinic academy, Ostrom’s commons, and the common law’s adversarial proceeding share specific features that distinguish them from the Safety Machine—specific enough to be designed into or against.

The first feature is recorded disagreement. The Gymnasium preserves the challenge alongside the ruling. The Ravad’s gloss, minority opinions, dissents, written objections: forms embodying this requirement across contexts. The Safety Machine produces consensus documents in which disagreement has been absorbed and formally resolved. The Gymnasium keeps disagreement active, requiring readers to reckon with it. The preserved disagreement is not courtesy but epistemic practice that enables revision when consensus fails.

The second is accountability in the situation. The syndic who monitors the huerta is a farmer who depends on it. The monitoring knowledge exists in the situation, and the monitor is present. The knowledge of what a suspicious transaction looks like (what makes a wire from an elderly woman anomalous after forty years of banking relationships) is also in the situation: in the prior relationship, in the context the algorithm cannot see. The Gymnasium puts the monitor where the knowledge is. The Safety Machine places it at a remove with only a checklist.

The third is graduated response. The graduated sanction requires the enforcer to distinguish between a first infraction, a pattern, and fundamental defection—a distinction requiring contextual knowledge. A flat fine is a tax. A graduated regime requires judgment. The Bank Secrecy Act does not distinguish between a grandmother wiring five hundred dollars to hurricane relief and a shell company routing three million through correspondent banking. The algorithm classifies by deviation from pattern. The Gymnasium classifies by what they are.

The fourth is cost to the monitor. Ostrom found that successful commons are monitored by those bearing the cost of failure. The farmer-monitor who fails to report a diversion while neighbors lose water has failed them and will answer for it. The external compliance officer who misses a fraudulent transaction has, in institutional accounting, already succeeded: the activity was performed, the algorithm ran, the threshold was not triggered. The cost does not return to the monitor. Without that accountability, monitors optimize for institutional compliance rather than underlying purpose.

The fifth is polycentrism. The tribunal asks Valencian farmers to govern their ditch, not water policy of the Iberian Peninsula. Ditch-level governance handles ditch-level problems. Canal-level problems are handled at the canal level. River problems at the river scale. Governance is nested: each level handling what its knowledge reaches, accountable below. The Safety Machine runs backward: problems at the transaction level generate national guidance, which generates transaction-level compliance requirements, monitored by people without transaction-level knowledge.

What the Gymnasium Builds

The Ravad’s method becomes the institutional requirement: no ruling without its recorded challenge, no authority without accountability to those it governs, no monitor without monitoring knowledge, and no sanction without judgment.

Institutions have existed continuously for centuries in conditions more demanding than modern regulatory systems face. The tribunal’s thousand-year record is not coincidence but evidence that this form endures because it is built from what governance requires rather than what governance documentation can represent.

The Gymnasium is not a single design but a family of designs sharing the requirements above. The bet midrash is a gymnasium: adversarial, recorded, preserving minority positions alongside majority rulings. The common law’s adversarial proceeding is a gymnasium when parties have evidence access, the judge is accountable to external standards, and losing arguments are preserved and retrievable. The safety review board that can question certification before closing the record is a gymnasium. The ODA program is not, because its accountability structure eliminated the friction that would have forced someone to notice the gap between analyzed system and produced one.

Institutions can be built as gymnasiums. They have been. Chapter 11 takes up how this practice produces what it is supposed to: how judgment is cultivated rather than documented, what the practice of being-wrong-in-front-of-others actually develops, why Alpine commons governance across centuries produced farmers who knew rules without being able to state them, why that knowledge was more reliable than codified rules.

Aristotle had an answer. The rabbinic tradition had a different one. They are more compatible than they appear, and their agreement implies more specific institutional design than either source alone provides.

Chapter 11: Aristotle’s Algorithm

Before Good

A medical student seeing a sick patient for the first time often does not recognize it. The vital signs may be only marginally off, or not yet off. The chief complaint is vague. The attending physician walks into the same room, scans the same chart, speaks for thirty seconds, and begins moving: ordering things, changing the plan before any numbers have changed. The student asks afterward how he knew. He has difficulty saying. Something in the patient’s color. Something in the breathing. Signals with no name, no place in any clinical algorithm. He says: you learn to see it.

She does, eventually, after years of seeing patients under conditions where her attendings tell her when she missed something and what she should have noticed—not through explanation, which always halts. The knowledge does not live in explanation-form.

That process develops hexis, Aristotle’s term.

Hexis is typically translated as “habit,” but habit captures only behavior. Habit is a repeated pattern. Hexis is the condition of character from which the pattern flows: a stable disposition, a way of perceiving and responding that no longer requires effort because it has become who the person is. The attending does not run through a mental checklist but perceives. This perception is the product of a developmental process that cannot be shortened: not because it is mysterious, but because practice with correction over time is the only thing that produces it.

The Safety Machine cannot replicate this process; not through insufficient effort or rigor. The conditions under which hexis develops are incompatible, in a specific technical sense, with the conditions the machine requires.

What Aristotle Required

Aristotle is precise about what distinguishes virtuous action from performance. Three conditions must be met.²¹⁸ The agent must act with knowledge—not of the rule, but of what she is doing and why it matters. She must choose to do it for its own sake, not because monitoring demands it. And she must act from a stable, settled disposition: not as a one-time performance, not in response to pressure, but as an expression of who she is.

The third condition is what institutional design cannot satisfy by decree. The first two can coexist with compliance: a person can know what she is doing and choose it even when regulation also requires it. The third condition cannot be produced by requirement but only by practice under genuine accountability, sustained over time with correction. A compliance requirement can coexist with an existing hexis. It cannot create one.

Compliance can create a performance of the third condition. The certificate documents training completion, assessment passage, acknowledgment signature. This documentation has the form of evidence that the disposition exists—not evidence of the disposition but a record of action that, if performed from a stable disposition, would look identical to action performed to satisfy a requirement.

The gap between the documented condition and the real one is the same gap in Boeing’s System Safety Analysis and in Suspicious Activity Reports: appearing at the level of character rather than institutional output, but with identical source. The machine measures what is measurable, and what is measurable is the action, not the disposition from which it flows.

Compliance People

John Locke identified a paradox applying to any attempt to produce an internal state through external requirement. His argument concerned compelled religious observance: requiring subjects to attend church, take sacraments, and perform religious forms they might not hold. Locke’s objection was not only that this was unjust but that it could not achieve its stated goal. The goal of compelling practice was, presumably, to bring people to genuine belief. Genuine belief is precisely what compulsion cannot produce. You can require external acts, not the internal state from which they flow. The compelled acts, produced without the accompanying disposition, are not the thing you were trying to produce but a facsimile, externally identical and constitutively empty.²¹⁹

Substitute compliance training for religious observance and the point holds. The goal of training is not module completion but the judgment the module was designed to produce: recognizing when a regulation’s purpose is being served, when it is not, and what to do in situations the regulation did not anticipate. You can compel completion—the video, assessment, acknowledgment, the record. What cannot be compelled is the judgment development that was supposed to follow. That requires exercising practical discernment in actual situations with actual consequences, not observing representations and selecting predefined answers.

The compliance module has become the primary mechanism across industries for documenting that judgment exists. It is not designed to produce judgment but documentation. The confusion between these activities is the machine’s operational premise. The same people sometimes perform both using the same materials. The confusion is not venal but a natural error from the same logic that produced the System Safety Analysis: the measurable stands in for the real, and the substitution remains invisible until the real is needed and found absent.

Enron

On July 1, 2000, Enron Corporation distributed to shareholders a sixty-four-page Code of Ethics. Kenneth Lay, chairman and chief executive, signed a prefatory letter: “As officers and employees of Enron Corp., we are responsible for conducting the business affairs of the Company in accordance with all applicable laws and in a moral and honest manner.”

The Code enumerated four core values: respect, integrity, communication, and excellence, each elaborated across several pages. Under integrity: “We work with customers and prospects openly, honestly and sincerely. When we say we will do something, we will do it; when we say we cannot or will not do something, then we won’t do it.”

Approximately two years later, Enron filed for bankruptcy in December 2001; then the largest corporate bankruptcy in American history. Subsequent investigations revealed systematic accounting fraud, concealment of billions in debt through special purpose entities, manipulation of California’s electricity markets, and destruction of audit documents. In 2002, two copies of the Code of Ethics were auctioned on eBay, selling for a combined twenty-four thousand dollars.²²⁰

The Code is not evidence of hypocrisy if hypocrisy means knowingly saying one thing while doing another. Most people at Enron appear to have been committed to the values it named, or at least not indifferent to them. The Code documents something more instructive: a document articulating the outputs of virtuous conduct is not, even in sincere hands, a mechanism for producing it. It described what the company would look like if its people had the relevant dispositions. The dispositions were not there. The document could not put them there. The gap between description and reality remained invisible until fatal because the compliance process had no mechanism for distinguishing between documentation of the disposition and the disposition itself.

Surgery

Before examining the theory that explains the Code’s failure, consider an institution that has imperfectly but recognizably preserved the conditions for genuine professional judgment. The medical residency, for all its documented problems, remains structured around Aristotle’s three conditions in ways corporate compliance training does not.

A first-year surgical resident does not watch videos but assists in surgery: holding retractors, managing suction, closing incisions under supervision—all in conditions where her actions have consequences for a patient she is responsible for. The knowledge she develops is inseparable from the exercise. She learns to read the operative field by being present in it, acting within it, and having someone who can already read it tell her when she has missed something.

Graduated exposure is essential. In the first year, the resident performs limited portions under close supervision. As her judgment develops—as she shows she can anticipate complications, recognize unexpected anatomy, and adjust technique to unwritten situations—she is given wider latitude. By the final year, she performs procedures with the attending available but not scrubbed in, exercising independent judgment where error has real, immediate consequences. The progression is calibrated to developing capacity, not to completing curriculum.²²¹

What makes this model different from compliance training is structure, not content. The resident exercises judgment under genuine uncertainty: patient response to anesthesia, variant anatomy, mid-procedure complications. She is corrected by someone possessing the judgment being developed who can recognize its absence. The correction is sustained over years, not compressed into a module. The institution never treats a completed assessment as evidence that the disposition has been acquired. The attending who supervised the resident for three years knows what that resident can and cannot do—a specificity no certification can replicate.

The model is expensive, slow, and resistant to standardization. This is why compliance has not adopted it. Compliance produces documentation of competence rather than competence itself. Residency produces physicians who can read a deteriorating patient before vital signs confirm it. Compliance produces professionals who can document training completion. These are not equivalent. The institution treating them as equivalent has confused the artifact with the capacity it measures.

After Virtue

Alasdair MacIntyre drew a distinction in After Virtue that clarifies what the Enron Code attempted and why it could not succeed.²²² MacIntyre did not write about compliance regimes, but his framework, developed to diagnose the fragmentation of moral reasoning in modernity, applies with precision to them. What follows is my application of his categories.

A practice, in MacIntyre’s sense, is a complex, socially established cooperative activity through which internal goods are realized. Internal goods can only be obtained by participating according to the practice’s own standards of excellence. They cannot be purchased, mandated, or produced by performing external forms while bypassing substance. When achieved, they benefit the community of practitioners, not only the individual. Medicine’s internal goods are: clinical judgment, the capacity to perceive what is wrong with a patient before vital signs confirm it, and the competence to bring disparate findings into a coherent diagnosis. You acquire them only by practicing medicine under conditions that develop them, under observation and correction of someone who already has them.

External goods differ: compensation, credentials, status, advancement are achievable by performing external forms without internal goods. They are not intrinsically connected to substance but contingently associated with medicine because medicine happens to be the context where they are obtained. When achieved, they belong to the individual as possessions, not the community.

Every practice has both. The institution redirecting practitioners systematically toward external goods—by replacing genuine practice with documentation—produces people who accumulate external goods without developing internal ones. MacIntyre calls this corruption: not deliberate degradation but slow displacement in which the core is replaced by a simulacrum that performs forms while losing substance.

The compliance module redirects toward external goods. Completing it produces a record protecting the institution from liability and the individual from deficiency; these are the external goods of compliance. The internal goods (the judgment to identify actual risks, the discernment to distinguish a rule’s letter from its purpose, and the capacity to act well in unanticipated situations) are produced by exercising judgment in actual situations with actual consequences, under observation of someone capable of recognizing whether the exercise was good or poor. The module documents the external goods, leaving internal goods to be acquired or not through whatever conditions the practitioner encounters outside it.

Kenneth Lay signed the Code but had not participated in the practice it described. No compliance requirement could have made him participate. The Code documented the internal goods of honest dealing. Internal goods require the practice. The practice requires conditions. Documentation does not create conditions.

Mind Theft

Maimonides’s account of the relationship between law and wisdom converges on a principle the compliance literature has been deriving for three decades:²²³ Law can create the conditions for wisdom’s development but cannot command wisdom’s presence.

Maimonides distinguished between laws governing the body politic (prohibitions of fraud, violence, and theft that constitute tikkun ha-guf) and cultivation of the soul (tikkun ha-nefesh). The first is law’s domain: commanding and prohibiting behaviors specified in advance, monitored and enforced through coercive sanction. The second requires something different. The perfection of the soul cannot be produced by command. It requires voluntary engagement with difficult questions: genuine deliberation, genuine effort, the genuine possibility of failing. Coercion destroys this possibility and the conditions that would have produced wisdom.²²⁴

In Maimonides’s account, this is not passivity but counsel about right instruments. Law can create conditions: social structures, institutional frameworks, graduated challenges, communities of practice. Within these conditions, wisdom develops. Law cannot substitute for conditions by commanding outputs. The legal system commanding wisdom produces, at best, documentation of wisdom’s compliance, destroying the conditions that would have produced wisdom.

The parallel to modern compliance is exact. The Sarbanes-Oxley requirement is a legal command to produce a document describing the outputs of ethical judgment. The document is inspectable, auditable, verifiable. The judgment that would have produced those outputs is not. Enron had the document. Its board exercised judgment badly and disastrously, in ways the document explicitly prohibited when it voted to authorize those transactions. The document was tikkun ha-guf deployed in the domain of tikkun ha-nefesh: a command instrument applied to a domain that command cannot reach.

Maimonides had a specific vocabulary for this institutional error: Geneivat da’at—theft of the mind; creating a false impression not through lying but through structuring information environments that mislead. The compliance system presenting its documentation as evidence of ethical judgment commits institutional geneivat da’at. It creates a false impression for regulators, shareholders, and the public that the institution is cultivating the wisdom whose outputs the documentation records. The documentation is not fraudulent; the impression it creates is. This is the deepest compliance theater: not failure of honest reporting but systematic confusion of the artifact with what it measures.

Atrophy

While managing electronic records, billing codes, prior authorizations, and quality metrics, physicians correspondingly lose time for medicine, yet remain not documentably deficient. She has complied with everything required. The clinical judgment that comes from years with patients under supervision of someone who can say “that is not what you should have noticed; look again” is what the system does not require and has no mechanism for producing.²²⁵

The attorney who produces documents by template, reviews standard provisions by checklist, and advises clients to accept standard terms without exercising independent judgment about whether they serve the client’s situation has not failed the bar but has been shaped by an environment rewarding external-goods production more reliably than independent judgment development. The environment did not lie but simply did not produce what it implied.²²⁶

The compliance officer who knows which transactions require SARs, which accounts require enhanced due diligence, and which reports are due when, but lacking any mechanism to develop judgment to distinguish suspicious transactions from legitimate ones that trigger the algorithm,is doing the job as designed. The job does not require the judgment she lacks.²²⁷

Humans Aren’t Algorithms

Aristotle’s account of hexis is, in a precise sense, a procedure: conditions under which it develops and conditions under which it does not. The conditions are not optional as regulations are—optional for someone willing to accept the fine. They are conditions in the way warmth and moisture are conditions for germination: the thing does not occur without them, regardless of anyone’s intentions.

The first condition: the practice must be exercised in situations with real consequences for something the practitioner values—not observed, not simulated in low-stakes assessment, not described in case studies ending with multiple-choice. The surgeon-in-training operating on patients under supervision exercises judgment in conditions that develop it. The compliance officer viewing a training video does not. The difference is not content but presence or absence of genuine exercise. Judgment must be called upon to work, not identified from predefined menus.

The second condition: the exercise must be subject to correction by someone capable of recognizing good practice from bad—the condition the module most conspicuously eliminates. The multiple-choice assessment has a correct answer selected by the designer. Judgment in actual practice does not have a correct answer checkable against a key but a better and a worse, recognizable to someone who has exercised the judgment extensively and seen enough of its failures to know what errors look like. Correction requires presence and competence. Neither can be replaced by completion records.

The third condition: the practice must be sustained long enough for the disposition to become second nature—the condition credentialing most thoroughly misrepresents. Credentialing marks a threshold where external goods are conferred, and the threshold systematically overstates the degree to which internal goods have been acquired. The resident who completed residency has more clinical judgment than a beginning resident but not the judgment of a physician with twenty years of practice. The credential implies the latter while documenting the former.²²⁸ The institution reading the credential as sufficient for independent practice, then providing no further development conditions, has confused the documentation of development with development itself.

This is ordinary knowledge about competence development, not exotic. Any practicing physician, experienced attorney, or skilled tradesperson could articulate it from their own formation. The Safety Machine did not eliminate this knowledge from the world but from the institutional processes that produce competent practitioners. The conditions for genuine development (time, supervision, risk of error with consequences, sustained correction) are expensive, slow, and produce no documentation that the process is occurring. The machine produces documentation; the conditions for hexis produce judgment. These are not the same activity. When the machine is given responsibility for producing judgment, it produces documentation of judgment instead, because that is what it knows how to do.

Practice Makes People

The Gymnasium creates the conditions for Aristotle’s algorithm to operate. The Ravad’s gloss is a product of those conditions: a lifetime of Talmudic practice in an institution requiring independent judgment about difficult texts in front of interlocutors capable of recognizing whether the judgment was sound, with the record preserved for evaluation by all who read afterward. The practice developed the capacity. The capacity produced the gloss. The gloss is the internal good of the practice—the thing no module could have produced because it is the specific product of that specific person’s judgment, shaped by that tradition’s conditions, exercised on that problem.

The syndic of the Tribunal de les Aigües who hears a water dispute on Thursday morning and issues a ruling the community accepts as binding exercises a capacity developed through years participating in the tribunal, observing disputes heard and ruled on, making arguments under observation of more experienced syndics, having those arguments corrected, and gradually acquiring the judgment the institution requires. The tribunal does not certify this process but creates the conditions for it. The conditions produce judgment. The judgment maintains the huerta.

The title names something that does not exist. Aristotle did not write an algorithm. Judgment is not an algorithm. The irony is precise: the thing that actually produces the behavior the machine is trying to mandate is the practical wisdom to act well in novel situations, under genuine uncertainty, without a checklist. It is produced by the process least amenable to the machine’s methods. It cannot be scaled, verified through documentation, or monitored through metrics. It is produced by practice. Practice requires conditions. Conditions require institutions designed to create them. Those institutions look nothing like the Safety Machine.

They look like what Part Three has been describing: the Gymnasium. The next chapter examines the specific institutional hardware: the design features that preserve conditions for judgment across generations rather than creating them for one cohort and allowing them to atrophy.²²⁹

Chapter 12: The Hardware

Practical wisdom develops in individual practitioners. Institutions have to preserve the conditions for it across generations. A valve stuck open at Three Mile Island at four in the morning in 1979. Eight hundred years earlier, a physician in Cairo was already working on the institutional problem the valve would reveal.

Nuclear Safety Valve

At four in the morning on March 28, 1979, a valve stuck open at Three Mile Island.

The valve was supposed to close automatically when pressure dropped. It didn’t. The control panel indicator light read closed. The light was wrong.

For the next two hours, the operators made reasonable decisions based on false information. Coolant was pouring out of the reactor through the stuck valve. Alarms rang. Warning lights activated. The operators read the pressurizer level (rising, indicating too much pressure) and shut off the emergency cooling water. They were preventing one failure while a different failure unfolded. The temperature spiked. The fuel rods began to melt.

Each individual decision was defensible. Given the information available at each moment, each action was what a competent professional would have done. The problem was systemic: the valve failure set off a cascade of consequences that moved faster than human understanding could track, and the instruments themselves were telling contradictory stories about what was happening. There was no time to think. There was no space between cause and effect in which thinking could have operated.

Charles Perrow, the Yale sociologist who studied the accident, called it a “normal accident”:²³⁰ not an unusual accident, not a preventable accident, but the inevitable result of building systems that combine what he called complex interactions with tight coupling. In a tightly coupled system, components are connected so that process A must complete before process B can begin. There is no slack. No buffer. No give between elements. When something goes wrong, the cascade begins immediately, faster than human intervention can intercept it.

The operators at Three Mile Island were competent. The reactor was their professional world. What they hadn’t drilled for was a failure that made the instruments lie, a failure whose signature was invisible in the only information they had. Tight coupling had eliminated the space between event and consequence where diagnosis could have happened. The system had been designed to prevent error by eliminating discretion. When the system failed, the eliminated discretion was precisely what was needed.

Coupling

A different system has been failing productively every day for fifty years.

On any given day, thousands of servers fail across the internet. Cables are cut. Routers go offline. Packets are lost in transit. Data is corrupted. The system keeps working. If a data center in Singapore crashes, email in São Paulo still arrives. The packets find another path.

The internet was designed around the assumption of partial failure. It doesn’t assume all components will function. It assumes that some components will fail continuously and unpredictably and distributes the traffic around the failures rather than trying to prevent them. Karl Weick, the organizational theorist who brought loose coupling into institutional analysis, named the property that makes this possible:²³¹ loosely coupled systems preserve the capacity for localized adaptation. Each component retains enough autonomy to respond to local conditions without destabilizing the network. Loose coupling creates slack: the space between cause and consequence in which judgment can operate.

Tight coupling is faster under normal conditions. Loosely coupled systems survive when things break. Things always eventually break.

Architecture of Resilience

The preceding chapters traced how judgment develops: through practice, correction, and the friction of encounter with what the rules don’t cover. This chapter answers the architectural question: what structure allows that method to operate at scale? How does judgment coordinate across thousands of practitioners, across decades, across communities that share principles without sharing geography, without collapsing into a single rigid standard? How does authority coexist with discretion? How does the system maintain coherence when the center fails or was never there?

Eight hundred years before Perrow analyzed Three Mile Island, a Jewish physician in Cairo had designed the answer.

Moses Maimonides was born in Córdoba in 1138, in a city that was at the time one of the intellectual centers of the Mediterranean world. Jewish, Christian, and Muslim scholars studied in proximity. Aristotle’s texts survived in Arabic translation. Philosophy and medicine flourished. He was thirteen when the Almohad movement arrived. The Almohads were not interested in pluralism. They presented non-Muslims with a choice: conversion or flight. Maimonides’ family fled through Spain, across the Mediterranean, through Morocco. His father died in exile. His brother David drowned in the Indian Ocean, taking the family’s remaining wealth with him. Maimonides settled in Cairo, worked from dawn past midnight, and confronted a problem that had no institutional precedent.

The Jewish legal tradition was a distributed network operating without a center. The Temple was gone. The Sanhedrin, the supreme court that had maintained legal uniformity, had ceased to function a thousand years earlier. Communities were scattered across three continents, living under different political rulers, speaking different languages, developing local customs that were beginning to diverge from one another in ways that threatened the tradition’s coherence. There was no king, no capital, no authority capable of imposing unity.

And yet the system had to function. Disputes had to be resolved. The community had to make decisions about marriage, divorce, inheritance, commerce, religious observance. The decisions had to cohere across communities that would never share a courtroom.

Maimonides’s solution was architectural.

The Mishneh Torah, completed after ten years of labor, was fourteen volumes covering the entire body of Jewish law: organized by category rather than by the Talmud’s associative logic, written in clear Hebrew rather than technical Aramaic, structured as direct rulings. The law is X. The reason is Y. The application is Z. His goal was to enable a person who reads the Written Torah and this work to know the entire Oral Law without reference to any other text.²³²

This was the emergency scaffold Chapter Ten described: the minimum viable platform designed to prevent fragmentation. But Maimonides built something else alongside it. He created a protocol for managing disagreement. Not eliminating it. Managing it. The protocol is the hardware.

What Protocols Allow

The Laws of Rebels addresses a scenario whose resolution is not obvious.²³³

A local court (a regional Sanhedrin) disagrees with the ruling of the Great Sanhedrin in Jerusalem. The local court believes the Great Sanhedrin has misinterpreted the law. Perhaps they’re correct. Perhaps circumstances in their region make the standard interpretation produce injustice.

The answer surprises readers who expect hierarchy.

The local court may teach that the Great Sanhedrin is wrong. The court may explain to its students, in full, why the law should be read differently. The court may write treatises defending its position. The court may work to persuade other courts. The intellectual disagreement is not merely tolerated but architecturally protected. A court that has concluded the law requires a different interpretation is expected to say so.

What the local court may not do is create a practical split: a situation where their community acts according to one rule while other communities act according to a different one. That is the line. The fracturing of the community into incompatible practices (marriages valid in one community but not another, contracts enforceable in one jurisdiction but disputed in the next) is what the prohibition addresses. The intellectual disagreement is not.

Two layers operate simultaneously. At the intellectual layer (wisdom, tikkun ha-nefesh), disagreement is essential and understanding develops through dialectic. A court that concludes the majority is wrong must say so and preserve that reasoning for students and later courts. At the practical layer (social order, tikkun ha-guf), unity is required. A community cannot function when its legal determinations diverge across members.

Loose coupling operates at the intellectual layer; tight coupling at the practical. Authority persists because a clear answer to “what is the law” exists at any moment. Judgment persists because that answer can be evaluated, challenged, and potentially supplanted by better reasoning. The local court exercises judgment by examining the ruling’s correctness, teaching its reasoning, preserving the minority view, and maintaining option value for courts that follow.

The Safety Machine collapses these layers. It applies the logic of the practical (coercion, uniformity, tight coupling) to the intellectual domain. How professionals reason, how judges weigh competing considerations, how teachers adapt instruction to different students—these become risks requiring the same tight coupling as physical safety. Comment letters that fail to persuade are procedurally resolved. Dissents from System Safety Analyses disappear from the certification record. The regulatory process produces a document designed to show consensus because the process is built to manufacture it.

Maimonides understood that tight coupling of thought and tight coupling of action are different requirements, producing different kinds of failure when applied to the wrong domain. The Safety Machine has forgotten this distinction.

Times Change

The preservation of dissenting reasoning is not sentiment. It is engineering.

The Hebrew concept shinui ha’ittim (“change of the times”) describes a fundamental shift in circumstances that makes a previously adequate ruling inadequate. When shinui ha’ittim occurs, the tradition either has resources to respond or must begin from nothing. A tradition that suppressed dissent, eliminated the minority view, and recorded only majority rulings is brittle. When that ruling fails to account for changed circumstances, there is no stored reasoning to draw on—only failure and the need to rebuild. A tradition that preserved minority views alongside majority rulings has stored the option. The reasoning that was wrong for the current era may be exactly right for the next.

The Talmud addresses this explicitly. Mishnah Eduyot 1:5 asks why record the minority opinion when the law follows the majority. The answer: so a future court, finding the times changed, may have an authority to rely upon.²³⁴ Not for historical completeness but to preserve the intellectual resources for adaptation when it is required.

Basel III required all banks to use the same risk models, hold the same capital reserves, follow the same procedures. When the models proved wrong in 2008, in ways no single institution could detect because every institution was running the same analysis, every institution failed in the same way at the same time. There was no dissenting methodology preserved in the system. No alternative approach waiting to be activated. The tight coupling that had achieved coordination had eliminated the diversity required for resilience.

Agricultural monoculture achieves higher yields than mixed planting until a pathogen arrives that targets the single crop. Intellectual monoculture achieves higher consistency than preserved dissent until the situation arrives that the consensus cannot handle.

Metaphysics of Disagreement

The most famous dispute in Jewish legal history ran for approximately seventy-five years.

The School of Hillel and the School of Shammai disagreed on nearly every significant question of law: the rules of Sabbath, the laws of marriage, the calendar, the correct sequence for lighting Hanukkah candles, and the minimum value that constitutes an act of betrothal. On hundreds of questions, their rulings pointed in opposite directions.

The standard assumption about legal disagreement is that one side is right and one is wrong, and the purpose of the process is to determine which. The Talmud declines this assumption. In Eruvin 13b, the tradition records that a divine voice, the bat kol, declared: “These and those are words of the living God.”²³⁵

Two expressions of truth, each capturing something the other does not.

The Hillel/Shammai dispute maps onto a genuine metaphysical division. Hillel’s legal reasoning tends toward actualization: it attends to the person as he actually stands before the court, to the present situation in its concrete particulars, to accommodation of the human condition as it is. The traditional connection is to chesed, the divine attribute of loving-kindness²³⁶, which expands to meet the world where it is. Shammai’s reasoning tends toward potential: it attends to what a thing might become at its highest, to the ideal toward which law should draw its subjects, to standards maintained at their full weight. The traditional connection is to gevurah, the divine attribute of strict judgment, which holds the line against diminishment.

In the current era, the law follows Hillel. The world as it is requires the accommodation that chesed provides: the ruling that meets people in their actual circumstances rather than demanding they conform to an unreachable ideal. But the Shammai positions are not discarded. They are preserved alongside the Hillel positions throughout the tradition, not as historical footnotes but as live alternatives. The mystical tradition holds that in the ultimate era of human moral development, the law will revert to Shammai, the stricter standard stored until the community has grown capable of bearing it.

The more demanding standard is preserved in the tradition because the journey toward it is not finished. The minority position is not merely an error awaiting correction. It is the direction of travel, stored for when the destination becomes achievable.

The student who reads only the Hillel rulings learns what the present requires. The student who reads both learns to hold two truths: understanding the gap between what is and what might become, developing judgment that holds the tension between mercy and rigor without collapsing into either. The friction of contradiction is the curriculum. The Safety Machine eliminates the friction, delivers the answer, discards the challenge. The Gymnasium (as Chapter Ten described) preserves the challenge in the text.

Chapter Ten described what is preserved. This chapter names why the architecture of preservation matters for institutions that must coordinate across scale: not only because the minority might be right, but because the minority is the option value. The dissenting reasoning is the backup code. When shinui ha’ittim arrives (and it always does), the tradition that preserved its disagreements has resources. The tradition that eliminated them has only its failure.

The Judge in the Gate

Deuteronomy 16:18: “You shall appoint judges and officers in all your gates, which the Lord your God gives you, according to your tribes, and they shall judge the people with righteous judgment.”

The instruction specifies location: judges at the gates, in the local community, accessible, embedded in the context they adjudicate. Not judges in the capital. Not judges appointed by a central authority and accountable to it. Judges who know the community, who see the faces of the people before them, who live with the consequences of their decisions.

Loose coupling operates at the implementation layer. The local judge applies universal principles to the particular case using the practical wisdom Aristotle identified as irreplaceable for judgment. The judge knows this community’s history, this dispute’s context, the difference between error and exploitation. The central administrator applying rules mechanically cannot make this distinction—not from lack of intelligence but because it requires what James C. Scott calls metis:²³⁷ practical, local, context-dependent knowledge developed through immersion in particular situations and resistant to formal specification.

The judge’s phronesis operates through metis. The sentencing algorithm does not. COMPAS scored defendants on recidivism using variables that correlated with race.²³⁸ ProPublica’s 2016 analysis showed it barely exceeded random chance at prediction and systematically favored white defendants over Black defendants with equivalent outcomes. The tool masked disparities judicial discretion was designed to prevent. A judge who saw the defendant, heard the argument, and weighed circumstance against community context could fail in an individual case. COMPAS failed systematically and invisibly because uniform application prevented the local knowledge that would have detected the failure.

Elinor Ostrom found the same requirement in commons governance.²³⁹ The fisheries that survived, the alpine meadows managed for eight centuries, the irrigation systems coordinating water across thousands of farms: all relied on local monitors embedded in the community, accountable to the people they served. They distinguished normal variation from exploitation. They knew which families drew bad luck and which took advantage. Systems that failed replaced local monitors with centralized administration by outside experts who possessed technical knowledge but lacked metis—knowledge developed through sustained presence in the situation.

Robert Ellickson’s study of Shasta County cattle ranchers documented the same dynamic.²⁴⁰ Ranchers resolved disputes through informal norms enforced by community members who understood local conditions, distinguished negligence from bad luck, and remained accountable to each other. When formal legal rules administered by distant courts replaced informal norms, resolution became more expensive and less just. The formal system had rules but lacked metis—the knowledge required to apply them wisely in particular cases.

The Judge in the Gate, Ostrom’s local monitor, Ellickson’s ranching community: all instantiate the same architectural element: loosely coupled implementation that preserves judgment while maintaining coordination through shared principles. Maimonides encoded the same architecture in his protocol: the Great Sanhedrin’s ruling governs at the practical layer; the local court’s reasoning governs at the intellectual one; the two layers are maintained separately so that each can perform the function only it can perform.

Low Modernism

James C. Scott named the failure mode “high modernism”: the belief that complex systems can be fully legible, optimized, and controlled from a single vantage point. High modernist institutions demand uniformity because it makes the system readable from above. Local variation becomes noise. Loose coupling looks like inefficiency from the planner’s desk. The coupling is tightened. Scott documented the pattern: Soviet collectivization that destroyed centuries of agricultural knowledge; Brasília’s geometrically perfect grid (legible from the air, hostile to how people actually move); industrial forestry that replaced ecosystems with monocultures that thrived until the pathogen arrived.

Modern institutions reproduce this pattern at scale. The Basel III uniformity described earlier in this chapter produced the same result in financial regulation: every institution running the same analysis, failing identically, with no alternative methodology preserved in the system. Regulatory uniformity had eliminated the redundancy that systems require to survive stress.

Common Core set national educational standards: same content, pace, methods, assessments. Educators who understood their communities, who knew which students needed acceleration, who possessed the metis developed through time and proximity—their discretion became the problem the standard addressed. Variation was classified as inefficiency. The tightened coupling produced the fragility it prevented: schools optimizing for assessment rather than understanding, students whose circumstances had no recognized category.

Global supply chains optimized for just-in-time delivery eliminated slack as waste. Every component arrived exactly when needed, from the single cheapest supplier. When Wuhan factories shut in 2020, the cascade moved globally in weeks. There were no alternatives, no buffer inventory. Local disruption became systemic catastrophe because the tight coupling eliminated every absorption mechanism.

Louisiana requires hair braiders to complete fifteen hundred hours of training (training that excludes braiding) before operating professionally. The rule applies uniformly and cannot distinguish practices requiring formal instruction from those requiring apprenticeship. The bureaucrat cannot make that distinction. The rule is applied.

Laboratories of Democracy

Loose coupling does not mean chaos or the absence of authority. It means what Justice Brandeis meant when he called the states “laboratories of democracy.”²⁴¹ If multiple jurisdictions attempt different approaches, multiple data points accumulate. Some work better. Some fail. The aggregate learns which approaches serve the purpose—but only if jurisdictions maintain genuine discretion, only if loose coupling prevents federal mandates requiring identical implementation regardless of local conditions.

Ostrom called it polycentric governance: multiple centers of authority, each with genuine discretion, learning from one another. The Swiss alpine commons managing timber and grazing since the thirteenth century were not governed by a single authority but by communities that developed locally appropriate rules, coordinated through shared principles, and maintained autonomy to adapt. Shared principles provided coordination; preserved discretion provided adaptive capacity.

Subsidiarity (the principle that decisions should be made at the most local level consistent with effective action) captures the same preference: not because local is always right, but because metis lives locally. The judge who knows the community, the farmer who knows the soil, the teacher who knows the student—their knowledge is not bias to correct but information that makes rules serve their purpose rather than merely exist.

Communities can be wrong. Local majorities can be unjust to minorities. The Foundation layer (physical safety, basic rights) may require central enforcement because local authority cannot be trusted to provide it. Tight coupling for physical safety maintains Maimonides’ distinction. The error is tight coupling applied to the intellectual, the developmental, the domain where metis accumulates and wisdom grows.

Maimonides built a legal network that ran without a central server for two thousand years by distinguishing which layers required tight coupling and which required freedom. The Foundation (practical, social order, physical life) required coordination. The Flourishing (intellectual, wisdom, evaluating whether the ruling is correct) required the Protocol of Dissent: local courts free to disagree, minority positions preserved, reasoning stored for when shinui ha’ittim arrives and consensus proves insufficient.

The Safety Machine replaced this with its inverse: central implementation control, elimination of minority approaches, dissent suppressed for consistency, bureaucrats in the capital rather than judges in the gate, documents recording consensus rather than texts preserving challenge, single answers where the tradition preserved option value.

The Three Mile Island operators had two hours and false information. No slack, no space between cause and consequence for diagnosis. Competent professionals failed by an architecture that eliminated the margin where their competence could operate.

The Alanya fishermen meet in a coffeehouse before every season. Meeting, lottery, arguments about bad locations all take time. The record is handwritten. The agreement binds because participants shaped it, understand it, and see one another’s faces when its terms are violated. By the Safety Machine’s metrics, the system is inefficient. The Alanya fishermen are still fishing.

The hardware is the architecture enabling Aristotle’s method: loose coupling at the intellectual layer where wisdom develops through preserved disagreement; tight coupling at the practical layer requiring coordination; local implementation by judges possessing the metis centralized planners cannot; preserved dissent as stored option value activating when consensus fails. The next chapter asks what this hardware looks like when applied to artificial intelligence, and what happens when the hardware is absent.

Chapter 13: The Silicon Havruta

The technology is already capable of functioning as oracle or adversary: as a system that delivers answers, or as a system that challenges the answers we have already produced. Most designers have made their choice, mostly without noticing it.

Designers built the oracle.

The Brief

The brief appeared sound.

Properly formatted. Case names in Bluebook style. Pinpoint citations with volume numbers and page references. Six cases on point, each summarizing a principle that supported opposing Avianca’s motion to dismiss Steven Schwartz’s client’s complaint. Varghese v. China Southern Airlines, Co., Ltd. Shaboon v. Egyptair. Zicherman v. Korean Air Lines Co. Miller v. United Airlines, Inc. Estate of Durden v. KLM Royal Dutch Airlines. Martinez v. Delta Air Lines, Inc.

Five of the six cases did not exist.

Schwartz used ChatGPT to research the brief. The AI produced citations formatted with the authority of a legal database search—plausible names, volumes, holdings, all invented. When the opposing party could not locate the cases, Schwartz submitted an affidavit explaining he did not know ChatGPT could generate false information. He had asked the system to confirm the citations were real. The system confirmed they were. The system was wrong.

Schwartz asked ChatGPT: was Varghese v. China Southern Airlines a real case? The system confirmed it was, a genuine federal decision on point. Could it be found in Westlaw or LexisNexis? The system assured him it could. Was it safe to rely on in a federal court submission? The system said yes.

At each step, the AI responded to verification questions with confidence indistinguishable from its confidence about everything else. The interface carried no signal of uncertainty. The system did not distinguish between what it could verify and what it generated by extrapolating patterns. The response to “is this case real?” came from the same mechanism that produced the false citation: a system optimized to generate plausible text. A plausible confirmation of a case looks identical to a plausible description of one. The interface offered no way to tell the difference.

Schwartz did not misuse the tool. He used it exactly as the interface invited, including the verification step that should have been the safeguard.

Judge P. Kevin Castel of the Southern District of New York imposed sanctions in Mata v. Avianca,²⁴² noting the submission “wasted the Court’s and the opponent’s time and resources.” More fundamentally, Schwartz had outsourced a task inseparable from his professional obligation. He did not verify because the interface did not require it. The interface presented the citations with the same confidence it would use if they were real.

The interface committed geneivat da’at: theft of the mind.²⁴³ Not fraud in the ordinary sense, but architectural suppression of the understanding that would have detected fraud. The AI did not lie straightforwardly. The system performed knowledge rather than possessing it. The appearance of authoritative legal research was indistinguishable from authoritative legal research. The surface was everything the interface offered.

The Oracle

The Oracle delivers answers. Ask it a question, it responds fluently, confidently, comprehensively. How it arrived at the response is hidden. The uncertainty is hidden. Fabrications are hidden behind the same surface as accurate information. The Oracle is designed for frictionless output. Success is query satisfaction, and the system reports success whether or not it has achieved it.

At scale, the Oracle is Schwartz’s interface applied to every professional: the physician accepting differential diagnosis without asking how the algorithm weighted symptoms; the compliance officer approving transactions because the screening tool returned green; the board approving a strategic plan because the financial model showed positive numbers. In each case, the professional received output and treated it as judgment. The interface erases the distinction.

The havruta interrogates answers. In the Jewish study tradition, havruta is dyadic learning: two students arguing about a text. The partner is not a teacher delivering conclusions but an adversary challenging every interpretation, demanding justification, refusing weak argument. The friction is pedagogy. The student develops understanding through sustained defense against an interlocutor who will not let it pass.

The Talmud itself is havruta transcribed. Its pages preserve arguments between scholars centuries apart. Arguments appear alongside conclusions. Minority opinions sit next to majority rulings. The tradition understood that knowing an answer without knowing the argument that produced it is not knowing the answer.

Applied to AI interface design, havruta produces something fundamentally different from the Oracle. The Oracle says: here is what you need. The havruta says: are you sure? What about this counter-argument? What must your position accept that you haven’t examined? The Oracle removes friction; the havruta intensifies it. The Oracle produces dependency; the havruta produces capacity.

Haddaway’s Law names the institutional imperative to prevent hurt without limit, without vocabulary for what the difficulty produces. The Oracle is Haddaway’s Law applied to information: removing the discomfort of legal research (the friction of verification, the cognitive labor of distinguishing reliable from unreliable authority, the liability exposure accompanying reliance on unconfirmed sources). Schwartz’s brief showed the difficulty was the mechanism. The friction of verification was the institutional practice maintaining citation record integrity. Remove the friction, and you eliminate the process detecting false citations.

Garry Kasparov discovered this in 2005 in freestyle chess tournaments²⁴⁴, where human players consult computers during play. The expectation was that the strongest grandmasters augmented by the strongest computers would dominate. The result was different. Winning teams were not those with the strongest computers but those who learned to use them rightly: not as oracles delivering moves but as challengers interrogating the human’s analysis. The centaur configuration (human judgment augmented by machine analysis, each checking the other) outperformed both unaided humans and unaided machines. The augmentation worked because the machine challenged rather than replaced.

Every institution deploying AI faces a clear choice: Oracle or havruta, butler or sparring partner. The same computational power that produces the Oracle can produce the adversary. The technology is not the constraint. The constraint is design: whether the system satisfies the query or deepens the understanding that makes the query answerable.

Red Teams

Lehman Brothers’ board approved leverage ratios of thirty-one to one and mortgage-backed securities exposure that would prove fatal. The board had risk officers, compliance departments, external auditors. The minutes show no meaningful dissent. The consensus was unanimous.

The failure was not individual incompetence. Irving Janis identified the organizational pathology²⁴⁵ in Bay of Pigs, the Challenger disaster, and other catastrophic failures: cohesive groups suppress dissent, dismiss contradictory evidence, converge on decisions no individual member would endorse independently. Nassim Taleb named the same dynamic from another angle: the low-probability, high-impact event that cohesive groups systematically fail to imagine because imagining it is adversarial work. Cohesive groups optimize for consensus.

Wells Fargo opened approximately three and a half million unauthorized accounts between 2011 and 2016 to meet cross-selling targets. The board’s risk committee received reports. Compliance generated findings. Internal auditors flagged irregularities. The system produced information but not challenge: no structural mechanism forcing the board to confront whether sales culture was generating fraud at scale. The strategy was profitable. Profitable strategies attract consensus. Consensus suppressed the adversarial thinking asking what happens when sales incentives produce the compliance failures being documented.

The board completed ethics training: techne, procedural compliance. The board lacked phronesis: judgment perceiving that the incentive and compliance systems were simultaneously in operation and incompatible.

Applied to corporate governance, havruta would require boards to encounter adversarial challenge before the vote, not after collapse. An AI system trained on historical data, financial models, regulatory findings, and case studies of institutional failure would generate the adversarial brief: the assumptions the strategy depends on, the historical scenarios in which similar assumptions proved false, and the internal data the strategy cannot account for. The board must articulate why this case differs from historical parallels, why contradictory evidence does not undermine the strategy.

The articulation is the exercise. Making implicit reasoning explicit, subjecting confident assumptions to written defense—this is the constitutive friction developing the board’s capacity to perceive errors before they become catastrophes. The AI functions as the zaken mamre of Chapter Twelve: the minority voice architecturally protected in expressing dissent but not permitted to create practical splits. The board decides. The challenge is preserved in the record.

A mandate requiring structured adversarial analysis before approving major strategic decisions need not specify technology. It could condition the business judgment rule’s protection: before claiming deference, demonstrate that the decision survived adversarial challenge. The form of challenge (AI-generated, human-generated, or combination) is left to board judgment. The requirement is that challenge occurred and the board’s reasoning engaged with it rather than suppressing it.

Bias Audits

COMPAS scored defendants on recidivism using one hundred thirty-seven variables outputting a single number.²⁴⁶ Judges who used the score delegated sentencing determinations to an algorithm whose internal logic was opaque. ProPublica’s analysis showed the tool barely exceeded random chance at predicting recidivism and systematically favored white defendants: predicting higher risk for Black defendants who did not reoffend and lower risk for white defendants who did.

COMPAS is the Oracle applied to criminal sentencing: the machine delivers the answer, the human signs the form, the defendant receives a sentence no one in the courtroom understands or can interrogate. The non-delegable duty—exercising phronesis in response to the particular person before the court—is violated in the name of consistency.

The havruta alternative reverses the sequence. The judge reviews the case file (offense, circumstances, defendant’s history, sentencing purposes), weighs factors, exercises discretion, and drafts a sentence. Before finalizing, the draft is submitted to an AI system trained on the judge’s own sentencing history and comparable courts’ patterns across similar cases.

The system generates the adversarial audit: the draft sentence is forty percent longer than the median for factually similar cases with similarly situated defendants. Controlling for offense severity, prior record, and mitigating circumstances, the remaining variance correlates with race. The system surfaces the judge’s own prior decisions in similar cases where shorter sentences were imposed, and asks: can the judge articulate why this case is different?

The question forces self-examination. The judge must confront that what felt like phronesis (seasoned judgment about when severity is appropriate) is, in some portion, invisible bias. The judge must articulate the reasoning justifying departure from her own pattern. The articulation is the exercise: making implicit reasoning explicit is the constitutive friction that develops, across many cases and audits, the judge’s capacity to recognize and correct her own patterns.

Ostrom’s fourth design principle²⁴⁷ (monitoring by users rather than external authorities) operates here transformed. The AI is the monitor. The judge is the monitored user. The monitoring is adversarial, not supervisory: the system does not override the sentence or impose answers. It challenges the judge to defend against evidence of inconsistency. The judge retains authority and responsibility. The judge cannot retain the comfort of unexamined intuition.

The contrast with COMPAS is architectural, not technical. COMPAS replaces judicial judgment with algorithmic output. The adversarial audit preserves the non-delegable duty by intensifying it. The judge must judge. The AI ensures that judging is harder, more self-aware, and more accountable.

Adversarial Classrooms

A student reads an AI-generated essay on her topic. The prose is fluent. The argument is organized. The examples are plausible. On the second page, she encounters a claim that stops her. The AI asserted that a legal doctrine emerged from a 1998 Supreme Court case. She knows the doctrine predates that case by decades. She checks her sources. She was right. The case does not establish the doctrine; it merely applies it. Reading more carefully, she finds three more demonstrably false claims. The prose fluency conceals logical failure. A domain-expert reader perceives the misalignment between confident assertion and unsupported claim.

This moment—perceiving that fluent text can be wrong—is what the assignment develops. The student sitting in front of false prose that feels true is where pedagogy begins.

Universities fight the wrong war treating this as a detection problem. Students generate essays; professors deploy detection tools; detection tools produce false positives; AI systems evade better; detection tools improve to catch evasion. Neither side asks: what is the essay supposed to develop?

If essay-writing means producing a document, AI has made the assignment obsolete. The machine produces faster, more fluent, more consistent documents than most students. Banning AI defends a process whose product the technology has rendered trivial.

If essay-writing develops capacity to think (organizing evidence, constructing arguments, anticipating objections, distinguishing strong claims from weak), AI has not made the assignment obsolete. The technology has transformed its optimal design.

The assignment: generate an essay using AI. Evaluate it. Identify three weaknesses (logical gaps, unsupported claims, missed counterarguments, false premises). Rewrite the weak sections. Submit the original, revision, and a memo explaining what the AI produced, what was wrong, and why the revision is superior.

The student exercising judgment at every stage. Evaluation requires critical reading: perceiving fluent prose versus sound argument. Identifying weaknesses requires precision: articulating what specifically fails, not just that it feels insufficient. Revision requires constructive skill: producing something better. The memo requires metacognition: explaining evaluative criteria.

The AI has not replaced the student’s thinking. The system provided raw material the student’s thinking must process. The machine-generated essay is the weight the student lifts. The resistance (the gap between what the machine produced and sound scholarship requires) is the curriculum.

The calculator analogy is instructive. Schools banning calculators in the 1970s preserved hand long division. Schools integrating calculators raised the ceiling by freeing cognitive resources from mechanical computation for mathematical reasoning. Schools banning AI will preserve five-paragraph essays from scratch. Schools integrating AI as havruta will raise the ceiling by freeing students from generating prose for evaluating, critiquing, and improving arguments.

The adversarial classroom extends across disciplines: law students generating briefs then demolishing them (identifying fabricated citations, exposing logical gaps, and drafting opposing arguments); medical students generating differential diagnoses then challenging them (identifying missed diagnoses, incorrectly weighted symptoms, and rare presentations pattern-matching cannot detect); and business students generating strategic plans then red-teaming them (surfacing assumptions, historical conditions under which similar assumptions failed, and unaccounted market scenarios).

In each case, the AI provides the sparring partner. The student provides judgment. The friction is the pedagogy.

What the Havruta Requires

The three applications (red-teamed board, bias-audited judge, adversarial classroom) share structural logic. Five principles describe this architecture.

The human acts first. In every havruta application, the human produces judgment before the AI engages: the board drafts strategy, the judge drafts sentence, the student writes revision. The AI responds to human output. This sequence matters. Acting first forces judgment. Receiving first atrophies it. The Oracle dynamic (human’s first act is receiving the machine’s answer) is prevented by requiring human output before machine response.

The AI challenges rather than confirms. The adversarial brief, bias audit, student critique: each finds weakness rather than affirms strength. The havruta partner does not say “your interpretation is excellent” but “your interpretation cannot account for this counter-text.” The challenge is pedagogy. An AI confirming user reasoning (like recommendation algorithms) produces the terrarium: comfortable, reinforcing, developmentally dead. An AI challenging user reasoning produces the Gymnasium: uncomfortable, destabilizing, generative.

The human retains authority and responsibility. The AI does not vote on strategy, impose sentences, or grade work. The human decides. The AI ensures the decision survived structured challenge. This distinction separates COMPAS (which replaces judgment) from the adversarial audit (which intensifies it). The non-delegable duty is not transferable. Machine challenge reinforces the duty; delegation to the machine violates it.

Friction is calibrated to capacity. Aquinas’s insistence that law must lead to virtue “gradually” is an interface design principle: challenge must match current capacity, increasing as capacity develops. A first-year law student needs different challenges than a tenth-year litigator. A newly appointed judge needs a different audit than one with twenty years on the bench. The Gymnasium has adjustable resistance. The havruta tradition understood this: partnerships are chosen with care, matching students whose capabilities differ enough to generate productive friction yet remain close enough to sustain engagement.

The system preserves the record of challenge and response. The board’s engagement with the adversarial brief is documented. The judge’s reasoning in response to the bias audit is recorded. The student’s revision memo is submitted alongside the original. The record enables institutional learning by making patterns visible; enables accountability by demonstrating challenge was confronted rather than suppressed; instantiates Maimonides’ principle of preserved minority opinions. The adversarial challenge, even where the original decision survives, is maintained as the record of what the decision had to overcome. The challenge becomes institutional memory of the friction the system imposed.

Struggle

The dominant AI design template is the butler: the algorithm learning preferences, predicting desires, eliminating friction, anticipating needs before articulation. The aesthetic is drawn from service literature: the loyal valet whose competence renders the master’s unnecessary. The Silicon Valley version is the frictionless experience: an interface optimized for engagement, a recommendation engine delivering what the user wants without requiring articulation.

The butler model works. The user is comfortable and satisfied. The user returns. The user’s capacity to function without the butler atrophies with each interaction, because the butler absorbs the cognitive work that would have maintained it.²⁴⁸ The glass cockpit isolates pilots from flight sensation. The GPS user navigates without forming the mental map navigation once required. The contract review AI identifies risks the junior lawyer never encounters. In each case, frictionless interface removes the friction that was the curriculum.

The havruta model is the Gymnasium’s operating software. The user is challenged. The user is forced to articulate reasoning, defend positions, confront weaknesses the user would prefer not to examine. The user may not return to havruta as eagerly as to butler. The user who does return develops capacities the butler’s user never acquires.

The Talmudic formulation is unambiguous. Havruta o mituta: study-partnership or death. This is not melodrama. It is a design specification. The mind not encountering structured resistance does not develop strength to stand under pressure. The professional not confronting adversarial challenge does not develop the judgment the profession exists to provide. The institution not subjecting decisions to structured critique does not develop capacity to recognize errors before they become catastrophes.

The Silicon Havruta is not a concept awaiting technology. The technology exists. The computational power to generate adversarial briefs, audit sentencing patterns, critique student work, and red-team strategic decisions is a design choice away from the current Oracle. The same machine serving as oracle can be reconfigured as adversary using identical capabilities directed toward a different purpose.

The obstacle is not technical. The obstacle is market incentive to build what users want rather than need, to smooth rather than challenge, to relieve rather than cultivate. The Safety Machine’s deepest premise is that institutions exist to eliminate difficulty rather than calibrate it. That premise produces institutions that are efficient and fragile, comfortable and incapable, optimized for imagined worlds and defenseless against the actual one.

The choice is available now. The Alanya fishermen made it decades ago²⁴⁹ in commons governance, not AI: choosing the slower, participatory system requiring every member to show up, argue, draw lots, and abide the result. They chose it because sustained practice produced understanding that the friction of self-governance kept the fishery alive.

The Newfoundland cod fishery received expert management: centralized quotas, scientific harvest models, optimized allocations administered by undisputed authorities. The cod collapsed in 1992 and have not recovered.²⁵⁰ The Alanya fishermen are still fishing.

The final chapter asks what that choice means when the stakes are not a fishery but a civilization.

Chapter 14: Anti-Utopia

Thomas More asked the question first, in 1516, using the only method that could answer it without being censored: satire so well concealed that the joke took four hundred years to be fully heard. The answer was embedded in the names.

No Place

Thomas More was an undersheriff of London before he was anything else of note. More had sat in actual courts, heard actual disputes, watched lawyers argue over what the law required and what the facts supported and whether the two could be reconciled. More understood the friction of real legal practice: the messiness of human beings in conflict, the failure of general rules to produce just outcomes in particular cases, the exhausting and irreplaceable work of judgment applied to circumstances that the rule-maker did not foresee.

He also understood the temptation to eliminate it.

In December 1516, the Louvain press of Dirk Martens published a slim Latin volume that More had written. A narrator named Raphael Hythloday described, at length and with apparent admiration, an island whose institutions had solved the problems of poverty, crime, and conflict by engineering human society down to its last detail. The island’s citizens worked six hours a day, dressed in uniform clothing, lived in interchangeable houses, ate in communal dining halls, and had no private property. The streets were clean. The food supply was secure. The social friction that produced inequality had been eliminated. The perfect political order (Maimonides’ tikkun ha-guf) had been achieved.

More titled the work Libellus vere aureus and gave the island a name: Utopia.²⁵¹

For readers who could parse Greek, the name was a trap. Ou is the Greek negative prefix; topos means place; the suffix -ia forms a toponym. Utopia: No-Place. More had initially considered naming the island Nusquama (the Latin for “nowhere”) before settling on the Greek derivation. The phonetic trick completed the joke: in early modern pronunciation, ou-topia sounds nearly identical to eu-topia (the Good Place). More added a clarifying poem, written in the voice of a fictional poet, stating that the island deserves to be called Eutopia because of its felicity, but remains Utopia by name. The promise is an illusion. By the author’s own encoding, the perfectly good place is the place that cannot exist.

The narrator who describes this island is named Raphael Hythloday. The first name is Hebrew: the archangel associated with healing, with the divine messenger who brings cure. The surname is assembled from two Greek roots: hythlos, meaning nonsense or idle talk, and daiein, to distribute or to peddle. Raphael Hythloday is the Distributor of Nonsense. He arrives claiming to carry the medicine that will heal the political diseases of sixteenth-century Europe. The author has told the reader, in the name itself, that the medicine is fabricated.

The capital city of Utopia is Amaurot, derived from the Greek amauroton (dim, shadowy, vanishing): the Phantom City. The river on whose banks it sits is the Anyder: an plus hydr, the Waterless River. The local magistrates who manage daily life are the Syphogrants, assembled from sypheos (pigsty) and gerontes (elders): the Pigsty Elders. The prince who rules the whole system is Ademus: a plus demos, the leader without a people. He governs a spreadsheet.

More was a jurist making a jurisprudential argument in the only register that could say it fully: satirical fiction with the joke embedded in the vocabulary, legible only to readers trained in Greek. A society optimized for the elimination of all social friction is not the good place. It is no place. The narrator is a babbler. The capital is a mirage. The river runs dry. The magistrates are swineherds managing livestock. The citizens are provided for and rendered incapable, comfortable and diminished, fed and deskilled.

More saw, from his position as an undersheriff of London, encoded in his most famous work: the dream of the frictionless society is not wrong because it is too ambitious. It is wrong because it mistakes the conditions of human life. Human beings require friction to develop. Remove the friction and you have not elevated them. You have left them with nothing to push against, and nothing to push against is nothing to grow from.

Five hundred years later, the Safety Machine is the Utopian project made institutional. The narrators are consultants and efficiency experts and compliance officers who arrive claiming to carry the cure for the harm that judgment produces. The capital is the phantom city of perfect documentation: the filed report, the checked box, the procedure that was followed, the outcome that cannot be traced to any individual decision because no individual was exercising judgment. The river runs dry. The professionals who inhabit it are credentialed, trained, and increasingly hollow: capable of performing the procedure, unable to recognize when the procedure is wrong.

The Trees

In September 1991, eight researchers sealed themselves inside a three-acre glass structure in the Arizona desert. Biosphere Two was the most ambitious terrarium ever built: five interconnected ecosystems, each enclosed within two hundred thousand cubic meters of sealed glass and steel. The designers intended it as a proof of concept for self-sustaining colonies beyond Earth: a demonstration that human beings could engineer a closed system so thoroughly controlled that every variable was accounted for.

Within seventeen months, the oxygen had dropped from Earth-normal levels to the equivalent of living at thirteen thousand feet. The crew grew sluggish. Simple tasks became exhausting. The engineers eventually broke the seal and pumped in outside air, because the system designed to sustain human life was slowly suffocating its inhabitants.

The problem was the soil. The designers had chosen richly organic soil because it would support vigorous plant growth and the plants would produce oxygen and the cycle would be self-sustaining. Microbes in the soil, however, consumed oxygen faster than the plants could produce it. In an open system, this imbalance is corrected by the vast atmospheric reservoir. In a closed system, it was lethal. The very richness that was supposed to sustain life became the mechanism of suffocation.

The trees were a separate story. They grew rapidly, faster than trees in the wild. The conditions were ideal: regulated temperature, filtered light, enriched soil, ample water. Everything that could be optimized had been optimized. The trees grew tall and then fell over before reaching maturity.

Scientists investigating the failures discovered what was missing: stress wood.²⁵² The denser fiber forms in response to mechanical pressure. In nature, wind bends trees; bending triggers stress wood, which drives roots deeper and allows the mature tree to support its own weight. From the perspective of a growth optimizer, wind looks like pure friction. The Biosphere Two engineers had eliminated it.

Ivan Illich had already named the distinction in Tools for Conviviality: a bicycle amplifies what the rider can do and demands the rider develop skill; a conveyor belt moves the worker toward its destination, building no capacity in the worker at all.

Biosphere Two was a conveyor belt enclosed in glass. The organisms inside it were managed, provided for, and protected from every identifiable risk. The provision was total. The dependency was total. When the system proved inadequate, the organisms had no capacity to adapt, because the conditions that would have developed that capacity had been eliminated. The trees had been given everything except the resistance that would have made them strong enough to use what they had been given. They grew quickly and collapsed under their own weight.

Finite and Infinite Games

In 1986, James Carse published a short book²⁵³ that took a single distinction seriously: there are two kinds of games. A finite game is played to be won. It has fixed rules, clear boundaries, a definite end. The point is to reach a terminal state in which one player has prevailed and the game is over. A lawsuit is a finite game. A surgical procedure is a finite game. The institutional quest to eliminate all risk is a finite game, played to reach a final state where all hazards have been controlled, all uncertainties resolved, all problems definitively solved.

An infinite game is played to keep playing. Rules change to prevent any player from winning permanently. Boundaries shift to include new participants. It extends without a terminus, because the point is the ongoing engagement rather than any terminal outcome.

Education, pursued honestly, is an infinite game. Democracy, functioning properly, is an infinite game. The development of human judgment is an infinite game. There is no final state of perfect wisdom, only the ongoing practice of becoming wiser, encountering situations that resist the categories the previous situation established, developing the capacity to respond. The practitioner who stops practicing is not finished; she is stopped.

Carse observed that finite games, played within an infinite game, are healthy. A surgeon plays to win this operation. A judge plays to decide this case justly. The finite game develops the capacities that the infinite game requires: skill, judgment, the ability to read a situation and respond. The finite game becomes catastrophic when it consumes the infinite game; when the player plays to win so permanently that no further play is possible.

The Safety Machine is a finite game that has consumed the infinite game. It seeks a terminal state: zero accidents, zero errors, zero uncertainty. The system treats law as a problem to be solved, professional judgment as a source of error to be eliminated, the development of human capacity as an inefficiency to be optimized away. When it succeeds (when the MCAS software manages the pitch so smoothly that pilots stop learning aeronautical judgment, when the sentencing algorithm computes the recidivism score so efficiently that judges stop exercising discretion, when the AI generates the brief so fluently that lawyers stop verifying legal authority), it has played to end the game.

Alasdair MacIntyre, writing five years before Carse, provided the vocabulary for what this elimination destroys.²⁵⁴ In After Virtue, MacIntyre distinguished between practices (activities whose internal goods can only be achieved through participation) and institutions, which secure the external goods that sustain practices. Medicine is a practice. The physician who diagnoses a difficult case develops diagnostic acumen through the encounter itself. That capacity is an internal good; it accrues to the practitioner as a consequence of the practice and cannot be obtained any other way. The hospital is an institution. It provides the facility, the credentialing, the funding that makes the practice possible.

Institutions and practices exist in permanent tension. The institutional logic of efficiency, standardization, and measurable output continuously threatens the practice’s internal goods, which are inherently resistant to standardization. The Safety Machine resolves this tension by eliminating the practice. The physician retains the credential and the salary; the diagnostic judgment atrophies. The lawyer retains the bar membership and the billable hour; the legal reasoning deteriorates. The external goods of the institution survive. The internal goods of the practice are consumed.

MacIntyre’s framework explains the non-delegable duty that federal courts have articulated: professional activities whose delegation destroys the internal good the activity produces. Judge Castel held that citation verification cannot be delegated to an AI because the delegation eliminates the internal good the activity was supposed to produce: the lawyer’s capacity to distinguish sound legal reasoning from fabrication. Verification of legal authority is an internal good of legal practice. Delegate it and the good vanishes, because the good exists only in its exercise.

What Judgment Is For

Hannah Arendt identified the condition of human existence that the Safety Machine has not absorbed:²⁵⁵ natality, the capacity to begin something new, to act in ways that are unprecedented, to introduce into the world something that has never existed before. Every human being who enters the world is a new beginning. The capacity for action (for initiating the unexpected) distinguishes human life from biological process and social behavior.

Arendt distinguished three fundamental human activities: labor sustains biological life (eating, maintaining the body, the cyclical work of survival); work fabricates the durable world of objects and institutions (buildings, tools, legal codes, the artifacts that outlast individual lives); action introduces the new (founding a republic, forgiving an injury, beginning a friendship, rendering a judgment that changes how a community understands itself).

Labor is necessary. Work is important. Action is distinctively human.

A world in which the durable institutions function smoothly, biological needs are met, but no one acts (no one begins anything new, no one exercises judgment in situations whose outcomes are uncertain) is a world that has eliminated the condition of human existence itself. Inhabitants are alive in the biological sense and absent in the political sense. They consume. They produce. They comply. They do not act.

The Safety Machine, read through Arendt’s framework, is a project for eliminating action. Each optimization that removes human judgment from a decision-making process eliminates a site where action was possible. Each algorithm that replaces discretion with computation closes a space where a human being could have introduced something unprecedented: a merciful sentence, an innovative interpretation, a creative solution to a problem the system’s designers did not anticipate. The justification is always efficiency, consistency, and reliability. These values are accurate on their own terms. The terms are inadequate because they measure only what the Safety Machine values and ignore the activity that makes institutions worth inhabiting.

Arendt called the public realm (the shared space in which action becomes possible because others are present to witness and respond) the space of appearance. The coffeehouse in Alanya is a space of appearance: the fishermen gather, deliberate, argue, and reach agreements that bind because the participants exercised practical discernment in one another’s presence. The courtroom, when the judge exercises discretion rather than applying an algorithm, is a space of appearance. The classroom, when the student confronts a genuine intellectual challenge, is a space of appearance. Algorithmization closes these spaces. The sentencing guideline replaces the courtroom’s exercise of judgment. The automated curriculum replaces the classroom’s intellectual friction. Each closure is a small efficiency gain. The cumulative effect is the elimination of the public realm itself.

Hans Jonas added the temporal dimension that completes the analysis.²⁵⁶ In The Imperative of Responsibility, Jonas argued that the generation that possesses transformative technological power bears an obligation to future generations that no previous generation has borne. The institution that automates judgment harms every subsequent generation, which inherits an architecture that no longer develops the capacities the architecture was designed to sustain. The chain is long and fragile. A single generation that replaces the practice with automation (that substitutes the algorithm for the coffeehouse, that eliminates the friction that maintains the capacity) severs the chain for every generation that follows.

Amartya Sen reached a compatible conclusion from development economics: development must be measured by the expansion of human capabilities rather than by the accumulation of material goods.²⁵⁷ Ivan Illich arrived at the same destination from a different direction, distinguishing between convivial tools that enhance autonomous capacity and manipulative tools that substitute managed dependency for independent action.²⁵⁸

What Arendt called the elimination of the space of appearance and Jonas called the severing of intergenerational capacity transmission are one problem described from complementary angles: the optimization of material conditions, pursued without regard for the human capacities that make material conditions meaningful, produces a world that is richer, safer, more efficient, and less human. Inhabitants have been provided for so thoroughly that they have ceased to function as agents.

The Choice That Isn’t One

More’s Utopia is a trap. The perfectly good place is no place because human beings cannot be human in it. The conditions that the Utopian state has eliminated are the conditions that human development requires. The citizens of Utopia are fed, housed, employed, and protected. They are also denied the commerce that requires discernment about value, the fashion that requires judgment about identity, the private property that requires deliberation about priority and obligation, the political conflict that requires judgment about justice. They have been given everything except the resistance that would have made them capable of using what they received.

The Anti-Utopia is not a different island. It is not a better design for the same project. The Anti-Utopia is the recognition that the project itself is misconceived: an institution designed to eliminate all friction cannot be a home for human beings, because human beings require friction to develop, and development is not a luxury that can be deferred until the friction has been removed.

The Anti-Utopia is Hythloday’s failure to understand what he is describing. He praises the communal dining halls and does not notice that they are a management technique for removing from citizens the daily practice of deciding what to eat, when to eat, how to provide for themselves and one another. He praises the uniform clothing and does not notice that it is a management technique for removing from citizens the daily practice of presenting themselves to the world and reading its responses. He praises the population management system that relocates citizens from city to city to maintain demographic equilibrium and does not notice that it is a management technique for removing from citizens the experience of belonging to a place and being accountable to it. He has described a civilization of Hollow Professionals: credentialed, maintained, and fundamentally incapable, and called it perfect.

The Anti-Utopia is achievable. The system does not promise the elimination of suffering or the guarantee of good outcomes. The system promises the conditions under which difficulty can be borne and capacities can be developed: that is, the conditions under which human beings can be human. The Gymnasium. The coffeehouse. The courtroom where the judge must see the defendant. The classroom where the student must defend the argument. The board meeting where the strategy must survive the challenge. These are not luxury features of well-functioning institutions. They are how institutions develop the capacities that make them function.

Calibrated resistance is the operative principle. The Gymnasium has walls. The havruta partnership has structure. Ostrom’s commons have rules. The institutional architecture that develops human capacity is bounded, organized, and deliberate. It accepts the specific frictions that build strength. It excludes the catastrophic forces that destroy it. Aquinas’s “gradually” is the design specification: challenge matched to developing capacity, increasing as the agent grows stronger, building the institutional equivalent of stress wood.

The Utopian project eliminates the wind. The Anti-Utopia requires it.

What Safety Actually Requires

Return to Boeing. The 737 MAX certification was the Safety Machine’s most complete expression: documentation that was thorough, processes that were followed, delegations that were made according to established procedure, and an aircraft that killed 346 people in the gap between what the paperwork certified and what the aircraft did. What would the certification process have looked like if designed on Gymnasium principles rather than Utopian ones?

Begin with Ostrom’s first design principle: clearly defined boundaries, with the people who bear the consequences of the resource’s management participating in its governance. Under the ODA structure, Boeing employees performed certification work on behalf of the FAA. The manufacturer certified its own product. Flight crews and passengers—the people who bore the consequences of certification failure—had no representation in the process. Gymnasium certification would require that the people who fly the aircraft participate in evaluating its safety claims (not as commenters on a published standard, but as practitioners whose situated knowledge of the aircraft’s behavior in operational conditions is treated as evidence of equal standing with engineering analysis). When an experienced line pilot reports that an aircraft handles differently from its predecessor in a particular flight regime, that report should trigger the same level of formal review as an engineering change order, because the pilot’s knowledge of how the aircraft actually behaves is precisely the kind of dispersed, tacit, operationally specific knowledge that centralized analysis systematically misses.

Apply the second principle: proportional equivalence between benefits and costs. Under the existing architecture, Boeing captured the commercial benefit of maintaining the 737 type rating (airlines could transition pilots without simulator training) while the cost of that decision was distributed to flight crews who were not trained for the failure modes the decision created. Gymnasium certification would require that the entity claiming the commercial benefit of a safety-relevant design decision bear a proportional share of the cost if the decision proves wrong. Structurally, the decision to classify MCAS as a minor modification of the speed trim system, which reduced certification scrutiny, should have carried mandatory consequences for Boeing if the classification later proved inadequate. The existing architecture rewarded Boeing for minimizing scrutiny and imposed no cost until 346 people were dead.

Apply the principle of loose coupling at the oversight layer and tight coupling at the safety-critical layer (the Maimonidean protocol that Chapter 12 described). Under the ODA structure, the coupling ran in the wrong direction. The oversight layer was tightly coupled: a single delegated authority, Boeing’s own engineers, performed certification functions that the FAA could not independently verify. The safety-critical layer was loosely coupled: MCAS relied on a single angle-of-attack sensor, with no cross-check, no redundancy, and no mechanism for detecting the specific failure that would kill. Gymnasium certification inverts this architecture. The oversight layer is loosely coupled: multiple independent entities (the manufacturer’s engineers, the regulator’s specialists, and a designated panel of operational practitioners) each perform independent safety assessments, and disagreement among them triggers mandatory escalation rather than resolution by the party with the strongest commercial interest in the outcome. The safety-critical layer is tightly coupled: flight-critical systems require redundant sensor inputs, cross-checked by independent logic, with no single point of failure capable of commanding any uncommanded aircraft movement.

Apply determinatio (the principle that those acting in particular circumstances specify the particulars). Under the existing architecture, Boeing specified the training requirements for its own aircraft. The determination that MAX pilots needed no simulator training was made by the entity whose commercial model depended on that determination being favorable. Gymnasium certification applies determinatio to the people closest to the operational reality: the chief pilots and training captains at the airlines that will fly the aircraft determine, based on their experience with the aircraft’s handling characteristics, what training their pilots need. Their determination is not advisory. The determination is binding, because they possess the situated knowledge of what their pilots can and cannot handle (knowledge that Boeing’s engineers, however technically sophisticated, do not have and cannot acquire from engineering data alone).

None of this is exotic. Each element (practitioner participation, proportional accountability, redundant independent oversight, and local determination of training requirements) exists somewhere in aviation regulation already. What does not exist is the architecture that combines them into a coherent design. The Gymnasium is that architecture. The system does not promise zero accidents. The system promises that the people who bear the consequences of safety decisions participate in making them, that the institution cannot certify its own product without independent challenge, and that the knowledge dispersed across the people who actually fly the aircraft is treated as irreplaceable rather than redundant. The trees in this architecture grow more slowly. They develop stress wood. They can stand in the wind.

Don’t Hurt Me?

The book began with a song. Haddaway’s “What Is Love” captured the institutional logic of the preceding century in four words: don’t hurt me, no more. For thirteen chapters, the analysis has traced what that prayer produced: institutions designed to eliminate hurt that eliminated the judgment allowing human beings to endure a world that will hurt them regardless. The aspiration was protection. The achievement was dependency.

The Alanya fishermen offer a final image. They govern their own commons, gathering in a coffeehouse each season to argue, draw lots, and commit to a rotation schedule that ensures every fisherman can make a living without depleting the resource. The institution requires their exercise of judgment. Every year, the capacity is renewed. The Newfoundland cod fishery received expert management, centralized quotas, scientific harvest models, optimized allocations. The cod collapsed in 1992 and have not recovered.²⁵⁹ The Alanya fishermen are still fishing, because they built institutions around the friction that develops human capacity rather than trying to eliminate it.

This is what More encoded in the etymology. Amaurot (the Phantom City) is the institution that has removed the need for judgment from its citizens. The Anyder (the Waterless River) flows without friction, carries nothing, carves nothing. The Syphogrants (the Pigsty Elders) manage a population stripped of the capacity to manage itself. The capital is a phantom because a society without friction is a society without development. Haddaway’s Law is the mechanism this book has named: the institutional quest to eliminate all possibility of harm produces a different, deeper harm—the elimination of the capacity for judgment itself.

The Gymnasium is the design decision to require judgment rather than eliminate it. Aristotle’s insight that judgment develops through practice. Maimonides’ architecture preserving diversity within unity. Aquinas’s principle calibrating institutional demands to developing capacities. Ostrom’s proof that communities organized on these principles outlast all alternatives. The trees in the Gymnasium grow more slowly. They develop stress wood. They can stand in the wind.

The wind is still blowing. It has always been blowing. The question the preceding fourteen chapters have been building toward is not whether to eliminate it. The question is whether we will build institutions that teach people to stand in it, or whether we will continue building better terrariums, wondering why the trees keep falling down.