Welcome to the VBNN Digital Library
Unlock a Vast Knowledge Ecosystem
Featuring over 30,000 books, academic papers, illustrations, and expert insights—continuously updated to support your research and professional growth.
Welcome to our library!
Here, you will find an exclusive collection created 100% by our own faculty, meaning you will not find these resources anywhere else. Over the last 20 years, our team has written much more than what is currently online, and we are actively working to upload our complete back catalog. We update our platform regularly, so be sure to check back from time to time. If you ever need help finding a specific resource, you can always contact us!
Maximize Your Access
Log in to instantly view and download tailored resources directly aligned with your specific program and curriculum.
Ready to begin? Sign in above to explore your personalized dashboard.
Please note: Login is only possible using your institutional email address; otherwise, the system will not recognize your account.
VBNN Library AI
Introducing our fully integrated Library AI. Designed to support your research, you may submit inquiries in any language and receive precise, evidence-based responses drawn exclusively from our published scholarly articles and textbooks.
Search...
Latest Publications:
Search this site
Results found for empty search
- Law and Economics (Efficiency, Rational Choice, and Legal Incentives)
Download the Book (PDF): Introduction In January 1944 a barge called the Anna C was tied up at a pier on the North River in New York Harbor, loaded with flour that belonged to the United States government. A tug working in the crowded harbor needed to get at another barge further out in the tier, and its crew readjusted the lines that held the Anna C in place. They did the job badly. The barge worked loose, drifted, struck a tanker whose propeller punched a hole in her hull, and sank with her cargo. Nobody was hurt. The dispute that followed was about money: who should pay for the lost flour and the lost barge. The answer turned on a small fact. The barge's owner employed a bargee, a man whose job was to stay aboard and look after the vessel. On the day of the accident he was not there, and had not been there for many hours. Had he been aboard, he might have noticed the barge drifting and called for help before she sank. So the question became whether the barge owner had been careless in leaving the vessel unattended, and whether that carelessness should reduce what the owner could recover from the tug company. Judge Learned Hand, writing for the Second Circuit in United States v. Carroll Towing Co. (1947), answered with a sentence that has become the most famous in the American law of accidents. The owner's duty to guard against harm, he wrote, was a function of three variables: the probability that the barge would break away, the gravity of the resulting injury if she did, and the burden of adequate precautions. Then he put it algebraically. If the probability is called P, the injury L, and the burden B, liability depends upon whether B is less than L multiplied by P. That sentence did not change the law. Negligence had meant the failure to take reasonable care for a century before Hand wrote, and it meant the same thing afterwards. What the sentence did was expose something that had been sitting inside the idea of reasonable care all along. When courts asked whether a defendant had behaved as a reasonable person would, they were asking, in ordinary language, whether the defendant had failed to take a precaution that was worth taking, and a precaution is worth taking when it costs less than the harm it is likely to prevent. Judges had been doing cost-benefit analysis for generations. Most of them would have denied it, and many still would. The vocabulary of reasonableness, prudence and ordinary care carried the calculation without announcing it. This book is about that hidden calculation and the body of thought that brought it into the open. Law and economics is the application of microeconomic reasoning to legal rules: the study of how rules change behavior by changing the costs and benefits people face, and of which rules produce the best results when judged by their effects. It began as a specialist interest of a few scholars at the University of Chicago in the late 1950s and early 1960s. Within a generation it had become the most influential intellectual movement in American legal scholarship, with its own journals, its own professorships, and several of its founders sitting on federal appellate courts. Today nearly every American law student meets it, usually in the first-year torts and contracts courses, and nearly every regulatory agency in Washington runs a version of it whenever it writes a major rule. The argument of this book The controlling claim of what follows is simple to state. Legal rules are prices. A rule that makes a factory pay for the smoke it sends into its neighbors' windows raises the price of smoke. A rule that excuses a driver who took ordinary care but still hit a pedestrian lowers the price of driving, relative to a rule that makes the driver pay regardless. People respond to prices. So the most important question to ask about a legal rule is often not whether it is fair to the parties in the courtroom, though that matters, but what it will make everyone else do before any accident, breach or dispute ever happens. From that single idea, three more follow, and they organize the chapters. The first is Ronald Coase's discovery that when people can bargain cheaply, the law's allocation of rights does not decide how resources are used, because the parties will trade rights to whoever values them most. The consequence Coase himself drew is the opposite of the one he is often credited with: because bargaining is usually costly, the law's allocation of rights matters enormously, and it should be made with an eye to what the parties would have agreed if they could. The second is that tort liability works as a system of deterrence, charging people for the harms they cause so that they have a reason to avoid them, and that the choice between negligence and strict liability is a choice about which margins of behavior the law will reach. The third is the one Hand's sentence reveals: that the standard of reasonable care, as courts have applied it for nearly two centuries, is a cost-benefit test, whether or not the judges and juries applying it think of it that way. The book also argues that this lens has limits, and that an honest account of law and economics has to take them as seriously as its successes. People do not always respond to incentives the way the models predict. Measuring costs and benefits often requires putting prices on things, including human lives, that resist pricing, and juries have shown that they punish defendants who do the arithmetic openly. Efficiency is not the only value a legal system serves, and a rule that maximizes total wealth can distribute it in ways a society finds intolerable. These objections do not defeat the economic approach. They define where it is strongest, which is in explaining why legal rules take the shape they do and predicting how people will respond to changes in them, and where it needs help from other ways of thinking. How the book proceeds Chapter 1 sets out the basic tools: what economists mean by efficiency, why they insist on looking at rules from before the event rather than after it, and how the idea of rational choice should and should not be understood. Chapter 2 is devoted to Coase and his 1960 article "The Problem of Social Cost," perhaps the most cited article in the history of legal scholarship, and to the old nuisance cases that illustrate it. Chapter 3 follows the framework that Guido Calabresi and Douglas Melamed built on Coase's foundation, distinguishing rules that protect rights with injunctions from those that protect them with damages, and asks what happens when real parties to real lawsuits get the chance to bargain. The middle of the book turns to accidents. Chapter 4 explains the economic theory of tort liability as deterrence, the distinction between the care people take and how much of a risky activity they engage in, and why that distinction explains when the law chooses strict liability over negligence. Chapter 5 is the heart of the argument about reasonable care: it traces the Hand formula backward into the nineteenth-century case law and forward into the modern Restatement, and it shows how courts that never mention algebra nonetheless weigh burdens against risks. Chapter 6 confronts the most uncomfortable implication of that weighing, which is that it requires a value for human life, and follows the question from the Ford Pinto litigation into the federal regulatory process, where agencies put an explicit number on a statistical life every time they write a major safety rule. The last two chapters widen the frame. Chapter 7 shows the same logic at work in contract remedies, criminal punishment, punitive damages and the decision to settle or go to trial. Chapter 8 takes up the criticisms: from behavioral economics, from those concerned with the distribution of wealth, and from philosophers who argue that tort law is about doing justice between two people rather than setting prices for everyone. The conclusion asks what a reader who has followed the argument should take from it. A word on scope. The book is chiefly about American law, with comparative notes drawn mostly from English cases, since the common law of negligence grew up on both sides of the Atlantic and some of its clearest cost-benefit reasoning is English. It is about the economics of private law, meaning torts, property and contracts, more than about antitrust, corporate law or taxation, where economic analysis is so thoroughly absorbed that it is simply the way the field is done. And it is written for readers who have not studied economics and do not need to. Nothing here requires more mathematics than the single inequality Learned Hand wrote in 1947. The book is educational and is not legal advice. The reader who finishes it should be able to look at almost any legal rule and ask the economist's question: who does this rule make pay, for what, and what will they do differently because of it? That question does not settle what the law should be. It is, however, almost always the right place to start. Chapter 1: Thinking Like an Economist About Law A lawyer and an economist look at the same case and see different things. The lawyer sees two parties, a set of past events, and a question about who owes what to whom. The judgment, when it comes, closes the matter. The economist sees the same parties, but mainly sees everyone who is not in the courtroom: all the people who will face similar choices in the future and who will make them knowing, more or less, how this case came out. For the economist the judgment does not close anything. It sets a price that will shape a thousand future decisions. That difference in perspective is the whole of law and economics in miniature. The rest is technique. This chapter sets out the core techniques, which are fewer and simpler than the field's reputation suggests: an idea of efficiency, a habit of looking at rules from before the event rather than after, and an assumption that people respond to incentives. Each of these has been misunderstood, by critics and by enthusiasts alike, and getting them right at the start saves much confusion later. The ex ante perspective Consider a simple dispute. A homeowner hires a contractor to build a deck. The contract says nothing about what happens if an unusually heavy storm damages the half-finished structure. A storm comes and does exactly that. Who bears the loss? Looked at after the storm, the question seems to be about fairness between these two people. Perhaps the contractor was not to blame for the weather; perhaps the homeowner is wealthier and better able to absorb the loss; perhaps the contractor had promised a finished deck and should deliver one. Each of these considerations has some pull, and a court deciding the case will hear arguments based on all of them. Looked at before the storm, the question changes. Whoever the law assigns the risk to will have a reason to guard against it and to price it. If the contractor bears the risk of storm damage to unfinished work, contractors will tarp their materials, schedule around forecasts, buy insurance, and charge slightly more. If homeowners bear it, homeowners will want to know about weather protection and may buy their own coverage. The rule that is best, from this vantage point, is the one that puts the risk on the party who can manage it most cheaply, because then the total cost of storms plus the cost of guarding against them will be as low as it can be. That party is probably the contractor, who builds decks all the time, knows how to protect a site, and can spread the cost of insurance across many jobs. The homeowner builds one deck in a lifetime. This is what economists mean by the ex ante perspective, from the Latin for "from before." It asks what a rule will do to behavior before the events it governs, rather than how it distributes losses after those events have occurred. The ex post perspective, "from after," is the one the parties in court naturally adopt, since the storm has already happened and the only question left is who pays. The ex ante perspective is not an economist's invention. Legal thinkers have always known that rules shape conduct; that is part of what makes them rules rather than one-off decisions. What law and economics added was persistence. It insists that the ex ante question be asked every time, and asked first, even in cases where the ex post equities seem to point clearly one way. It is the source of most of the field's surprising conclusions. A rule that seems harsh to the particular defendant may be the rule that prevents the most harm overall, and a rule that seems generous to a sympathetic plaintiff may raise costs for everyone in the plaintiff's position, including future plaintiffs. A standard illustration is the landlord who is forbidden, out of sympathy for poor tenants, from evicting tenants who fall behind on rent. Ex post, this helps the tenant in front of the court. Ex ante, landlords facing a higher risk of unpaid rent will raise rents, demand larger deposits, screen tenants more strictly, or take units off the rental market. Some of the costs of the protective rule fall on precisely the group it was meant to protect. Whether the rule is still worth having is a separate question, one that turns on how large each effect is. But nobody can answer it without first asking the ex ante question. What economists mean by efficiency The word "efficiency" does a great deal of work in law and economics, and it means something narrower than it does in ordinary speech. At least three distinct standards travel under the name, and much confusion in debates about the field comes from sliding between them. The strictest is Pareto efficiency, named for the Italian economist Vilfredo Pareto. A change is a Pareto improvement if it makes at least one person better off and nobody worse off. A situation is Pareto efficient if no further Pareto improvement is possible. The standard is attractive because it is hard to object to: if a change harms no one and helps someone, what reason could there be to refuse it? Voluntary trades are the paradigm, since a trade freely made presumably leaves both parties better off by their own lights. The trouble is that almost no legal rule passes the test. Every rule that decides a contested case, or allocates a disputed right, makes somebody worse off than they would have been under the alternative. If the law could adopt only Pareto improvements, it could scarcely decide anything. So in practice economists evaluating legal rules use a weaker standard, named for the British economists Nicholas Kaldor and John Hicks, who proposed versions of it in 1939. A change is a Kaldor-Hicks improvement if the gains to the winners are large enough that they could, in principle, compensate the losers and still come out ahead. The compensation need not actually be paid. The test asks only whether the total gains exceed the total losses. This is the standard behind cost-benefit analysis: a policy passes if its benefits, summed across everyone, exceed its costs, summed across everyone. When a law-and-economics scholar calls a rule efficient, Kaldor-Hicks efficiency is almost always what is meant. The third standard is the one Richard Posner, the most prolific figure in the field, proposed in the late 1970s as a normative foundation for the common law: wealth maximization. Wealth, in Posner's sense, is the total value of goods and services in a society, measured by what people are willing and able to pay for them. A rule is good if it increases that total. This is close to Kaldor-Hicks efficiency, with a specific commitment to measuring value by willingness to pay. The phrase "and able" is where the critics bite. A poor person may value a loaf of bread more intensely than a rich person values a second car, but the rich person can pay more, so the car counts for more in the wealth calculus. Posner acknowledged the difficulty and later retreated somewhat from treating wealth maximization as a complete moral theory, defending it instead as a pragmatic guide for judges deciding common-law cases. Behind all three sits the older idea of utilitarian welfare, the sum of human happiness or satisfaction, which economists generally regard as the ultimate goal but which they cannot measure directly. Willingness to pay is a proxy for it, and like any proxy it can be distorted, most obviously by differences in wealth. The differences among these standards are easiest to see side by side, and Table 1 sets them out. Table 1. Four standards for judging a legal rule. Standard Test Measures value by Main objection Pareto Someone gains, nobody loses Each person's own preferences Almost no legal rule can satisfy it Kaldor-Hicks Winners could compensate losers Sum of gains and losses Compensation is hypothetical, not paid Wealth maximization Total wealth rises Willingness and ability to pay Weights the preferences of the rich more heavily Utilitarian welfare Total well-being rises Satisfaction or happiness Cannot be observed or measured directly The practical consequence for this book is that when a court, a scholar or an agency says a rule is efficient, the right follow-up question is: efficient by which standard, and who bears the losses that the calculation nets out? Kaldor-Hicks efficiency is a claim that a pie has grown. It says nothing about how the pie is divided, and a reader who forgets this will mistake a claim about size for a claim about fairness. Rational choice, properly understood The third tool is the assumption that people are rational. This is the part of economics that non-economists find hardest to accept, and much of the resistance comes from a misreading of what the assumption claims. In economics, rational choice means only that people have preferences, that those preferences are reasonably consistent, and that people tend to choose the option they prefer among those available to them. It does not mean people are selfish; a person who prefers to give money to charity is behaving rationally when she does so. It does not mean people calculate; a driver who slows down on an icy road is responding to costs and benefits even if no arithmetic crosses her mind. And it does not mean everyone responds to every incentive. The claim is about tendencies in the aggregate: that when something becomes more costly, people on the whole will do less of it, and when it becomes cheaper, they will do more. Gary Becker, the Chicago economist who won the Nobel Prize in 1992 largely for extending this assumption to areas economists had thought beyond their reach, including crime, discrimination and family life, liked to say that the approach was a method rather than a description of any particular person's psychology. It predicts that raising the expected cost of a crime will reduce that crime somewhat, not that every potential criminal consults a mental table of penalties. On that modest version, the assumption is hard to deny. Speeding tickets reduce speeding. Deposits on bottles increase returns. Higher tort damages make firms more careful. The law itself presupposes as much, since a legal system that did not expect its sanctions to influence conduct would have little reason to impose them. The stronger version of the assumption, that people process information perfectly and never make systematic mistakes, is plainly false, and since the 1990s a large body of work in behavioral law and economics has documented how people depart from it: they overweight vivid risks, underweight probabilities they have not experienced, value things more once they own them, and judge past decisions by outcomes they could not have foreseen. Chapter 8 takes up this work in detail. For now the point is that the economic analysis of law does not depend on the strong version. It depends on the weak one, which is almost impossible to reject, and it becomes more accurate, not less, when the behavioral findings are added. Positive and normative analysis One more distinction organizes nearly everything in the field. Positive law and economics asks what legal rules do: how people respond to them and why the rules have the shape they have. Normative law and economics asks what the rules should be. The two are easily run together, and some of the field's worst arguments slide from one to the other without noticing. Positive analysis is where law and economics has been most successful. Its predictions about behavior are testable and have often been confirmed. Its explanations of legal doctrine have been illuminating, sometimes startlingly so. The most famous positive claim, advanced by Posner in the early 1970s, is that the common law, meaning the body of judge-made rules in property, contract and tort, tends to be efficient: that judges, without saying so and perhaps without knowing it, have developed rules that minimize the combined costs of accidents, disputes and precautions. Chapter 5 examines this claim as applied to the negligence standard, where it is strongest. If the claim is right, the economic approach explains a great deal of the law that judges produce, even if no judge ever took an economics course. Normative analysis is more contested, because it requires a premise that economics cannot supply: that efficiency is the goal, or one of the goals, that law should pursue. Most scholars in the field accept a qualified version of this premise. They argue that among rules that do not differ much in fairness, the more efficient one should be preferred, and that the legal system is a poor instrument for redistributing wealth compared with taxes and transfers. Others reject the premise, insisting that justice between the parties is the point of private law and that efficiency is at most a side effect. The disagreement is real and unresolved, and it runs through the chapters that follow. Where the movement came from It helps to know a little of the history. Economists had written about law before: Adam Smith lectured on jurisprudence, and Jeremy Bentham's utilitarian program was an attempt to reform English law according to calculated consequences. But the modern field dates from the University of Chicago Law School in the 1950s, where the economist Aaron Director taught antitrust to law students and persuaded a generation of them that much of what courts said about monopoly made no economic sense. Director founded The Journal of Law and Economics in 1958. Coase, who joined Chicago in 1964 and later edited the journal, had published the field's founding article there in 1960. At almost the same moment, and independently, a young Yale law professor named Guido Calabresi published an article in 1961 arguing that the rules governing accidents should be designed with an eye to reducing their costs, and in 1970 he expanded the argument into a book, The Costs of Accidents, that remains the most searching economic treatment of tort law. Calabresi was no Chicago conservative; he later became dean of Yale Law School and was appointed to the Second Circuit by President Clinton. His presence at the founding is a useful reminder that law and economics is a method, not a political program, though its most visible practitioners have often leaned toward markets and against regulation. Posner's Economic Analysis of Law, first published in 1973, turned these scattered insights into a comprehensive treatise covering nearly every field of law. It went through nine editions over the following four decades. Posner sat on the Seventh Circuit from 1981 until 2017, and his opinions, several of which appear in later chapters, are the most sustained effort by any American judge to decide cases in openly economic terms. Frank Easterbrook, another Chicago law professor, joined him on that court in 1985. Steven Shavell and A. Mitchell Polinsky, economists teaching in law schools, supplied much of the formal theory of accident law and remedies in the 1980s and 1990s. By the end of the century, economic analysis was so thoroughly embedded in legal scholarship that it no longer seemed like a movement at all. The idea at the core of all of it is the one this chapter began with: a legal rule is a price, and people respond to prices. The next chapter shows how far that idea could be pushed by the economist who took it most seriously, and how often he has been misread. Chapter 2: The Problem of Social Cost In the 1870s a confectioner named Bridgman ran a business on Wigmore Street in London. For more than twenty years he had used two large mortars, pounded by heavy pestles, in a kitchen at the back of his premises. Next door, on Wimpole Street, a physician named Sturges bought a house and, some years later, built a consulting room at the end of his garden, against the wall of Bridgman's kitchen. Once the room was built, Sturges discovered that the noise and vibration of the pestles made it nearly impossible to examine patients or listen to their chests. He sued to stop the pounding. In Sturges v. Bridgman (1879), the English Court of Appeal granted him an injunction. The confectioner, who had been there first, had to stop. The case was taught for decades as a lesson about the law of nuisance, and in particular about the doctrine that "coming to the nuisance" is no defense: the fact that Bridgman's noise predated Sturges's consulting room did not give him a right to continue. In 1960 an English-born economist then teaching at the University of Virginia used it to make a very different point, one that reorganized the way lawyers think about conflicting uses of property. His name was Ronald Coase, and the article was "The Problem of Social Cost." The reciprocal nature of harm The conventional way of thinking about a case like Sturges v. Bridgman treats it as a story with a perpetrator and a victim. The confectioner makes noise; the noise harms the doctor; the question is whether the confectioner should be made to stop or to pay. Economists before Coase thought about such situations in much the same way. The dominant analysis came from the Cambridge economist A. C. Pigou, who argued that when an activity imposes costs on third parties, costs its author does not bear, the activity will be carried on more than is socially desirable, and the government should correct this by taxing the activity, regulating it, or making its author liable for the harm. Coase's first move was to deny that the situation had a perpetrator and a victim in any sense that helped the analysis. The harm, he argued, was reciprocal. The noise interfered with the doctor's practice only because the doctor had chosen to put his consulting room against the confectioner's wall. If the confectioner were allowed to pound, the doctor would be harmed. If the doctor were allowed to stop the pounding, the confectioner would be harmed. Either way, one use of the land would be sacrificed to the other. The real question was not who had caused the harm but which activity should be allowed to harm which, and the answer depended on which use was more valuable. This was more than a debating point. It reframed the legal question entirely. The court was not deciding whether to punish a wrongdoer; it was deciding how a scarce resource, the right to use the space around the shared wall in a particular way, should be allocated between two competing claimants. That is an economic question, and Coase proposed to answer it economically. The theorem Suppose, Coase argued, that the confectioner and the doctor could bargain with each other freely and cheaply. Then the court's decision would not determine whether the pounding continued. It would determine only who paid whom. Take the case where the court finds for the doctor, as the actual court did. If the confectioner's pounding is worth more to him than quiet is worth to the doctor, the confectioner can offer to pay the doctor to waive the injunction. Say the pestles are worth £200 a year to Bridgman and quiet is worth £100 a year to Sturges. Any payment between £100 and £200 makes both better off, so a deal will be struck and the pounding will continue. If the values are reversed, the doctor keeps his injunction and the pounding stops. Now take the case where the court finds for the confectioner. If quiet is worth more to the doctor than pounding is worth to the confectioner, the doctor can pay the confectioner to stop, and he will. If pounding is worth more, no deal is possible and the pounding continues. In every case, the outcome is the same regardless of how the court rules: the land ends up in its most valuable use. If pounding is worth more than quiet, there will be pounding; if quiet is worth more, there will be quiet. The legal rule changes only the distribution of wealth, determining whether the doctor pays the confectioner or the confectioner pays the doctor, not the use of the resource. This proposition, which the economist George Stigler named the Coase Theorem a few years later, is usually stated like this: if transaction costs are zero, the initial allocation of legal rights does not affect the efficiency of the outcome, because parties will bargain to the efficient result. Coase illustrated it with a now-famous hypothetical about a cattle rancher whose straying herd damages a neighboring farmer's crops. Whether the rancher is liable for crop damage or not, if the two can bargain, the number of cattle and the amount of fencing and planting will end up the same. Only the direction of the payments will differ. What Coase actually meant The theorem is one of the most cited propositions in all of social science, and one of the most misunderstood. It is often read as a claim that the law does not matter, or that markets solve every problem of conflicting land use without government help, or that the Pigouvian tradition of taxing and regulating harmful activities is always a mistake. Coase spent much of the rest of his long life, he died in 2013 at the age of 102, insisting that these readings got him backwards. The world of zero transaction costs, Coase wrote in later reflections, was the world of economic theory, not the world we live in. He described it as a thought experiment designed to show what was missing from existing analysis. In the real world, transaction costs are almost never zero, and they are often prohibitive. Transaction costs, in the broad sense Coase meant, include everything it takes to reach and enforce an agreement: finding the other party, learning what each side values, negotiating, drafting, monitoring compliance, and going to court if the deal breaks down. For two neighbors sharing a wall, these costs may be modest. For a factory whose smoke drifts over ten thousand homes, they are enormous. No realistic process could gather ten thousand homeowners, discover how much each values clean air, and negotiate a single deal with the factory owner, particularly when each homeowner has an incentive to understate his valuation and let the others pay. So the real lesson of the theorem is the converse of its famous statement. Because transaction costs are positive, and often high, the initial allocation of rights frequently does determine how resources are used. Where bargaining is impossible, the resource stays wherever the law puts it. That makes the law's allocation of rights one of the most consequential economic decisions a society makes, and it means that courts and legislatures should allocate rights with an eye to efficiency: ideally, giving each right to the party who would have ended up with it had bargaining been costless. This is sometimes called the normative Coase theorem, or the principle of mimicking the market. Coase drew a second lesson, aimed at the Pigouvians. Government intervention also has costs. A regulation or tax designed to correct a harmful activity requires information the government may not have, may be captured by the interests it regulates, and may itself distort behavior. The question in every case is comparative: which institutional arrangement, among the imperfect ones actually available, produces the best result? Sometimes it will be liability rules, sometimes regulation, sometimes doing nothing and letting the parties sort it out. Coase's complaint against his predecessors was not that they favored government action but that they compared a flawed market to an idealized government, and so always found the market wanting. The old cases, read again Much of "The Problem of Social Cost" is devoted to English nuisance cases, and one of Coase's most provocative observations was that judges had often been reasoning in economic terms without saying so. In deciding whether a particular use of land was a nuisance, English courts routinely considered the character of the neighborhood, the utility of the defendant's activity, and the burden a prohibition would impose. The Sturges v. Bridgman court itself remarked that what would be a nuisance in one neighborhood might not be in another. These considerations, Coase argued, amounted to a rough assessment of which use of land was more valuable. The judges were allocating rights to their highest-valued uses, without the vocabulary to describe what they were doing. This observation became the seed of the positive economic theory of the common law, which Chapter 5 develops for the negligence standard. It also raised a question that Coase left open: in cases where bargaining was possible, did the parties actually bargain around the court's decision? If the doctor valued quiet at £100 and the confectioner valued pounding at £200, one would expect the confectioner to buy the doctor off after the injunction. Nobody at the time knew whether this happened. Chapter 3 describes what later researchers found when they went looking. American courts confronted the same problem in the twentieth century, and two cases have become standard illustrations of how the Coasean framework plays out when bargaining is costly. In Boomer v. Atlantic Cement Co. (1970), a group of landowners near Albany, New York sued a large cement plant whose dust, smoke and vibration damaged their properties. The plant had cost over $45 million to build and employed more than 300 people. The traditional New York rule was that a nuisance causing substantial damage should be enjoined, and the plaintiffs asked for an injunction shutting the plant. The New York Court of Appeals refused. It held instead that the plant could continue operating if it paid the landowners permanent damages compensating them for the lasting reduction in their property values, a sum the lower court had put at $185,000 in total. The dissenting judge objected that this amounted to letting the cement company buy a license to pollute. In a sense he was right: that is exactly what the majority did, and the economic justification is Coasean. An injunction would have given the landowners the right to shut down a plant worth vastly more than their damages, and with it the power to hold out for a payment far exceeding their actual losses. Given the number of owners and the stakes, a negotiated resolution was unlikely. The court in effect imposed the transaction the parties would have reached if they could have bargained: the plant would continue, and the owners would be paid. Spur Industries, Inc. v. Del E. Webb Development Co. (1972) produced an even more inventive solution. Spur operated a cattle feedlot in the Arizona desert west of Phoenix, in an area that had been agricultural for decades. Del Webb, the developer, bought land nearby and built Sun City, a large retirement community, expanding steadily toward the feedlot. By the late 1960s, residents in the southern part of Sun City were complaining of flies and odor from the feedlot's thousands of cattle and their manure. Webb sued to have the feedlot enjoined as a nuisance. The Arizona Supreme Court agreed that the feedlot was a nuisance to the residents and ordered it to move or close. But because Webb had brought the residents to the nuisance, knowingly building a city next to a long-established agricultural operation for its own profit, the court required Webb to pay Spur the reasonable cost of moving or shutting down. The Spur decision is often cited as the fourth box in a framework that Chapter 3 describes. For present purposes, what matters is how it answers Coase's question about which activity should harm which. The court concluded that the land's highest-valued use was now residential, so the feedlot had to go. But it recognized that the developer, not the feedlot, had created the conflict, and that making the feedlot bear the whole cost of relocation would reward developers for building next to existing operations and then suing them. By making the party who gained from the change pay for it, the court gave future developers a reason to account for the costs their projects would impose on their neighbors. Coase in the wider world The theorem's reach extends well beyond nuisance. It explains, for example, why property rights in the electromagnetic spectrum, which Coase had written about in 1959, can be auctioned and traded rather than allocated by a regulator: the auction and resale market move frequencies toward their most valuable uses, as the Federal Communications Commission has done since the 1990s. It underlies the design of cap-and-trade systems for pollution, including the sulfur dioxide trading program created by the Clean Air Act Amendments of 1990, which set a total limit on emissions, created tradable allowances, and let power plants that could cut emissions cheaply sell allowances to plants for which cuts were expensive. It explains why fisheries managed with individual transferable quotas tend to avoid the race to fish that depletes open-access stocks. In each case the insight is the same. Define rights clearly, make them tradable, and keep transaction costs low, and the parties will do much of the work of allocating resources efficiently. Where transaction costs cannot be kept low, the law has to do that work itself, by assigning rights, or liabilities, to the parties who would have ended up with them in a well-functioning market. The most important field test of the theorem, however, came not from any government program but from the hills of northern California. In the 1980s the legal scholar Robert Ellickson studied how ranchers and farmers in Shasta County dealt with the problem of cattle straying onto neighbors' land, the very hypothetical Coase had used. Parts of the county were "open range," where by law a rancher was not liable for damage done by cattle straying onto unfenced land; other parts were "closed range," where he was. Ellickson found that the legal rule made almost no difference to how residents behaved. Neighbors did not bargain in the shadow of the law; most of them did not know what the law was. They resolved trespass disputes through informal norms of neighborliness: a rancher whose cattle strayed was expected to retrieve them promptly and to make good any serious damage, whatever the formal rule said, and residents who violated these norms faced gossip, social pressure and occasional mild retaliation. His book reporting these findings, Order Without Law (1991), showed that the Coasean result could be reached without any bargaining at all, through social norms that tended to minimize the combined costs of the neighbors' interactions. Ellickson's findings cut in two directions. They confirmed that close-knit groups can reach efficient outcomes without law. They also suggested that the precise legal rule often matters less to such groups than lawyers assume. But Shasta County was a small, stable community where people dealt with each other over many years. Where parties are strangers, where there are many of them, or where they interact only once, as with a factory and its downwind neighbors or a driver and a pedestrian, norms of this kind are weak or absent. There the law must bear the full load, and the question of how it should allocate rights and liabilities becomes pressing. That is the question the next chapter takes up. Chapter 3: Property Rules, Liability Rules, and the Cathedral Coase showed that the law has to decide who holds a right. He said less about a second decision that turns out to be just as important: how the right, once assigned, will be protected. A landowner may have the right to be free from a neighbor's smoke. But does that mean she can get a court order stopping the smoke, or only that she can collect money for the damage it does? The difference between these two forms of protection is the difference between Boomer v. Atlantic Cement and the traditional rule it departed from, and it determines who has the power to set the price when rights collide. In 1972 Guido Calabresi and a young lawyer, A. Douglas Melamed, published an article in the Harvard Law Review that supplied the vocabulary lawyers still use for this choice. Its title was "Property Rules, Liability Rules, and Inalienability: One View of the Cathedral." The cathedral of the subtitle was a reference to Monet's paintings of Rouen Cathedral, each showing the same building in different light. The authors meant that their article offered one view of the law, not the only one. It has become among the most cited law review articles ever published. Three kinds of protection Calabresi and Melamed began from the premise that every legal system must first decide which of two conflicting parties is entitled to prevail: whether the factory is entitled to pollute or the neighbor entitled to clean air, whether the confectioner is entitled to pound or the doctor to quiet. They called these decisions entitlements. The second decision is how to protect the entitlement, and here they identified three possibilities. An entitlement is protected by a property rule when nobody can take it from its holder without the holder's consent. If the neighbor's entitlement to clean air is protected by a property rule, the factory must stop polluting unless it can persuade the neighbor to sell the right at a price the neighbor accepts. The remedy for violating a property rule is typically an injunction, a court order forbidding the conduct, backed by the court's contempt power. The holder sets the price. An entitlement is protected by a liability rule when others may take it without the holder's consent, provided they pay compensation in an amount fixed by a court. If the neighbor's entitlement is protected by a liability rule, the factory may continue polluting, but must pay damages equal to the harm. The remedy is money, and the court sets the price. An entitlement is inalienable when it cannot be transferred at all, even with the holder's consent. A person cannot sell herself into slavery, sell her vote, or, in the United States, sell a kidney. Inalienability rules forbid the market altogether, usually because the transaction would harm third parties or because society judges that the holder cannot be trusted to value the entitlement correctly. The insight of the framework is that the choice between property and liability rules can be made on economic grounds, and that transaction costs are again the key. Where the parties can bargain cheaply, a property rule is usually better. It forces anyone who wants the entitlement to negotiate for it, and so ensures that the entitlement moves only when the buyer values it more than the seller, as revealed by the seller's own willingness to part with it. A court does not need to know how much the holder values the right; the holder's refusal to sell below a certain price reveals it. Where bargaining is costly, whether because there are many parties, because holdouts can extract excessive payments, or because the harm occurs between strangers who cannot negotiate in advance, a liability rule is usually better. It allows the entitlement to move to its more valuable use without negotiation, at a price the court estimates. The clearest example of a liability rule in everyday life is the law of accidents. A pedestrian is entitled not to be run over. But no driver can negotiate in advance with every pedestrian she might strike, so the pedestrian's entitlement is protected only by damages: the driver who injures a pedestrian through carelessness must pay compensation. A property rule, requiring the driver to obtain each pedestrian's consent before exposing him to any risk, would make driving impossible. The whole of tort law, which the next three chapters examine, operates largely in the domain of liability rules for exactly this reason. The fourth rule Calabresi and Melamed then made an observation that gave the framework its lasting appeal. If there are two ways to assign an entitlement, to the polluter or to the neighbor, and two ways to protect it, by property rule or liability rule, then there are four possible combinations. Traditional nuisance law recognized three. Table 2 sets out all four as applied to a factory and its neighbors. Table 2. The four rules of the Calabresi and Melamed framework. Rule Entitlement held by Protected by What happens Example One Neighbor Property rule Factory is enjoined unless neighbor sells the right Traditional injunction; Sturges v. Bridgman Two Neighbor Liability rule Factory may continue but pays damages Boomer v. Atlantic Cement Three Factory Property rule Factory may continue unless neighbor buys it out No nuisance found; neighbor must pay to stop Four Factory Liability rule Neighbor may stop factory but must compensate it Spur Industries v. Del E. Webb Rules one through three were familiar. Rule one is the classic nuisance injunction. Rule two is the damages remedy of Boomer. Rule three is what happens when a court finds no nuisance at all: the factory is free to continue, and if the neighbors want it to stop, they must pay it to do so. Rule four, in which the neighbor may force the polluter to stop but must pay the polluter's costs of stopping, had no obvious precedent when Calabresi and Melamed wrote. They proposed it as a logical possibility. At almost the same moment, in the spring of 1972, the Arizona Supreme Court decided Spur Industries, ordering the feedlot to move but requiring the developer to pay for the move. The court did not cite the article and there is no reason to think it had seen it. The coincidence became part of the article's legend and a vivid demonstration that the framework captured something about how judges actually reason. Why would a court ever choose rule four? Because it combines the two things the court in Spur wanted: to move the land toward its more valuable use, which was residential, and to put the cost of the change on the party who stood to gain from it. More generally, rule four is useful whenever the party who wants an activity stopped is also the party best placed to judge whether stopping it is worth the cost. By making that party pay, the rule gives it a reason to demand the change only when the benefit exceeds the cost. Eminent domain works the same way: the government may take private land for public use, but it must pay just compensation, which forces it, at least in principle, to consider whether the project is worth what the land is worth. Choosing between the rules The Cathedral framework does not tell a court which rule to choose in any particular case, but it tells the court what to look at. Several considerations recur in the literature that grew from it. The first is the number of parties and the likelihood of holdouts. When one factory faces one neighbor, a property rule is workable; the two can negotiate. When one factory faces a thousand neighbors, a property rule protecting each of them gives every neighbor a veto, and each can hold out for a large share of the factory's value. The combined demands may exceed what the factory is worth even though the factory's value far exceeds the neighbors' total harm. A liability rule avoids this by letting the court set the price. Boomer is often defended on exactly this ground. The second is the court's ability to estimate value. A liability rule works only if the court can set damages roughly right. If damages are set too low, the entitlement will be taken even when its holder values it more than the taker; if too high, beneficial transfers will be deterred. For losses that are hard to measure, such as the subjective value of a family home, the peace of a neighborhood, or the aesthetic enjoyment of a view, courts may systematically undercompensate. Property rules avoid this problem by letting the holder name the price, which is one reason the law traditionally protects land with injunctions rather than damages. The third is the distributional consequence. The choice of rule affects who ends up with the wealth. Under rules one and two, the neighbor starts with the entitlement and is paid for any loss of it; under rules three and four, the factory does. Calabresi and Melamed emphasized that distributional goals were a legitimate basis for choosing among rules, and that a society might prefer to give entitlements to poorer parties even where efficiency was indifferent. A large later literature has argued about how these factors should be weighed. Louis Kaplow and Steven Shavell argued in 1996 that for harmful externalities such as pollution, liability rules are generally superior, because the court's estimate of harm is usually good enough and liability rules avoid the costs of bargaining failure. For disputes over the possession of things, such as who owns a car, they argued that property rules are generally superior, because the risk of misestimating value is greater and taking by force is a harm in itself. Others have defended property rules more broadly, pointing out that liability rules invite takings by anyone confident that a court will underestimate the holder's valuation. The debate continues, but its terms were set in 1972. Do parties bargain after judgment? The Coase theorem predicts that when the parties can bargain, they will bargain around a court's allocation of entitlements if the court has put them in the wrong place. The Cathedral framework relies on this prediction when it favors property rules in low-transaction-cost settings: the injunction is supposed to be only a starting point for negotiation. So it matters a great deal whether real parties actually do bargain. In 1999 Ward Farnsworth, then a young law professor, set out to find out. He identified about twenty nuisance cases litigated to judgment in which one side had obtained an injunction or successfully resisted one, and he interviewed the lawyers on both sides to learn what had happened afterwards. His results, published in the University of Chicago Law Review under the title "Do Parties to Nuisance Cases Bargain After Judgment? A Glimpse Inside the Cathedral," were striking. In none of the cases did the parties bargain after the judgment. The losing party did not offer to buy the entitlement from the winner, and the winner did not offer to sell it, even in cases where the stakes suggested a deal might have been possible. The lawyers explained why. By the time a nuisance case reached judgment, the parties had typically been fighting for years. They had come to dislike each other intensely. Many regarded the dispute as a matter of principle or honor rather than money, and would not have sold their rights at any price, or bought them from the other side at any price. Some felt that accepting money for giving up an entitlement the court had confirmed would be distasteful, a kind of betrayal of the victory. Litigation itself, Farnsworth suggested, may have transformed the parties' preferences, turning a dispute over the use of land into a contest in which the right itself had become a source of value independent of its market worth. Farnsworth's findings do not refute the Coase theorem, which is a claim about what would happen under zero transaction costs. They show that the transaction costs that matter in real disputes include not only the costs of finding and negotiating with the other party but also animosity, pride and the emotional residue of litigation. These costs may be highest precisely in the cases that reach judgment, since parties who could get along would probably have settled earlier. The practical upshot is that courts cannot count on the parties to correct a misallocation of entitlements. The initial allocation will often stick, which makes getting it right all the more important. The endowment effect A related complication comes from psychology. In a series of experiments reported in 1990, Daniel Kahneman, Jack Knetsch and Richard Thaler gave coffee mugs to half the students in a class and asked the owners how much they would accept to sell their mugs and the non-owners how much they would pay to buy one. If the mugs had a single value to each student, the two prices should have been roughly the same, and about half the mugs should have changed hands. In fact the owners demanded roughly twice or more what the buyers would pay, and far fewer trades occurred than predicted. Simply owning the mug, for a few minutes, made the students value it more. This endowment effect, which has been replicated in many settings though its size varies and its causes remain debated, has a direct bearing on the Coase theorem. If people value an entitlement more once it is theirs, then the initial allocation affects not only who is paid but also where the entitlement ends up, because the gap between selling price and buying price will block some trades that would otherwise occur. A neighbor who is awarded an injunction against a factory may demand more to give it up than she would have paid to obtain it had the court ruled the other way. The legal assignment of the right thus shapes the parties' preferences as well as their wealth. For the design of legal rules, the endowment effect adds a further reason to take the initial allocation seriously. It also offers a reason to be cautious about property rules where courts cannot be confident of their allocation, since mistakes will be harder for the parties to undo. Some scholars have argued that it favors liability rules in close cases, since a court-set price does not depend on the holder's inflated valuation. Others reply that the endowment effect may reflect genuine attachment that the law ought to respect, particularly for homes and other things bound up with a person's identity. From land to accidents The Cathedral framework was built around disputes over land, but its most important application lies elsewhere. In the vast majority of harmful interactions, the parties do not know each other in advance and could not negotiate even if they wanted to. A driver cannot bargain with every pedestrian; a manufacturer cannot negotiate with every future user of its product; a hospital cannot contract separately with every patient over every conceivable risk. In all these settings, the law's only realistic option is a liability rule, and the central question becomes how to set it: what harms should the injurer pay for, and under what conditions? That is the domain of tort law, and it is where the economic analysis of law has had its deepest and most lasting influence. Chapter 4: Accidents as an Economic Problem Every year in the United States tens of thousands of people die and millions are injured in accidents: on the roads, at work, in hospitals, in their homes, from products that fail. The law of torts is the body of rules that decides when the people who suffer these injuries can make someone else pay for them. To a lawyer, the central questions of that body of rules are about responsibility: who was at fault, whose conduct caused the harm, what the victim is owed. To an economist, the central question is different. Accidents are costly, and so is preventing them. What set of rules will make the sum of those costs as small as possible? That question was put most forcefully by Guido Calabresi in The Costs of Accidents, published in 1970. The book opened with a claim that shocked many of its first readers: that no society tries to eliminate accidents. Every society could eliminate most traffic deaths by banning cars, or by limiting speeds to ten miles an hour, and none does, because the cost in lost mobility would be too great. A society accepts a certain number of accidents as the price of activities it values. The real question is how many, and at what cost. Once that is admitted, accident law becomes an exercise in balancing the costs of accidents against the costs of avoiding them. The costs of accidents Calabresi divided the costs of accidents into three kinds. Primary costs are the losses from the accidents themselves, together with the costs of the precautions people take to avoid them: the injuries and property damage, plus the cost of brakes, guardrails and careful driving. Secondary costs arise from the way losses are distributed. A loss of a hundred thousand dollars concentrated on one family may ruin it; the same loss spread across a million people, each paying ten cents through insurance premiums or slightly higher prices, is barely felt. Because people generally dislike risk, spreading losses reduces their total burden. Tertiary costs are the costs of running the system that allocates the losses: lawyers, courts, insurance administrators and the time of everyone involved. A good accident law, on this account, minimizes the sum of all three. The difficulty is that they pull in different directions. Rules that best reduce primary costs, by making people bear the consequences of their own carelessness, may concentrate losses and raise secondary costs. Rules that best spread losses, such as universal social insurance that pays for every injury regardless of cause, may weaken the incentive to take care. Rules that finely calibrate liability to each person's fault may be expensive to administer. Every accident law is a compromise among these goals. The economic analysis of tort law that developed after Calabresi has concentrated mostly on the first category, primary costs, and within it on deterrence: on how liability rules affect the precautions people take. This is the domain where the economic approach has the most to say that ordinary legal thinking does not already know. Deterrence through liability The basic mechanism is the one described in Chapter 1. When a person causes harm to someone else and does not have to pay for it, the harm is invisible in her private calculation. She will take precautions only to the extent they benefit her, and she will take too few from society's point of view. Economists call the unpaid harm an externality. Tort liability internalizes it: by making the injurer pay for the harm she causes, the law puts that harm into her calculation, and she will then take precautions whenever they cost less than the harm they prevent. Consider a simple example. A company operates a warehouse beside a public sidewalk. Its forklifts occasionally knock boxes off the loading dock into the path of pedestrians. The company can install a barrier for $2,000 a year that would prevent these incidents. Without the barrier, the expected harm to pedestrians, meaning the likelihood of an injury multiplied by its cost, is $5,000 a year. From society's standpoint the barrier is clearly worth installing: it costs $2,000 and saves $5,000. But if the company does not have to pay for pedestrians' injuries, the barrier saves it nothing, and it will not install it. If it must pay, the barrier saves it $5,000 in expected liability, and it will. The example carries a hidden requirement that becomes important later. For liability to produce the right incentive, the company must expect to pay the full harm it causes, not more and not less. If damages are systematically too low, because injuries go unreported, or victims do not sue, or courts undervalue some kinds of loss, the company will take too little care. If they are too high, it may take too much, spending more on precautions than the harm they prevent, or it may abandon valuable activities entirely. The deterrence theory therefore gives a reason to care about the accuracy of damages, one that is independent of the victim's claim to be made whole. Negligence and strict liability American tort law uses two main liability rules for accidents. Under negligence, an injurer is liable only if she failed to exercise reasonable care. Under strict liability, she is liable for harm she causes whether or not she was careful. Most accidents are governed by negligence; strict liability applies to a limited set of activities, including abnormally dangerous activities like blasting and, in modified form, harm caused by defective products. A central early result of the economic theory, developed most fully by Steven Shavell in an article published in 1980, is that for a certain class of accidents, both rules produce efficient precautions. Consider accidents that only the injurer can prevent; the victim can do nothing to reduce the risk. Economists call these unilateral accidents. A pedestrian struck by a falling box on a public sidewalk is a fair example. Under strict liability, the injurer pays for every harm she causes. She therefore bears the full social cost of accidents and will choose the level of precaution that minimizes the sum of precaution costs and expected harm, since that sum is exactly what she pays. Under negligence, if the court sets the standard of care at the efficient level, the injurer will meet the standard, because meeting it costs her less than the liability she would face if she fell short. Once she meets it, she pays nothing for the accidents that still occur, but she has already taken the right amount of care. Either way, the injurer takes efficient precautions. Matters become more interesting when the victim can also take precautions. These are bilateral accidents, and they are the more common kind: a pedestrian can look both ways, a patient can follow medical advice, a product user can read the warnings. Under pure strict liability, the victim is compensated for every harm and so has no financial reason to take care. The injurer will take efficient care, but the victim will not. This is why strict liability is almost never applied in pure form. It is typically paired with a defense of contributory or comparative negligence, which reduces or bars recovery for victims who failed to take reasonable care, restoring their incentive. Under negligence, by contrast, the victim has a natural incentive to be careful. If the injurer meets the standard of care, the injurer is not liable, so the victim bears the loss from any accident that still happens. Knowing this, a rational victim will take care to avoid such losses. And the injurer, as before, will meet the standard. So negligence, even without any defense based on the victim's conduct, produces efficient care by both parties, provided the standard is set correctly. The economic account thus gives a neat explanation for the traditional structure of tort law: negligence as the default rule, strict liability with a contributory fault defense as the exception. Care versus activity level Shavell's analysis produced a second result, and it explains when the law should choose strict liability over negligence. It turns on the difference between how carefully a person does something and how much of it she does. A driver can reduce the risk she poses to pedestrians in two ways. She can drive more carefully, slowing down, watching the road, keeping her brakes in repair. This is her level of care. Or she can drive less, taking fewer trips, walking to the shops, living closer to work. This is her activity level. Both affect the total number of accidents. Every mile driven, however carefully, carries some residual risk. The negligence standard, as courts apply it, reaches care but not activity level. A court asks whether the driver was careful on the occasion of the accident. It does not ask whether the trip was worth taking at all, because judging the value of each trip would require information no court has. A driver who drives carefully is not negligent however many unnecessary trips she takes. Under negligence, then, the driver bears the costs of her carelessness but not the residual costs of her careful driving, which fall on victims. She will drive too much, because each additional mile costs her only the fuel and time, not the residual risk it imposes on others. Strict liability reaches both. A driver who pays for every accident she causes, careful or not, bears the full cost of each additional mile, including the residual risk. She will drive carefully, and she will also drive only when the trip is worth its full cost, including that risk. This result explains a pattern in tort doctrine that lawyers had long recognized without a satisfying explanation. Strict liability applies to activities, like blasting with dynamite, storing large quantities of explosives, keeping wild animals, and certain uses of hazardous chemicals, that remain dangerous even when conducted with all reasonable care. For these activities, the main margin of safety is often not how carefully they are done but how much, and where, they are done at all. A company blasting near a residential neighborhood can be as careful as possible and still endanger the people nearby. The efficient response may be to blast less, or somewhere else, and only strict liability gives the company a reason to consider that. The traditional English origin of the doctrine is Rylands v. Fletcher (1868), in which a mill owner built a reservoir on his land that burst into old mine shafts and flooded a neighbor's colliery. The House of Lords held the mill owner liable even though he had not been negligent, because he had brought onto his land something likely to do mischief if it escaped. American law developed the idea into the category of "abnormally dangerous activities," and the Restatement of Torts has for decades listed factors for identifying them, including whether the risk can be eliminated by reasonable care and whether the activity is appropriate to the place where it is carried on. The economic account reads these factors as proxies for the activity-level problem: strict liability is warranted when care alone cannot control the risk, so that the choice of how much and where becomes decisive. An opinion by Judge Richard Posner in Indiana Harbor Belt Railroad Co. v. American Cyanamid Co. (1990) applied this reasoning directly. A railroad car containing about 20,000 gallons of acrylonitrile, a toxic and flammable chemical, leaked in a switching yard near Chicago, and the railroad sought to hold the manufacturer strictly liable for the cleanup cost. Posner refused. The leak, he reasoned, was the kind of accident that ordinary care in maintaining and loading the car would prevent, so negligence would give adequate incentives. Nor was there any realistic way for the shipper to reduce its activity level or route the chemical around metropolitan Chicago, a hub through which rail traffic in the region had to pass. Strict liability would add nothing to deterrence and would impose liability without purpose. Table 3 draws together the main results of this part of the theory. Table 3. How liability rules affect injurers' and victims' incentives. Rule Injurer's care Injurer's activity level Victim's care Typical legal domain No liability Too little Too high Efficient Harms the law does not recognize Negligence Efficient Too high Efficient Most accidents Strict liability Efficient Efficient Too little Rarely used in pure form Strict liability with contributory fault defense Efficient Efficient Efficient Abnormally dangerous activities; products The table simplifies. It assumes courts set the standard of care correctly and assess damages accurately, and it leaves out the victim's own activity level, which negligence does not reach either. Shavell showed that no single rule can get both parties' activity levels right in a bilateral accident, because whichever party bears the residual losses will choose an efficient activity level while the other will not. The practical recommendation is to put residual losses on the party whose activity level matters more: on the blaster rather than the homeowner, on the driver rather than the pedestrian, perhaps on the product manufacturer rather than the consumer. Contributory and comparative negligence For most of the nineteenth and twentieth centuries, American negligence law included the rule of contributory negligence: a plaintiff who was herself negligent in any degree could recover nothing, however careless the defendant had been. The rule was harsh, and beginning in the late 1960s nearly every state replaced it with comparative negligence, under which a plaintiff's recovery is reduced in proportion to her share of fault. A handful of jurisdictions, including Alabama, Maryland, North Carolina, Virginia and the District of Columbia, retain contributory negligence in some form. From the standpoint of the simple deterrence model, the choice between these rules matters less than lawyers expected. If courts set standards correctly and parties respond rationally, both rules give each party a reason to meet its standard of care, and in equilibrium nobody is negligent, so the apportionment rule is never applied. The difference appears only when the model's assumptions fail: when courts make errors, when parties misjudge the standard, or when care cannot be precisely controlled. Under those conditions, comparative negligence tends to perform somewhat better, because it softens the all-or-nothing consequences of small errors. But the main case for comparative negligence has always been one of fairness rather than efficiency, and the economic analysis is notable mainly for having shown how little the choice affects deterrence in principle. Who is the cheapest cost avoider? Calabresi offered a more practical guide for assigning liability than the formal models, and it remains useful. Faced with an accident that either party might have prevented, he suggested, the law should ask which party could have avoided it most cheaply, and put the loss there. He called this party the cheapest cost avoider. The idea is not that the cheapest cost avoider is morally responsible, but that placing liability on it gives the strongest incentive to the party that can do the most good for the least expense. Where it is unclear who the cheapest cost avoider is, the loss should go to the party best placed to figure that out and to act on the answer, whether by taking precautions, by buying insurance, or by bargaining with the other side. The idea has proved especially influential in the law of products liability. Manufacturers know far more than consumers about the risks of their products, can test designs before selling them, and can spread the cost of injuries across all their customers through the price. For many product risks, the manufacturer is the cheapest cost avoider, and the move toward strict products liability in the 1960s, beginning with Greenman v. Yuba Power Products, Inc. (1963) in California, is often explained on these grounds. For risks that consumers can easily see and avoid, such as the sharpness of a knife, the cheapest cost avoider may be the consumer, and the law has generally declined to impose liability for them. The theory of deterrence, then, gives a coherent account of why tort law has the broad structure it does. It explains why negligence is the default, why strict liability applies where it does, and why the victim's own conduct matters. But it rests on one assumption that has not yet been examined: that courts can identify what reasonable care requires. Under a negligence rule, everything turns on where the standard is set. If it is set too low, injurers take too little care; if too high, too much. How do courts set it? The answer, which the next chapter develops, is that they do so by weighing costs against benefits, and that they have been doing so, in ordinary language, for as long as the negligence standard has existed. Chapter 5: The Hand Formula and the Hidden Calculus of Reasonable Care The standard of care in American negligence law is stated in words that sound nothing like economics. Pattern jury instructions, the model charges that trial judges read to juries in most states, define negligence as the failure to use reasonable or ordinary care, meaning the care a reasonably careful person would use under similar circumstances. They say nothing about costs or benefits, probabilities or burdens. Jurors are told to consider what a prudent person would have done, and trusted to know. Yet when judges explain why a particular defendant was or was not negligent, particularly in the appellate opinions that form the body of negligence law, they reach again and again for the same three considerations: how likely the harm was, how serious it would be, and what it would have taken to prevent it. The economic account of negligence holds that these three considerations are the content of reasonable care, and that judges and juries weighing them are performing a cost-benefit analysis whether or not they would describe it that way. This chapter examines the evidence for that claim, which is stronger than many lawyers expect, and the objections to it, which are more serious than many economists admit. Learned Hand and the barge The facts of United States v. Carroll Towing Co. were set out at the start of this book: a barge, a careless tug crew, a drifting vessel, and an absent bargee. The legal question on which the famous passage turned was whether the barge owner's failure to keep an attendant on board was itself negligent. If so, under the admiralty rule of the day, the owner's recovery against the tug company would be reduced. Hand began by observing that there was no general rule about when a barge owner must keep a bargee aboard. The cases were all over the place. The reason, he said, was that the owner's duty to protect against injuries from a barge breaking loose was a function of three variables: the probability that she would break away, the gravity of the resulting injury if she did, and the burden of adequate precautions. Then came the algebra. Calling the probability P, the injury L, and the burden B, liability depended on whether B was less than L multiplied by P. Applying the test, Hand noted that the probability of a barge breaking loose varied with the place and time. In New York Harbor in January 1944, during the war, the harbor was crowded and tugs were constantly shifting barges in and out of tiers; the chance that a moored barge would be set adrift by careless handling was significant. The bargee's absence, without any excuse the court found credible, during the working hours of daylight in such conditions, was negligent. The burden of keeping him aboard was slight compared with the risk his presence would have reduced. The barge owner's recovery was reduced accordingly. Hand had used the same approach, in words rather than letters, in earlier cases. In Conway v. O'Brien (1940), a guest-passenger case, he had written that the degree of care demanded of a person is the resultant of three factors: the likelihood that his conduct will injure others, the seriousness of the injury if it happens, and the interest which he must sacrifice to avoid the risk. And two years after Carroll Towing, in Moisan v. Loftus (1949), he acknowledged the method's chief limitation: of the factors, only the burden could ordinarily be measured, and the probability and gravity of harm could at best be estimated. The formula, he recognized, identified what mattered without supplying the numbers. The formula in the language of reasonableness The economic reading of the Hand formula is straightforward. B is the cost of the precaution the defendant failed to take. P times L is the expected harm the precaution would have prevented. If B is less than PL, the precaution was worth taking, and a person who failed to take it imposed on others a risk that cost more than avoiding it would have. That is negligence. If B exceeds PL, the precaution was not worth its cost, and failing to take it was reasonable. The formula thus defines negligence as a failure to take cost-justified precautions, which is exactly the standard the deterrence theory of Chapter 4 requires. A negligence rule with the standard set by the Hand formula gives injurers the incentive to take all precautions whose cost is less than the harm they prevent, and no others. Economists have added one refinement. The formula as Hand wrote it compares the total cost of a precaution with the total harm it prevents. The correct comparison, for choosing how much care to take, is at the margin: whether a little more precaution would prevent more expected harm than it costs. A driver who is already going at a moderate speed is not negligent merely because driving at a crawl would prevent some accidents; the question is whether going slightly slower would prevent enough additional harm to justify the additional delay. Courts in practice ask the marginal question, because the plaintiff must identify a specific precaution the defendant failed to take, and the question is whether that particular untaken precaution was worth its cost. The legal scholar Mark Grady made this point in an influential 1989 article on "untaken precautions," arguing that negligence litigation is structured as a comparison between what the defendant did and some specific additional step the plaintiff says the defendant should have taken. The formula before Hand The deeper question is whether the Hand formula was a new idea or a new description of an old one. Posner's answer, in "A Theory of Negligence," published in the first volume of the Journal of Legal Studies in 1972, was emphatic: the formula described what courts had been doing all along. Posner examined a large sample of American appellate negligence decisions from the late nineteenth and early twentieth centuries and argued that the outcomes were broadly consistent with the cost-benefit reading of reasonable care. The negligence standard, he concluded, was an economic standard whose purpose was to bring about the efficient level of accidents and safety. The historical evidence is suggestive, though it does not all point the same way. The classic common-law definition came from Baron Alderson in the English case Blyth v. Birmingham Waterworks Co. (1856), in which a water main froze and leaked during an unusually severe frost: negligence, he said, is the omission to do something which a reasonable man, guided upon those considerations which ordinarily regulate the conduct of human affairs, would do, or doing something which a prudent and reasonable man would not do. The water company was held not liable, because it had built its pipes to withstand ordinary frosts and could not reasonably have been expected to provide against a frost of such severity. The considerations that "ordinarily regulate the conduct of human affairs" are unspecified, but the result is easy to explain in Hand's terms: the probability of such a severe frost was low, and the burden of burying every main deeper was high. Henry Terry, an American legal scholar teaching in Japan, came closer to Hand's formulation in a 1915 article in the Harvard Law Review titled "Negligence." Terry argued that conduct is negligent when the risk it creates is unreasonably great, and that the reasonableness of a risk depends on the magnitude of the risk, the value of the interest exposed to it, and the value of the object the actor is pursuing and the likelihood of achieving it by the risky conduct. This is the Hand formula in all but notation, a generation before Carroll Towing. Benjamin Cardozo's opinion for the New York Court of Appeals in Adams v. Bullock (1919) shows the reasoning at work in a case. A trolley company ran its line under a bridge, using an overhead wire. A twelve-year-old boy crossing the bridge swung an eight-foot wire over the side; it touched the trolley wire, and he was badly burned. Was the company negligent for failing to guard against such an accident? Cardozo held that it was not. The wire was placed where no one could reach it without some extraordinary act. To guard against the risk, the company would have had to insulate its wires along the whole route or bury them underground, which would have made the trolley system impractical. Chance of harm, he found, had been small; the burden of precaution great. No reasonable person would have required the trolley company to bury its lines to prevent such an unusual accident. English courts produced two cases that are now taught side by side as the clearest judicial statements of the cost-benefit logic, though neither uses algebra. In Bolton v. Stone (1951), a woman standing outside her house was struck by a cricket ball hit out of a nearby ground. Balls had been hit out onto the road only a handful of times in some thirty years. The House of Lords held the cricket club not liable: the risk was so small that a reasonable person would have been justified in disregarding it. Sixteen years later, in Overseas Tankship (U.K.) Ltd. v. Miller Steamship Co., known as The Wagon Mound (No. 2) (1967), the Privy Council considered a ship whose crew had carelessly let furnace oil spill into Sydney Harbour, where it later caught fire and damaged other vessels. The chance that oil floating on water would ignite was small, and a lower court had relied on Bolton v. Stone to find no negligence. The Privy Council disagreed. Lord Reid explained that Bolton did not permit a reasonable man to ignore any small risk. It permitted him to ignore a small risk only when there was a valid reason for doing so, such as that it would involve considerable expense to eliminate it. In The Wagon Mound stopping the discharge of oil would have cost nothing; indeed the spilled oil was a loss to the ship. Where a risk is real, however small, and the precaution is costless or nearly so, failing to take it is negligent. The pair states the Hand formula almost perfectly. A low probability of harm excuses the failure to take a precaution when the precaution is costly, as in Bolton, but not when it is cheap, as in The Wagon Mound. The same probability can produce different results depending on the burden. The judges did not multiply anything. They weighed the same three factors Hand named, and they reached the results his formula would predict. Table 4 sets these decisions alongside one another. Table 4. Cost-benefit reasoning in leading negligence decisions. Case Court and year Probability of harm Gravity Burden of precaution Result Blyth v. Birmingham Waterworks English Court of Exchequer, 1856 Very low (extreme frost) Moderate High (redesign mains) Not negligent Adams v. Bullock New York Court of Appeals, 1919 Very low Serious Very high (insulate or bury lines) Not negligent United States v. Carroll Towing Second Circuit, 1947 Significant (crowded wartime harbor) Serious Low (bargee aboard) Negligent Bolton v. Stone House of Lords, 1951 Very low Serious Substantial Not negligent The Wagon Mound (No. 2) Privy Council, 1967 Low Serious Negligible (stop the spill) Negligent Custom and the T.J. Hooper One case shows the formula's reach especially clearly, because it concerns a situation in which reasonable people disagree about whose judgment should govern. In The T.J. Hooper (1932), two tugs towing coal barges along the Atlantic coast ran into a storm in March 1928, and the barges sank. Weather forecasts broadcast by radio had warned of the storm, and had the tugs carried radio receivers, they would probably have put into harbor in time. At the time, most tugs on the coast did not carry receivers. The owners argued that they could not be negligent for failing to carry equipment that the industry as a whole did not use. Learned Hand, writing for the Second Circuit, rejected the argument. Custom, he held, was evidence of what reasonable care required, but it was not conclusive. A whole calling might have unduly lagged in adopting new and available devices. Courts must in the end say what is required, and there are precautions so imperative that even their universal disregard will not excuse their omission. The receivers were cheap and the protection they offered was great. The economic reading of The T.J. Hooper is that the court applied the Hand formula over the objections of the industry. Custom is often good evidence of what precautions are cost-justified, because firms that face liability have reason to adopt them. But custom can lag, especially when the benefits of a precaution fall on others and the industry has not yet been made to bear them. A court that deferred entirely to custom would lock in whatever level of care happened to prevail. By insisting on its own assessment of costs and benefits, the court pushed the industry toward the efficient standard. The case remains good law, and its principle, that compliance with industry custom is evidence of due care but not a complete defense, is followed in most American jurisdictions, with the important exception of medical malpractice, where professional custom has traditionally set the standard of care, a deference that the economic account finds harder to justify. The Restatement and the modern law In 2010 the American Law Institute published the Restatement (Third) of Torts: Liability for Physical and Emotional Harm, the most recent authoritative synthesis of American negligence law. Its definition of negligence largely adopts the Hand formula in words. A person acts negligently, it says, if the person does not exercise reasonable care under all the circumstances, and the primary factors to consider in deciding whether conduct lacks reasonable care are the foreseeable likelihood that the conduct will result in harm, the foreseeable severity of any harm that may ensue, and the burden of precautions to eliminate or reduce the risk of harm. The reporters' commentary acknowledges that this is a balancing test and discusses the Hand formula explicitly. The Restatement (Third) of Torts: Products Liability, published in 1998, takes a similar approach to design defects. A product is defectively designed, under its section 2(b), when the foreseeable risks of harm could have been reduced or avoided by adopting a reasonable alternative design, and the omission of that design makes the product not reasonably safe. The test is a risk-utility comparison, weighing the added cost and any loss of usefulness of the safer design against the reduction in risk. Some courts had already adopted a version of it; the California Supreme Court in Barker v. Lull Engineering Co. (1978) held that a product's design could be found defective if the benefits of the challenged design did not outweigh its risk of danger, among other grounds. Although some states continue to apply a test based on what an ordinary consumer would expect, risk-utility balancing has become the dominant approach in American design defect law. Judge Posner applied the Hand formula openly in a number of Seventh Circuit opinions. In McCarty v. Pheasant Run, Inc. (1987), a hotel guest was assaulted by an intruder who entered through a sliding door that was closed but not locked. She sued the hotel, arguing that it should have taken additional security measures. Posner's opinion walked through the Hand formula, noting that the plaintiff had offered no evidence of what additional precautions would have cost or how much they would have reduced the risk, and that she herself could have locked the door at essentially no cost. The jury's verdict for the hotel was affirmed. The opinion is a model of the formula's use as a way of organizing evidence, and also of its demands: it requires the plaintiff to identify a specific untaken precaution and show that it was worth taking. Unknowingly, or knowingly? If the cost-benefit reading of negligence is right, why do the jury instructions not say so? Why do most judges, deciding most negligence cases, speak the language of reasonableness rather than of costs and probabilities? Part of the answer is that the language of reasonableness is not a disguise for cost-benefit analysis but a compression of it. Ordinary people do not need to multiply anything to know that a homeowner who leaves a rake lying tines-up on a dark path is careless and one who fails to install floodlights across a rural property is not. Their intuitions about reasonable conduct already incorporate a rough sense of risks and burdens, formed by long experience of the costs of accidents and precautions. When a jury decides that a defendant failed to act reasonably, it is drawing on those intuitions, and the formula merely makes explicit what the intuitions contain. On this view, judges and juries apply cost-benefit analysis unknowingly in the sense that a person who catches a ball applies the laws of motion: correctly, without formal calculation, and without being able to state the equations. Part of the answer, though, is that the cost-benefit reading may not be the whole truth. Several scholars have questioned how closely the actual case law tracks the Hand formula. Richard Wright, in an article titled "Hand, Posner, and the Myth of the 'Hand Formula'" (2003), argued that Hand himself did not treat the formula as a strict aggregate cost-benefit test, that courts rarely apply it as such, and that many negligence decisions reflect judgments about the rights of the parties, such as the principle that one may not impose significant risks on others merely to save oneself trivial costs, that the formula cannot capture. Stephen Gilles, while more sympathetic to the economic reading, has pointed out that the formula plays a limited role in jury instructions and has explored why courts have generally preferred the reasonable-person standard. The Restatement's own drafters, for their part, framed the balancing factors as primary rather than exclusive and acknowledged that other considerations enter the analysis. There are also cases the formula has difficulty explaining. In Helling v. Carey (1974), the Washington Supreme Court held two ophthalmologists negligent as a matter of law for failing to give a routine glaucoma pressure test to a patient under forty, even though the profession's standard at the time did not call for the test in patients of that age. The court reasoned that the test was simple, inexpensive and harmless, and the harm it could prevent, blindness, was grave. That is a Hand-formula argument. But critics pointed out that the incidence of glaucoma in patients under forty was very low, perhaps one in 25,000, so that the cost of testing all such patients to find each case might be high relative to the harm prevented. The case shows both the formula's pull and the danger of applying it by intuition without the numbers. The most reasonable conclusion from this debate is that the Hand formula is a powerful and largely accurate account of the core of negligence law, the part that concerns how much care a person must take in activities that expose others to risk, and a less complete account of the whole. Judges weigh probabilities, harms and burdens in nearly every contested negligence case, and they have done so for as long as negligence has been a cause of action. Whether they call it cost-benefit analysis matters less than the fact that they do it. The difficulty arises when the weighing must be done explicitly, with numbers, and one of the numbers is the value of a human life. That is where the next chapter begins. Chapter 6: Juries, Hindsight, and the Price of a Life The Hand formula asks a court to compare the burden of a precaution with the expected harm it would prevent. When the harm is a damaged barge or a lost cargo of flour, the comparison is uncomfortable only in its uncertainty. When the harm is a death, the comparison seems to require something many people find repugnant: a price on a human life. If a safety device costs $50 per car and would prevent one death for every million cars, then installing it across a million cars costs $50 million per life saved. Whether the device is required depends on whether a life is worth more or less than $50 million. Nobody who designs cars, drugs, buildings or regulations can avoid this comparison. Every decision to stop short of the maximum possible safety implicitly accepts some risk of death in exchange for lower cost or greater usefulness. The only choice is whether to make the comparison openly, with an explicit number, or implicitly, without one. This chapter follows the question into two settings that have answered it in opposite ways. Juries, when they discover that a company has made the comparison explicitly, have often punished it. Federal agencies, by contrast, are required to make it explicitly, and publish their numbers. The Pinto The story most often told about corporate cost-benefit analysis concerns the Ford Pinto, a subcompact car Ford sold from the 1971 model year through 1980. The Pinto's fuel tank was located behind the rear axle, and in rear-end collisions it could be punctured, leading to fires. In 1972 a Pinto stalled on a California freeway and was struck from behind; the car burst into flames. The driver, Lily Gray, died. Her passenger, a thirteen-year-old boy named Richard Grimshaw, was terribly burned and survived after many operations. At trial, the plaintiffs presented evidence that Ford engineers had known of the fuel tank's vulnerability and that inexpensive modifications could have reduced the risk, but that the company had gone ahead with the design. The jury awarded Grimshaw $125 million in punitive damages, at the time an extraordinary sum. The trial judge reduced it to $3.5 million, and the California Court of Appeal affirmed the reduced award in Grimshaw v. Ford Motor Co. (1981), holding that the evidence supported a finding that Ford had acted with conscious disregard of public safety. The Pinto became a byword for corporate callousness largely because of a separate document. A 1973 internal Ford memorandum, which became public through journalism and litigation in the late 1970s, estimated the costs and benefits of a proposed federal safety standard for fuel leakage in rollover accidents. It calculated that modifications costing $11 per vehicle across 12.5 million cars and light trucks, about $137 million in total, would prevent some 180 burn deaths, 180 serious burn injuries and 2,100 burned vehicles each year, benefits it valued at about $49.5 million. For the value of a death it used $200,000, a figure the National Highway Traffic Safety Administration had itself published. On these numbers the modifications failed a cost-benefit test. The legal scholar Gary Schwartz, in a careful 1991 article titled "The Myth of the Ford Pinto Case," showed that much of the popular account was wrong. The memorandum concerned rollover accidents, not the rear-end collisions at issue in Grimshaw, applied to Ford's whole fleet rather than to the Pinto specifically, and was prepared for submission to the regulator, not as a secret internal decision about whether to let customers burn. It played no real part in the evidence on which the Grimshaw jury decided. The number of Pinto fire deaths, Schwartz concluded, was far smaller than figures circulating in the press had claimed. None of this made the Pinto a safe car, and the jury's finding that Ford had behaved badly in the case before it was supported by other evidence. But the lesson the public drew, that a company which weighs lives against dollars is a company that should be punished, came from a document that did exactly what the Hand formula requires. A similar episode followed two decades later. In Anderson v. General Motors, a Los Angeles jury in 1999 considered a case in which a Chevrolet Malibu was struck from behind and its fuel tank caught fire, severely burning six occupants. The plaintiffs introduced a 1973 memorandum by a General Motors engineer that estimated, on the basis of assumptions about the frequency of fatal fuel-fed fires, that such deaths cost the company about $2.40 per vehicle in potential liability, and that compared this with the cost of design changes. The jury returned a verdict of about $4.9 billion, nearly all of it punitive damages. The trial judge reduced the punitive award to roughly $1 billion. The memorandum's arithmetic, again, was a crude version of the calculation negligence law asks every designer to make. What juries do with cost-benefit analysis The economist W. Kip Viscusi, who has spent much of his career estimating the value people place on reducing risks to life, set out to test how jurors respond to corporate risk analysis. In experiments reported in the early 2000s, including in articles titled "Corporate Risk Analysis: A Reckless Act?" (2000) and "Jurors, Judges, and the Mistreatment of Risk by the Courts" (2001), he presented several hundred jury-eligible adults with hypothetical cases in which a company had decided not to adopt a safety improvement, and varied whether the company had conducted a formal cost-benefit analysis before deciding. The results were uncomfortable for anyone who believes the tort system encourages sound safety decisions. Mock jurors were more likely to award punitive damages when the company had done a risk analysis than when it had not. Worse, when the company had used a higher value for human life in its analysis, which should have made its decision look more protective of safety, jurors tended to award more, apparently because the higher number anchored their sense of an appropriate penalty. A company that had done no analysis at all, and so had no idea whether its decision was reasonable, fared better than one that had thought carefully and written down its reasoning. Viscusi's conclusion was that the tort system, as juries actually operate it, penalizes the very behavior the Hand formula asks for. A firm that explicitly weighs costs against risks leaves a paper trail that can be read to a jury as evidence of callousness. The rational response is to stop writing things down, or to stop doing the analysis, or to channel it through lawyers so that it is privileged from discovery. None of these responses improves safety. Two psychological tendencies help explain the jurors' behavior. One is the sense, which most people share, that it is wrong to trade lives for money, and that anyone who does so deliberately has crossed a moral line, even though everyone makes such trades implicitly all the time. Philip Tetlock has called these "taboo trade-offs" and documented the moral outrage people feel when they learn that someone has made one explicitly. The other is hindsight bias: the tendency, once an outcome is known, to overestimate how foreseeable it was. In a well-known 1995 study, Kim Kamin and Jeffrey Rachlinski asked people to decide whether a city should have hired a bridge operator to guard against flood damage. Those told only the facts available beforehand mostly concluded the precaution was not needed; roughly a quarter thought it was. Those told that a flood had occurred and caused damage concluded, in substantially larger numbers, more than half, that failing to take the precaution had been negligent, even when they were instructed to avoid hindsight. Juries in accident cases always know the outcome. The probability term in the Hand formula is, for them, very hard to estimate as it would have looked in advance. The value of a statistical life If juries resist explicit valuations of life, the federal government has embraced them. Every major regulation issued by an executive agency in the United States must be accompanied, under a series of executive orders dating back to President Reagan's Executive Order 12291 in 1981 and continued in President Clinton's Executive Order 12866 in 1993, by an analysis of its costs and benefits. For health and safety rules, the principal benefit is usually lives saved, and the agencies must put a dollar value on them. The number they use is called the value of a statistical life, and it is widely misunderstood. It is not the value of any particular person's life, and it is not a claim about what a life is worth in some ultimate sense. It is an estimate of how much people are willing to pay for small reductions in the risk of death. Suppose that, on average, people would pay about $1,300 a year to reduce their annual risk of dying by one in ten thousand, whether through a safer car, a less dangerous job or a home smoke detector. Then a group of ten thousand people would collectively pay about $13 million to eliminate one expected death among them. That $13 million is the value of a statistical life: the amount a population is willing to spend to prevent one death that will occur to someone unknown among its members. Economists estimate these numbers mostly from labor markets, by measuring how much more workers are paid for jobs with higher risk of fatal injury, holding other features of the job constant. They also use studies of what consumers pay for safety features and, less reliably, surveys asking people directly about their willingness to pay. The estimates vary, but federal agencies have converged on figures in the range of roughly $10 million to $13 million in recent years, adjusted periodically for inflation and income growth. Table 5 gives some of the figures agencies have used. Table 5. Values of a statistical life used by federal agencies (dollars of the year indicated). Agency Year of guidance Value of a statistical life Federal Emergency Management Agency 2020 $7.5 million Environmental Protection Agency 2010 $9.1 million Department of Transportation 2016 $9.6 million Department of Transportation 2022 $12.5 million Department of Transportation 2023 $13.2 million Source: agency guidance documents as compiled in published summaries of U.S. government practice; agencies update their figures periodically. The figures differ across agencies partly because of the dates of their guidance and partly because of methodological choices, but their order of magnitude is stable, and they are sixty or more times larger than the $200,000 figure the government itself was using in the early 1970s, when the Pinto memorandum was written. Much of that increase reflects inflation and rising incomes; a substantial part reflects better studies of what people actually pay for safety. It is worth noticing that a regulator using today's numbers would have reached a very different result on the fuel-leakage standard the Pinto memo examined. The value of a statistical life is a direct answer to the question the Hand formula poses when the harm is death. A precaution that costs less than the value of a statistical life per expected life saved is, on this approach, worth taking. The tort system does not use the number in that way, because tort damages for death are measured differently. A wrongful death action typically compensates survivors for lost financial support, services and companionship, not the value the deceased placed on her own life, and when the plaintiff's experts have offered testimony about the "hedonic" value of life, meaning the value of the pleasure of living, derived from willingness-to-pay studies, many courts have excluded it; the Seventh Circuit did so in Mercado v. Ahmed (1992). Tort damages for death are therefore often lower than the value of a statistical life, which by the logic of Chapter 4 means that tort liability alone may underdeter conduct that risks death. That gap is one of the reasons regulatory safety standards exist alongside tort law. Cost-benefit analysis and the courts The rules governing regulatory cost-benefit analysis have been an area of active change, and they bear on the book's argument because they show the same tension between explicit and implicit weighing that appears in negligence law. The details are set largely by the Office of Management and Budget in a document called Circular A-4, first issued in 2003. Among its most consequential choices is the discount rate, which determines how much weight future benefits receive compared with present costs. The 2003 version directed agencies to discount future effects at both 3 percent and 7 percent. In November 2023, under President Biden, OMB issued a revised Circular A-4 that lowered the default rate to about 2 percent, a change that would give greater weight to benefits, such as climate and health effects, that arrive far in the future. On January 31, 2025, President Trump signed an executive order titled "Unleashing Prosperity Through Deregulation," which, among other things, revoked the 2023 revision and reinstated the 2003 version of Circular A-4. The same order required agencies to identify at least ten existing regulations for elimination for each new one issued and directed that the total incremental cost of new regulations be significantly less than zero. The practical effect of these requirements continues to depend on how agencies and OMB apply them. Courts, meanwhile, have had to decide when statutes require agencies to weigh costs against benefits, and here too the direction has been toward more explicit weighing. In Whitman v. American Trucking Associations (2001), the Supreme Court held that the Clean Air Act forbids the Environmental Protection Agency from considering costs when setting national ambient air quality standards, because the statute directs that the standards be set at the level requisite to protect public health. In Entergy Corp. v. Riverkeeper, Inc. (2009), it held that the Clean Water Act permitted, though did not require, cost-benefit analysis in setting standards for power plant cooling water intake structures. And in Michigan v. EPA (2015), it held that the EPA had acted unreasonably when it decided to regulate power plants' emissions of mercury and other hazardous air pollutants without considering cost at the threshold stage, because the statute directed it to regulate if doing so was "appropriate and necessary," and it is hardly appropriate, Justice Scalia wrote for the majority, to impose billions of dollars in economic costs in return for a few dollars in health benefits. The federal courts of appeals have sometimes demanded rigorous cost-benefit analysis where statutes call for agencies to consider economic effects. In Business Roundtable v. SEC (2011), the D.C. Circuit struck down a Securities and Exchange Commission rule giving shareholders easier access to corporate proxy materials, finding that the agency had inadequately assessed the rule's economic consequences. The Supreme Court's decision in Loper Bright Enterprises v. Raimondo (2024), overruling the Chevron doctrine under which courts had deferred to reasonable agency interpretations of ambiguous statutes, adds a further dimension. After Loper Bright, courts decide for themselves, without deference, what a statute means, including whether and how it requires an agency to consider costs. How much this changes the practice of regulatory cost-benefit analysis will depend on how lower courts read particular statutes, and that process is still unfolding. Two cultures of weighing Put side by side, the tort system and the regulatory system present a paradox. Both require someone to weigh the cost of a precaution against the risk to life it would reduce. The regulator is required to do it openly, with a published number, and is criticized when it does not. The private company that does it openly risks punishment by a jury, and the rational company learns to do it quietly or not at all. The paradox is less puzzling than it looks. The regulator's analysis is prospective: it evaluates a rule before anyone is hurt, when the lives at stake are statistical and nobody knows whose they will be. The jury's judgment is retrospective: it evaluates a decision after a particular person has died or been maimed, when the victim has a name and a face and sits, sometimes, in the courtroom. The same trade-off that seems prudent when made about unknown future victims seems monstrous when made, in effect, about a known one. Hindsight bias magnifies the effect by making the harm seem to have been more foreseeable than it was. The economic analysis of law cannot dissolve this tension, but it can clarify what is at stake in it. If the tort system punishes explicit risk analysis, it gives firms an incentive to make safety decisions without thinking carefully about them, which is not a recipe for safety. The appropriate response is not to abandon the moral intuition that lives are not merely commodities, but to find ways of keeping it from distorting the incentives the law creates, for instance by instructing juries about the role of cost-benefit analysis in design decisions, by relying more heavily on regulatory standards set prospectively, or by tying punitive damages to conduct that reflects genuine indifference to risk rather than to the mere existence of an analysis. The next chapter takes up punitive damages, along with several other areas where the economic logic of incentives shapes doctrine outside the law of accidents. Chapter 7: The Same Logic Elsewhere: Contracts, Crime, Punishment and Litigation The economic analysis of accidents rests on a small number of ideas: that legal rules set prices, that people respond to prices, and that the right price for a harmful act is the harm it causes. These ideas are not special to torts. They apply, with adjustments, wherever the law attaches consequences to conduct. This chapter follows them into four areas where they have reshaped how lawyers and judges think: the remedies for breach of contract, the design of criminal punishment, punitive damages in civil cases, and the decision whether to settle a lawsuit or take it to trial. In each, the economic approach explains features of the law that are otherwise puzzling, and in each it has provoked resistance that illuminates its limits. Why contract law does not punish breach A person who breaks a promise has, by ordinary moral standards, done something wrong. Yet American contract law does not punish breach. The standard remedy is expectation damages: the breaching party must pay the other the amount needed to put her in the position she would have been in had the contract been performed. No more. Courts do not award punitive damages for breach of contract except in rare cases where the breach is also an independent tort, and they will not enforce contract clauses that set damages at a level designed to penalize rather than to estimate the loss. Specific performance, an order requiring the breaching party actually to perform, is available only in limited circumstances, typically for land and unique goods. The economic account explains this pattern with the idea of efficient breach, developed in the early 1970s by Robert Birmingham and popularized by Richard Posner. Suppose a manufacturer agrees to sell a specialized machine to a buyer for $100,000; the machine is worth $120,000 to the buyer. Before delivery, a second buyer appears who values the machine at $150,000 and offers to pay that much. If the manufacturer breaches the first contract and sells to the second buyer, paying the first buyer expectation damages of $20,000, the first buyer is exactly as well off as if the contract had been performed, the manufacturer is better off, and the machine has gone to the person who values it most. Everyone is at least as well off, and some are better off: a Pareto improvement. A rule that punished breach, or that required specific performance, would prevent this outcome, or would require the parties to negotiate their way to it at some expense. On this view, expectation damages are the contract-law analogue of the Hand formula. They price breach at exactly the cost it imposes on the promisee, so that the promisor will breach only when breaching is worth more to her than that cost, which is to say only when breach is efficient. The theory has been sharply criticized. Daniel Friedmann, in "The Efficient Breach Fallacy" (1989), argued that in the example above the first buyer already has a right to the machine, and that the efficient course is for the manufacturer to perform and for the second buyer to negotiate with the first; the transaction costs of that negotiation are no higher than those of the litigation that breach invites. Seana Shiffrin argued in 2007 that a contract law that treats breach as a permissible option, so long as damages are paid, diverges from the morality of promising in a way that may corrode the moral character of those who live under it. Others have pointed out that expectation damages in practice undercompensate: they exclude the promisee's attorney's fees, which under the American rule each party bears, and they exclude losses that are hard to prove. If damages systematically fall short, the promisor will breach more often than is efficient. A famous Oklahoma case shows both the appeal and the difficulty of the efficiency approach. In Peevyhouse v. Garland Coal & Mining Co. (1962), a coal company leased a family's farm for strip mining and promised to restore the land when it finished. It did not. Restoration would have cost about $29,000, but the restored land would have been worth only about $300 more than the unrestored land. The Oklahoma Supreme Court limited the family's recovery to the $300 diminution in value. Economically, the result made sense: spending $29,000 to produce $300 of value would waste resources. But the Peevyhouses had specifically bargained for restoration, perhaps because they valued the land for reasons the market did not reflect, and the result let the coal company escape a promise it had made in exchange for a lower price. The case is still taught as the sharpest illustration of the conflict between efficiency and keeping one's word. Default rules and Hadley v. Baxendale A second contribution of the economic approach to contract law concerns what happens when a contract is silent. Every contract leaves gaps, because parties cannot foresee every contingency and it would be prohibitively expensive to try. The law fills the gaps with default rules, which apply unless the parties agree otherwise. The economic question is how to choose them. The usual answer is that default rules should be the terms most parties would have chosen had they addressed the question, since this minimizes the number of contracts that must be drafted around them. Ian Ayres and Robert Gertner, in a 1989 article titled "Filling Gaps in Incomplete Contracts," added a twist. Sometimes, they argued, the law should choose a default that most parties would not want, precisely to force the party with private information to reveal it by contracting around the default. They called these penalty defaults. Their leading example was the English case Hadley v. Baxendale (1854). The crankshaft of a mill in Gloucester broke, and the mill owners sent it by a carrier to be used as a pattern for a new one. The carrier delayed delivery, and the mill stood idle for several extra days. The owners sued for their lost profits. The Court of Exchequer held that a breaching party is liable only for losses that arise naturally from the breach or that both parties could reasonably have contemplated at the time of contracting. The carrier had not been told that the mill would stand idle without the shaft, and mills often kept spares, so lost profits were not recoverable. Ayres and Gertner read the rule as a penalty default. The mill owners knew the stakes of delay; the carrier did not. A default rule making carriers liable for all consequential losses would require carriers to charge every customer for the risk of unusually large losses, so that ordinary customers would subsidize those with unusual needs. The Hadley rule instead puts the burden on the party with the unusual need to disclose it, allowing the carrier to take extra care or charge extra for that shipment. The rule thus induces the exchange of information that makes efficient precautions possible, and it places the burden of disclosure on the party who can supply the information most cheaply. Crime and punishment Gary Becker's 1968 article "Crime and Punishment: An Economic Approach" began, according to his own account, with a practical problem: he was late for an oral examination at Columbia and had to decide whether to park illegally close to the building or legally farther away. He weighed the probability of a ticket and its cost against the inconvenience of the legal space, and parked illegally. It occurred to him that criminals might make similar calculations, and that the standard view of crime as the product of psychological abnormality or social pathology might be incomplete. Becker's model treats a potential offender as comparing the benefits of an offense with its expected cost, which is the probability of being caught and punished multiplied by the severity of the punishment. The implication for public policy is that the same level of deterrence can be achieved through different combinations of probability and severity. A society can deter by catching most offenders and punishing them moderately, or by catching few and punishing them severely. Since catching offenders requires police, prosecutors and courts, which are expensive, while fines cost almost nothing to impose, Becker argued that the efficient approach for many offenses was a low probability of detection combined with high fines, set so that the expected penalty equaled the harm the offense caused. The model has several important refinements. George Stigler pointed out in 1970 the principle of marginal deterrence: if every offense carries the maximum penalty, an offender who has committed a minor crime has no reason not to commit a more serious one. Penalties must therefore rise with the gravity of the offense, so that offenders have a reason to stop short. This logic underlies graduated sentencing and explains, for instance, why making kidnapping a capital offense might make kidnappers more likely to kill their victims. The limits of fines also matter: most offenders cannot pay large fines, so imprisonment, which is costly to the state and harmful in ways beyond the offender himself, becomes necessary for serious crimes. The empirical literature on deterrence has found that the probability of punishment generally matters more than its severity. Increases in policing and in the certainty of apprehension have measurable effects on crime rates in many studies; increases in sentence length, particularly for offenders already facing long sentences, have much weaker and less consistent effects. This is consistent with the behavioral findings discussed in Chapter 8: people discount distant consequences heavily and tend to underweight low-probability events, so that a high but uncertain penalty several years in the future may carry little weight at the moment of decision. Becker's framework survives these findings, but in a modified form: offenders respond to incentives, but the incentives that matter most are the immediate and certain ones. Punitive damages as a multiplier Chapter 6 showed that juries sometimes award punitive damages in ways that penalize careful risk analysis. The economic approach offers an account of what punitive damages should do, and it is quite different from the usual legal account. Courts generally describe punitive damages as serving to punish and deter defendants who have acted with malice, fraud or reckless disregard for the rights of others. A. Mitchell Polinsky and Steven Shavell, in a 1998 article in the Harvard Law Review titled "Punitive Damages: An Economic Analysis," argued that the main economic function of punitive damages is to correct for the fact that not every injurer is held liable. If a firm causes harm of $1 million in each of ten incidents but is sued and held liable in only one of them, its expected liability per incident is $100,000, a tenth of the harm it causes. It will take too little care. To give it the right incentive, the damages in the one case in which it is held liable should be multiplied by the inverse of the probability of liability: here, by ten. Total damages should thus equal the harm divided by the probability that the injurer will be found liable. On this view, punitive damages should depend on the chance that the defendant would escape liability, not on how reprehensible its conduct was. A firm that concealed its misconduct, making detection unlikely, should pay a large multiple; a firm whose harms are always detected and sued upon should pay no punitive damages at all, however badly it behaved, because compensatory damages already give it the right incentive. Judge Posner, writing for the Seventh Circuit in Kemezy v. Peters (1996), a civil rights case involving a police officer who had beaten the plaintiff, listed several functions that punitive damages serve, including this one of compensating for the probability of escaping detection, along with deterring conduct that has no social value and preventing the defendant from simply buying the right to injure others. The Supreme Court has approached punitive damages through the Due Process Clause rather than economics. In BMW of North America, Inc. v. Gore (1996), it held that a $2 million punitive award against BMW for failing to disclose that a new car had been repainted before sale was grossly excessive, identifying three guideposts: the reprehensibility of the conduct, the ratio of punitive to compensatory damages, and the difference between the punitive award and civil penalties for comparable misconduct. In State Farm Mutual Automobile Insurance Co. v. Campbell (2003), it added that few awards exceeding a single-digit ratio between punitive and compensatory damages would satisfy due process. And in Exxon Shipping Co. v. Baker (2008), exercising its authority over federal maritime law rather than constitutional law, it capped punitive damages arising from the 1989 Exxon Valdez oil spill at a one-to-one ratio with compensatory damages. The Polinsky-Shavell framework fits uneasily with this body of law. A fixed ratio cap does not track the probability of escaping liability, which is what the economic theory says should matter. And the Court's emphasis on reprehensibility focuses on the defendant's culpability, a retributive consideration. The economic view does not deny that retribution has a place, but it insists that if punitive damages are to deter efficiently, they must be calibrated to the risk that harms will go unremedied. Settle or sue? The last application concerns litigation itself. The vast majority of civil lawsuits in the United States settle rather than going to trial. Economists explain this by modeling litigation as a bargaining problem. Each side forms an estimate of the likely outcome at trial and of what trial would cost. If the plaintiff's estimate of what she could win, less her trial costs, is below the defendant's estimate of what he would lose, plus his trial costs, there is a range of settlements that both prefer to trial, and a rational pair of litigants will settle within it. Trial costs themselves create a zone of agreement even when the parties agree completely about the likely outcome. Cases go to trial chiefly when the parties are mutually optimistic, each thinking it is more likely to win than the other thinks. George Priest and Benjamin Klein drew a striking implication from this model in "The Selection of Disputes for Litigation" (1984). If the cases that go to trial are those in which the parties disagree about the outcome, then they will tend to be close cases, where the outcome is genuinely uncertain. Clear cases settle, because both sides can see how they will come out. So among cases that reach trial, plaintiffs should win about half the time, whatever the underlying legal standard, provided the stakes are roughly symmetric. The prediction has been tested extensively. Plaintiff win rates vary more than the simple hypothesis predicts, because stakes are often asymmetric, because some parties care about precedent or reputation, and because information is not evenly shared; but the tendency toward the middle is often visible, and the selection effect has become an essential caution in interpreting trial statistics. A rise or fall in plaintiff win rates tells us little about the law, since the mix of cases that reach trial changes in response. The selection model also gave rise to one of the most interesting claims in the whole field: that the common law tends to evolve toward efficiency even without judges trying to make it efficient. Paul Rubin and George Priest argued independently in 1977 that inefficient rules impose larger costs on the parties who are subject to them, so those parties have more reason to litigate cases challenging them. Inefficient rules will therefore be relitigated more often than efficient ones, and each round of litigation gives courts another chance to change them. Over time, efficient rules would survive and inefficient ones would be worn away, much as natural selection weeds out less fit organisms. The argument was ingenious and remains debated. Its critics point out that it depends on parties' litigation decisions being driven by the social costs of rules rather than private stakes, that repeat players such as insurers and large firms may have their own reasons to seek rules that favor them whether or not those rules are efficient, and that judges are not simply passive filters. Gillian Hadfield and others showed that under plausible assumptions the process could as easily entrench inefficient rules. Still, the idea shows how far the incentive logic can reach: even the development of legal doctrine itself can be modeled as the product of rational responses to costs and benefits. In each of these four settings, the economic approach offers a single unifying question: what behavior does this rule reward, and what does it cost? The answers are not always the ones lawyers expect, and they are not always the ones the law adopts. But they explain much of the law's structure, and where the law departs from them the departure usually signals the presence of some other value, such as retribution, promise-keeping or the dignity of a person's word, that economics does not capture well. The next chapter takes those other values, and the most serious criticisms of the economic approach, seriously. Chapter 8: The Limits of the Lens A method that explains as much as law and economics does invites the suspicion that it explains too much. If every rule can be described as a price, and every outcome as a response to incentives, then perhaps the approach is less a discovery about law than a vocabulary that can be draped over anything. Critics have made this charge since the 1970s, and they have made several more specific ones: that people do not behave as the models assume, that efficiency ignores the question of who gets what, and that private law is about justice between particular people rather than about managing the behavior of the population at large. Each criticism has force. Each has also changed the economic approach, rather than refuting it. This chapter takes them in turn and asks what survives. Behavioral law and economics The rational choice assumption, as Chapter 1 explained, comes in a weak and a strong form. The weak form holds that people tend to do less of things that become more costly. The strong form holds that people process information perfectly, form accurate beliefs about probabilities, and make consistent choices across time and context. From the 1970s onward, psychologists led by Daniel Kahneman and Amos Tversky documented a long list of systematic departures from the strong form. In 1998 Christine Jolls, Cass Sunstein and Richard Thaler published "A Behavioral Approach to Law and Economics" in the Stanford Law Review, a manifesto for incorporating these findings into the economic analysis of law. They organized the departures under three headings: bounded rationality, bounded willpower and bounded self-interest. Bounded rationality covers the ways people's judgments depart from the ideal of accurate estimation. People rely on rules of thumb that generally serve them well but produce predictable errors. They judge the probability of an event by how easily examples come to mind, so they overestimate vivid, recent or well-publicized risks, such as plane crashes and terrorist attacks, and underestimate dull, common ones, such as falls at home. They are overoptimistic about their own abilities; most drivers believe they are above average. They evaluate outcomes relative to a reference point rather than in absolute terms, feeling losses more sharply than equivalent gains. And, as Chapter 6 showed, they overestimate in hindsight how predictable an outcome was. Bounded willpower refers to the difficulty people have acting on their own long-term interests. They procrastinate, undersave and overconsume, and they know it, which is why they value commitment devices such as automatic retirement savings. Bounded self-interest captures the finding that people care about fairness and will sacrifice their own material interests to punish those who treat them unfairly. In the "ultimatum game," one player proposes how to split a sum of money and the other can accept the split or reject it, in which case neither gets anything. A purely self-interested responder would accept any positive offer. In experiments, responders routinely reject offers they consider unfair, typically those below a fifth or so of the total, and proposers, anticipating this, typically offer something close to half. Each of these findings has implications for the analysis in earlier chapters. Bounded rationality suggests that tort liability may deter less than the models predict, because potential injurers who are overoptimistic about their own care, or who underweight low-probability risks, will not respond fully to expected liability. It suggests that the Hand formula, which asks a jury to estimate the probability of harm as it would have appeared beforehand, is systematically distorted by hindsight bias, making defendants look more negligent than they were. The endowment effect, discussed in Chapter 3, undercuts the Coasean assumption that bargaining will move entitlements to their highest-valued users regardless of the initial allocation. Bounded self-interest helps explain the Farnsworth findings, in which litigants refused to bargain with adversaries they had come to resent. These findings have also generated a distinctive set of policy recommendations. The best known, associated with Thaler and Sunstein's book Nudge (2008), is that the law can improve outcomes by designing the context in which people make choices, for instance by making beneficial options the default, while leaving people free to choose otherwise. Automatic enrollment in retirement savings plans, adopted by many employers and encouraged by the Pension Protection Act of 2006, is the most prominent success: participation rates rise sharply when employees must opt out rather than opt in. Behavioral law and economics has its own critics. Some argue that the laboratory findings do not reliably carry over to markets, where experienced participants and competitive pressure may correct individual errors. Some worry that the approach licenses paternalism, allowing officials, who are subject to the same biases as everyone else, to override people's choices on the ground that they know better. And a number of well-known findings, including some on priming and on the size of the endowment effect, have proved harder to replicate than their initial publication suggested. The fair assessment is that behavioral economics has refined rather than replaced the rational choice framework. Its most robust findings, such as loss aversion, hindsight bias and the power of defaults, are well established and matter for legal design. They show that the incentive effects of legal rules are real but imperfect, and that the design of rules must take account of how people actually perceive and respond to them. The distribution problem The second criticism concerns distribution. Kaldor-Hicks efficiency, the standard the field uses, asks whether the gains to winners exceed the losses to losers. It does not ask who the winners and losers are. A rule that transfers $100 from a poor person to a rich one, and creates $101 of value in the process, is efficient. Willingness to pay, the measure of value behind these calculations, is also sensitive to wealth: the rich are willing to pay more for almost everything, so their preferences count for more. Critics argue that a normative theory built on these foundations will systematically favor the interests of the wealthy. Ronald Dworkin pressed a more fundamental objection in a 1980 article titled "Is Wealth a Value?" He argued that social wealth, as distinct from the well-being of the people who hold it, has no value in itself, so that a society which increased its total wealth by taking from those who valued something less intensely and giving to those who could pay more would not, by that fact alone, have become better in any sense worth caring about. Posner's later retreat from wealth maximization as a complete moral theory was in part a response to arguments of this kind. The most influential response from within the field came from Louis Kaplow and Steven Shavell in "Why the Legal System Is Less Efficient than the Income Tax in Redistributing Income" (1994). They argued that if a society wants to redistribute wealth, it should do so through the tax and transfer system, not by distorting legal rules. A legal rule that favors poorer parties, for instance by making landlords liable for more of their tenants' losses than efficiency would require, redistributes crudely, since not every tenant is poor or every landlord rich, and it distorts behavior in the domain the rule governs as well as imposing the same work disincentives that any redistribution carries. An income tax that raises the same amount and transfers it to the poor avoids the first distortion. Therefore, they concluded, legal rules should be chosen for efficiency, and distribution should be left to taxes. The argument is powerful and has been widely influential, but it depends on an assumption that critics regard as unrealistic: that the tax system will in fact make the redistribution that legal rules forgo. If political constraints prevent the tax system from redistributing, then refusing to consider distribution in legal rules may simply mean that less redistribution occurs. Other scholars, including Chris Sanchirico, have pointed out that in some circumstances legal rules can redistribute more efficiently than taxes, for instance when a rule's effects are closely correlated with income. And some have argued that the distribution of legal entitlements matters for reasons beyond income, such as dignity, security or the ability to participate in social life, that no tax transfer can fully replace. The distributional critique also bears on the value of a statistical life. Because willingness to pay rises with income, estimates of the value of a statistical life are higher for wealthier people and populations. Federal agencies use a single national value rather than adjusting for the income of the population a rule protects, a choice that departs from strict willingness-to-pay logic precisely because the alternative, valuing the lives of poorer people less, is widely regarded as unacceptable. That departure is a small, telling acknowledgment that the efficiency framework needs moral limits imposed from outside. Corrective justice The third criticism is the deepest, because it challenges the economic approach's understanding of what private law is. The economic approach treats a tort case as an occasion for setting a price that will influence the behavior of everyone who might someday be in the defendant's position. The plaintiff is, in effect, a private attorney general, rewarded with damages for bringing a case that serves the public purpose of deterrence. But this seems to miss something obvious about the structure of a lawsuit. The plaintiff recovers from the defendant, not from a general fund; the defendant pays the plaintiff, not the state. Why should that be, if the point is deterrence? A fine paid to the government would deter as well, and compensation could come from social insurance. Philosophers of private law, including Ernest Weinrib, Jules Coleman, and later John Goldberg and Benjamin Zipursky, answer that the bilateral structure of the lawsuit reveals its purpose. Private law, they argue, is about corrective justice: when one person wrongfully injures another, the injurer has an obligation to repair the loss, and the lawsuit is the mechanism by which that obligation is enforced. Goldberg and Zipursky call their version "civil recourse," emphasizing that the law empowers the victim to seek redress from the particular person who wronged her. On any of these views, the defendant pays because he did wrong to this plaintiff, not because making him pay will induce others to behave better. Deterrence may be a welcome side effect, but it is not the point, and a theory that makes it the point misunderstands the practice it claims to explain. Weinrib, in The Idea of Private Law (1995), pressed the argument hard: economic analysis, he contended, cannot explain why damages flow from this defendant to this plaintiff, because it treats both as instruments for achieving a social goal, and the bilateral link between them is incidental. Coleman took a more conciliatory position, accepting that economic considerations play a role in setting the content of the standard of care while insisting that the structure of liability reflects corrective justice. The debate has produced some convergence. Many corrective-justice theorists accept that the standard of care has an economic dimension: what counts as wrongful conduct depends in part on the costs of precaution, and the Hand formula is a reasonable account of how to decide when a risk is unreasonable. Many economists accept that the bilateral structure of tort law is a constraint on what the system can do and that the sense of justice it embodies helps explain its legitimacy. The two views are perhaps best seen as answering different questions. Economics asks what rules will produce the best consequences; corrective justice asks what we owe one another. A satisfactory account of tort law needs both, and the negligence standard, which asks whether a defendant imposed risks on others that a reasonable person would not have imposed, sits exactly where they meet. The measurement problem A fourth limit is practical rather than philosophical. The economic analysis of law requires estimates of costs and benefits, and many of them are very hard to make. Learned Hand acknowledged in Moisan v. Loftus that only the burden of precaution could usually be measured; the probability and severity of harm could only be estimated. Courts do not have the information needed to compute the efficient standard of care in most cases, and when they try to apply the Hand formula explicitly they depend on the evidence the parties choose to present. Regulators, who have more resources, still face deep uncertainty about how much a regulation will reduce risks, what compliance will cost, and how to value benefits such as ecological preservation that are not traded in markets. The measurement problem does not defeat the economic approach, but it tempers its ambitions. The economic analysis of law is most reliable when it identifies the direction of an effect: that strict liability will reduce an activity more than negligence, that higher damages will induce more care, that a property rule will lead to holdouts when there are many parties. It is less reliable when it claims to identify the precise level at which a standard should be set or the exact amount that damages should be. The reasonable person standard, applied by juries drawing on common experience, may in practice estimate costs and benefits as well as any explicit calculation, and it has the advantage of drawing on the community's sense of what is fair. The formula clarifies what the standard is about; it rarely makes the decision easier. What survives After these criticisms, what remains of the economic approach? A great deal. The idea that legal rules shape conduct by changing its costs is not seriously disputed. The Coasean insight that the law must allocate rights with an eye to transaction costs has become part of how every lawyer thinks about property and regulation. The theory of deterrence explains the broad architecture of tort law with an economy that no rival theory matches. And the Hand formula, whatever its limits as a complete account of negligence, identifies the considerations that courts actually weigh when they decide whether conduct was reasonable. What the criticisms establish is that the economic approach is a lens, not a complete theory of law. It brings certain features into sharp focus and leaves others blurred. It sees incentives clearly and justice between particular persons dimly. It measures what can be measured and struggles with what cannot. It is at its best when it is used alongside other ways of understanding law, correcting their blind spots and being corrected by them in turn. Conclusion: Reading the Law as a Set of Prices Learned Hand's barge sank more than eighty years ago, and the question it raised has not gone away. Every day, courts decide whether someone took reasonable care, and every day, whatever words they use, they weigh the likelihood and severity of harm against the burden of avoiding it. What the preceding chapters have tried to show is that this weighing is not an accident of one judge's taste for algebra but the working core of a legal system that, for generations, has been setting prices on risky conduct, and that seeing those prices clearly changes how one ought to think about law. Three conclusions follow that could not have been stated at the outset. The first is that the most important effects of a legal rule are usually felt by people who never go to court. The tug crew and the barge owner in Carroll Towing were settling accounts over a single accident. The rule the case announced reached every barge owner in the harbor, and in time every person deciding whether a precaution was worth its cost. This is not a reason to ignore the parties in front of the court; the corrective-justice theorists are right that private law is, among other things, a practice of holding particular people to account for particular wrongs. But a judge, a legislator or a citizen who evaluates a rule only by how it treats the litigants will reliably misjudge it. The first question to ask of any rule is what it will make people do before anything goes wrong. Many rules that look generous to victims turn out to raise the costs of the activities those victims depend on; many that look harsh turn out to prevent more harm than they cause. The only way to know which is which is to ask. The second is that the allocation of rights matters most precisely where the market cannot correct it. Coase is remembered for a theorem about a world without transaction costs, but his lasting lesson concerns the world we actually inhabit, in which bargaining is costly, parties are numerous, and, as Farnsworth's litigants showed, people who have fought each other in court will often refuse to deal even when a deal would benefit both. In that world, the law's initial assignment of entitlements tends to stick. That makes decisions such as those in Boomer and Spur Industries far more consequential than they appear, and it gives courts and legislatures a responsibility to allocate rights, and to choose between injunctions and damages, with an eye to where those rights would have ended up if bargaining had been possible. The Calabresi and Melamed framework is not merely a taxonomy. It is a checklist for asking whether the law has put the power to set the price in the right hands. The third is the one this book has been building toward. The standard of reasonable care is a cost-benefit standard, applied by judges and juries who mostly do not describe it that way, and the fact that they do not describe it that way is both a strength and a vulnerability. It is a strength because the language of reasonableness lets ordinary people apply the standard using intuitions formed by long experience, without having to produce numbers they do not have, and because it keeps the standard anchored to a community's sense of what people owe one another. It is a vulnerability because when the weighing becomes explicit, as it does whenever an engineer writes down the cost of a safety feature and the lives it might save, juries have shown that they will punish the very analysis the standard requires. The Pinto and the Malibu cases, and the experiments that followed them, point to a system at odds with itself: asking firms to weigh costs and risks while penalizing those caught doing so. What remains unsettled Several important questions remain open, and the reader who takes the economic approach seriously should keep them in view. It is unsettled how far regulatory cost-benefit analysis will change in the coming years. The reinstatement in 2025 of the 2003 version of Circular A-4, with its higher discount rates, alters how future benefits are weighed against present costs, and the requirement that agencies offset new regulations by eliminating old ones introduces a constraint that does not come from cost-benefit logic at all. After Loper Bright, courts will decide for themselves whether particular statutes require agencies to weigh costs, without deferring to the agencies' readings. The explicit, prospective weighing of lives against dollars that the regulatory state has practiced for four decades is thus in flux, and its direction will be set by courts and administrations over the next several years. It is unsettled how the behavioral findings should be integrated into the design of liability rules. If hindsight bias systematically inflates juries' assessments of the probability of harm, then the negligence standard as applied is stricter than the negligence standard as stated, and defendants face more liability than the Hand formula would justify. Some scholars have proposed jury instructions designed to counteract the bias, or greater reliance on regulatory compliance as evidence of due care; neither has been shown to work reliably. And it is unsettled, perhaps permanently, how efficiency should be weighed against other values. The economic approach has made a strong case that legal rules are poor tools for redistributing wealth, and that the costs of ignoring incentives are real and often borne by the people rules are meant to protect. It has not made, and probably cannot make, the case that efficiency is the only thing law should care about. Peevyhouse remains a hard case not because the economics is unclear but because the economics is clear and still leaves a sense that something owed has gone unpaid. What the reader should do differently For the reader who is not a lawyer, the practical upshot of this book is a habit of mind. When a new rule is proposed, whether a cap on damages, a change in a safety regulation, a new form of liability for online platforms or a shift in the burden of proof, ask three questions. Who does this rule make pay, and for what? What will they do differently because of it, and will the change be worth what it costs? And who will bear the losses that the aggregate calculation nets out? The first two are the economist's questions. The third is the question economics alone cannot answer, and the one a democratic society must answer for itself. For the reader who is a lawyer or a judge, the upshot is sharper. The language of reasonableness is not a mystery. When a court asks whether a defendant acted reasonably, it can be helped by asking which precaution the plaintiff says should have been taken, what that precaution would have cost, and how much harm it would likely have prevented, judged as the matter appeared before the accident rather than after it. That is the Hand formula, and it is also simply a disciplined way of thinking about what reasonable care means. It will not decide every case. It will make the reasons for deciding a case visible, and in a legal system that depends on the public's confidence that its judgments are not arbitrary, making reasons visible is no small thing. The law has been setting prices on conduct for as long as it has existed. Law and economics did not invent that practice; it gave it a name, a method and a set of questions. The questions are the lasting contribution. They do not replace judgment about what is fair, but they make it harder to mistake a good intention for a good rule. Notes Introduction 1. United States v. Carroll Towing Co., 159 F.2d 169 (2d Cir. 1947). Chapter 1 1. Nicholas Kaldor, "Welfare Propositions of Economics and Interpersonal Comparisons of Utility," Economic Journal 49 (1939): 549; John R. Hicks, "The Foundations of Welfare Economics," Economic Journal 49 (1939): 696. 1. Richard A. Posner, "Utilitarianism, Economics, and Legal Theory," Journal of Legal Studies 8 (1979): 103; Richard A. Posner, The Economics of Justice (Harvard University Press, 1981). 2. Richard A. Posner, Economic Analysis of Law (Little, Brown, 1973; 9th ed., Wolters Kluwer, 2014). 3. Guido Calabresi, "Some Thoughts on Risk Distribution and the Law of Torts," Yale Law Journal 70 (1961): 499. 4. Gary S. Becker, "The Economic Way of Looking at Behavior," Nobel lecture, Journal of Political Economy 101 (1993): 385. Chapter 2 1. Sturges v. Bridgman (1879) 11 Ch. D. 852 (Court of Appeal). 1. R. H. Coase, "The Problem of Social Cost," Journal of Law and Economics 3 (1960): 1. 2. A. C. Pigou, The Economics of Welfare (Macmillan, 1920). 3. R. H. Coase, "The Federal Communications Commission," Journal of Law and Economics 2 (1959): 1; R. H. Coase, The Firm, the Market, and the Law (University of Chicago Press, 1988), which contains his later reflections on the theorem. 4. Boomer v. Atlantic Cement Co., 26 N.Y.2d 219 (1970). 5. Spur Industries, Inc. v. Del E. Webb Development Co., 108 Ariz. 178, 494 P.2d 700 (1972). 6. Robert C. Ellickson, Order Without Law: How Neighbors Settle Disputes (Harvard University Press, 1991). Chapter 3 1. Guido Calabresi and A. Douglas Melamed, "Property Rules, Liability Rules, and Inalienability: One View of the Cathedral," Harvard Law Review 85 (1972): 1089. 1. Louis Kaplow and Steven Shavell, "Property Rules Versus Liability Rules: An Economic Analysis," Harvard Law Review 109 (1996): 713. 2. Ward Farnsworth, "Do Parties to Nuisance Cases Bargain After Judgment? A Glimpse Inside the Cathedral," University of Chicago Law Review 66 (1999): 373. 3. Daniel Kahneman, Jack L. Knetsch, and Richard H. Thaler, "Experimental Tests of the Endowment Effect and the Coase Theorem," Journal of Political Economy 98 (1990): 1325. Chapter 4 1. Guido Calabresi, The Costs of Accidents: A Legal and Economic Analysis (Yale University Press, 1970). 1. Steven Shavell, "Strict Liability versus Negligence," Journal of Legal Studies 9 (1980): 1; Steven Shavell, Economic Analysis of Accident Law (Harvard University Press, 1987). 2. Rylands v. Fletcher (1868) L.R. 3 H.L. 330. 3. Indiana Harbor Belt Railroad Co. v. American Cyanamid Co., 916 F.2d 1174 (7th Cir. 1990). 4. Greenman v. Yuba Power Products, Inc., 59 Cal. 2d 57 (1963). Chapter 5 1. United States v. Carroll Towing Co., 159 F.2d 169 (2d Cir. 1947); Conway v. O'Brien, 111 F.2d 611 (2d Cir. 1940); Moisan v. Loftus, 178 F.2d 148 (2d Cir. 1949). 1. Mark F. Grady, "Untaken Precautions," Journal of Legal Studies 18 (1989): 139. 2. Richard A. Posner, "A Theory of Negligence," Journal of Legal Studies 1 (1972): 29. 3. Blyth v. Birmingham Waterworks Co. (1856) 11 Ex. 781. 4. Henry T. Terry, "Negligence," Harvard Law Review 29 (1915): 40. 5. Adams v. Bullock, 227 N.Y. 208 (1919). 6. Bolton v. Stone [1951] A.C. 850 (H.L.); Overseas Tankship (U.K.) Ltd. v. Miller Steamship Co. Pty. (The Wagon Mound (No. 2)) [1967] 1 A.C. 617 (P.C.). 7. The T.J. Hooper, 60 F.2d 737 (2d Cir. 1932). 8. American Law Institute, Restatement (Third) of Torts: Liability for Physical and Emotional Harm § 3 (2010); American Law Institute, Restatement (Third) of Torts: Products Liability § 2 (1998). 9. Barker v. Lull Engineering Co., 20 Cal. 3d 413 (1978). 10. McCarty v. Pheasant Run, Inc., 826 F.2d 1554 (7th Cir. 1987). 11. Richard W. Wright, "Hand, Posner, and the Myth of the 'Hand Formula,'" Theoretical Inquiries in Law 4 (2003): 145; Stephen G. Gilles, "The Invisible Hand Formula," Virginia Law Review 80 (1994): 1015. 12. Helling v. Carey, 83 Wash. 2d 514 (1974). Chapter 6 1. Grimshaw v. Ford Motor Co., 119 Cal. App. 3d 757 (1981). 1. Gary T. Schwartz, "The Myth of the Ford Pinto Case," Rutgers Law Review 43 (1991): 1013. 2. W. Kip Viscusi, "Corporate Risk Analysis: A Reckless Act?" Stanford Law Review 52 (2000): 547; W. Kip Viscusi, "Jurors, Judges, and the Mistreatment of Risk by the Courts," Journal of Legal Studies 30 (2001): 107. 3. Philip E. Tetlock, "Thinking the Unthinkable: Sacred Values and Taboo Cognitions," Trends in Cognitive Sciences 7 (2003): 320. 4. Kim A. Kamin and Jeffrey J. Rachlinski, "Ex Post ≠ Ex Ante: Determining Liability in Hindsight," Law and Human Behavior 19 (1995): 89. 5. Executive Order 12291, 46 Fed. Reg. 13193 (1981); Executive Order 12866, 58 Fed. Reg. 51735 (1993). 6. W. Kip Viscusi, Pricing Lives: Guideposts for a Safer Society (Princeton University Press, 2018). 7. Mercado v. Ahmed, 974 F.2d 863 (7th Cir. 1992). 8. Office of Management and Budget, Circular A-4, "Regulatory Analysis" (September 17, 2003; revised November 9, 2023); Executive Order 14192, "Unleashing Prosperity Through Deregulation" (January 31, 2025). 9. Whitman v. American Trucking Associations, Inc., 531 U.S. 457 (2001); Entergy Corp. v. Riverkeeper, Inc., 556 U.S. 208 (2009); Michigan v. EPA, 576 U.S. 743 (2015). 10. Business Roundtable v. SEC, 647 F.3d 1144 (D.C. Cir. 2011). 11. Loper Bright Enterprises v. Raimondo, 603 U.S. 369 (2024). Chapter 7 1. Robert L. Birmingham, "Breach of Contract, Damage Measures, and Economic Efficiency," Rutgers Law Review 24 (1970): 273. 1. Daniel Friedmann, "The Efficient Breach Fallacy," Journal of Legal Studies 18 (1989): 1; Seana Valentine Shiffrin, "The Divergence of Contract and Promise," Harvard Law Review 120 (2007): 708. 2. Peevyhouse v. Garland Coal & Mining Co., 382 P.2d 109 (Okla. 1962). 3. Ian Ayres and Robert Gertner, "Filling Gaps in Incomplete Contracts: An Economic Theory of Default Rules," Yale Law Journal 99 (1989): 87; Hadley v. Baxendale (1854) 9 Ex. 341. 4. Gary S. Becker, "Crime and Punishment: An Economic Approach," Journal of Political Economy 76 (1968): 169; George J. Stigler, "The Optimum Enforcement of Laws," Journal of Political Economy 78 (1970): 526. 5. A. Mitchell Polinsky and Steven Shavell, "Punitive Damages: An Economic Analysis," Harvard Law Review 111 (1998): 869; Kemezy v. Peters, 79 F.3d 33 (7th Cir. 1996). 6. BMW of North America, Inc. v. Gore, 517 U.S. 559 (1996); State Farm Mutual Automobile Insurance Co. v. Campbell, 538 U.S. 408 (2003); Exxon Shipping Co. v. Baker, 554 U.S. 471 (2008). 7. George L. Priest and Benjamin Klein, "The Selection of Disputes for Litigation," Journal of Legal Studies 13 (1984): 1. 8. Paul H. Rubin, "Why Is the Common Law Efficient?" Journal of Legal Studies 6 (1977): 51; George L. Priest, "The Common Law Process and the Selection of Efficient Rules," Journal of Legal Studies 6 (1977): 65; Gillian K. Hadfield, "Bias in the Evolution of Legal Rules," Georgetown Law Journal 80 (1992): 583. Chapter 8 1. Christine Jolls, Cass R. Sunstein, and Richard Thaler, "A Behavioral Approach to Law and Economics," Stanford Law Review 50 (1998): 1471. 1. Richard H. Thaler and Cass R. Sunstein, Nudge: Improving Decisions About Health, Wealth, and Happiness (Yale University Press, 2008). 2. Louis Kaplow and Steven Shavell, "Why the Legal System Is Less Efficient than the Income Tax in Redistributing Income," Journal of Legal Studies 23 (1994): 667. 3. Ronald M. Dworkin, "Is Wealth a Value?" Journal of Legal Studies 9 (1980): 191. 4. Ernest J. Weinrib, The Idea of Private Law (Harvard University Press, 1995); Jules L. Coleman, Risks and Wrongs (Cambridge University Press, 1992); John C. P. Goldberg and Benjamin C. Zipursky, Recognizing Wrongs (Harvard University Press, 2020). Further Reading Robert Cooter and Thomas Ulen, Law and Economics, 6th ed. (Pearson, 2012). The standard introductory textbook, covering property, contracts, torts and crime with clear explanations of the underlying economic models. The best next step for a reader who wants the theory in more detail. Richard A. Posner, Economic Analysis of Law, 9th ed. (Wolters Kluwer, 2014). The treatise that turned a set of insights into a field. Opinionated and wide-ranging, it applies economic reasoning to nearly every area of law and shows the approach at its most ambitious. R. H. Coase, The Firm, the Market, and the Law (University of Chicago Press, 1988).** Reprints "The Problem of Social Cost" together with Coase's own later essay on how it had been read and misread. Essential for anyone who wants to know what the theorem actually claims. Guido Calabresi, The Costs of Accidents: A Legal and Economic Analysis (Yale University Press, 1970).** The founding economic treatment of accident law, and still the most thoughtful on how deterrence, loss spreading and administrative cost trade off against one another. Steven Shavell, Foundations of Economic Analysis of Law (Harvard University Press, 2004).** A rigorous and comprehensive statement of the modern theory by one of its principal architects, particularly strong on accident law, liability rules and litigation. Robert C. Ellickson, Order Without Law: How Neighbors Settle Disputes (Harvard University Press, 1991).** The Shasta County study, and a sustained argument about when informal norms rather than legal rules govern behavior. A model of empirical work in the field. W. Kip Viscusi, Pricing Lives: Guideposts for a Safer Society (Princeton University Press, 2018).** An accessible account of how the value of a statistical life is estimated and used, by the economist who has done more than anyone to develop it, including discussion of its role in litigation and regulation. Cass R. Sunstein, The Cost-Benefit Revolution (MIT Press, 2018).** A defense of regulatory cost-benefit analysis by a former administrator of the Office of Information and Regulatory Affairs, candid about its limits and its politics. Ernest J. Weinrib, The Idea of Private Law (Harvard University Press, 1995; revised ed., Oxford University Press, 2012).** The most rigorous statement of the corrective-justice critique of economic analysis. Demanding, but the necessary counterweight to the instrumental view. John C. P. Goldberg and Benjamin C. Zipursky, Recognizing Wrongs (Harvard University Press, 2020).** A readable modern account of tort law as a law of wrongs and civil recourse, with sustained engagement with the economic approach. Christine Jolls, Cass R. Sunstein, and Richard Thaler, "A Behavioral Approach to Law and Economics," Stanford Law Review 50 (1998): 1471.** The article that launched behavioral law and economics, and still the best short introduction to what psychology adds to the rational choice model. Hashtags: #LawAndEconomics #EconomicAnalysisOfLaw #LegalEconomics #LegalIncentives #EconomicEfficiency #RationalChoiceTheory #RationalChoice #LegalTheory #Jurisprudence #Microeconomics #BehavioralEconomics #BehavioralLawAndEconomics #CostBenefitAnalysis #EconomicIncentives #CoaseTheorem #RonaldCoase #TransactionCosts #PropertyRights #PropertyLaw #LiabilityRules #PropertyRules #CalabresiMelamed #TortLaw #TortLiability #NegligenceLaw #StrictLiability #HandFormula #LearnedHand #AccidentLaw #AccidentPrevention #LegalDeterrence #OptimalDeterrence #Externalities #SocialCosts #ParetoEfficiency #KaldorHicksEfficiency #WealthMaximization #EconomicWelfare #ContractLaw #ContractRemedies #LawOfDamages #PunitiveDamages #CriminalDeterrence #RiskAllocation #CheapestCostAvoider #ValueOfStatisticalLife #EconomicRegulation #LegalDecisionMaking #DistributiveJustice #EfficiencyAndJustice
- Data Provenance, Copyright, and Fair Use in Generative AI Systems (Litigation, training-data tracing, and compliant, auditable web collection)
Download the Book (PDF): Introduction In the summer of 2025, two federal judges in San Francisco, sitting in the same courthouse and ruling within three days of each other, decided the first two major American cases about whether it is lawful to train a large language model on copyrighted books. Both judges concluded that the training itself could be fair use. Both judges also said, in different ways, that the answer depended heavily on facts that had nothing to do with neural networks: where the books came from, whether they had been bought or downloaded from pirate libraries, and whether the resulting model was likely to flood the market with substitutes for the very works it had learned from. A year later, one of the defendants had agreed to pay one and a half billion dollars to settle claims arising from the way it had acquired its training data, not from the training. The model itself was never found to infringe. The library it was built from was the problem. That outcome is the clearest single illustration of the argument of this booklet. The legal fight over generative artificial intelligence is usually described as a contest of abstractions: is machine learning "transformative," is a model "a copy," does an opt-out regime respect authors or burden innovators. Those questions matter, and the chapters that follow take them seriously. But when courts actually decide cases, and when regulators actually write rules, the abstractions keep collapsing into a smaller set of concrete, factual questions. How was the data obtained? Did the rights holder signal that it did not consent, and did the developer honour that signal? Does the model retain and reproduce particular works, or does it learn something more general from them? Do its outputs compete with the originals in a market that the rights holder is entitled to? And can the developer prove its answers to any of these? Every one of those questions is a question about provenance: the documented origin, history, and chain of custody of the data that went into a system. The controlling claim of this booklet is that provenance has become the hinge on which the law of generative AI turns, and that organisations which build systems on open internet data need to treat it as an engineering discipline, not as an afterthought assembled by lawyers once litigation begins. A developer that can show what it collected, from where, under what terms, with what exclusions honoured, and with what safeguards against memorisation, is in a fundamentally different position from one that cannot, regardless of how the deeper doctrinal questions are eventually resolved. Why provenance, and why now For most of the deep learning era, the provenance of training data was treated as a research nuisance. Datasets were assembled by scraping, mirrored across academic servers, combined, filtered, and recombined. Licences were stripped or never recorded. The dataset known as Books3, which became a central exhibit in several lawsuits, circulated for years as part of a larger open-source corpus before anyone outside a small community asked where its nearly two hundred thousand books had come from. Image datasets containing billions of links were compiled from the public web with little attention to who held rights in the images or whether the sites hosting them had asked not to be crawled. When the Data Provenance Initiative, a volunteer research collective, audited more than 1,800 widely used fine-tuning datasets in 2023, it found that licence information was missing for the large majority of datasets on popular hosting platforms and that, where licences were recorded, they were frequently wrong. Three changes have ended that era. The first is litigation. Since late 2022, rights holders have filed well over a hundred lawsuits against AI developers in the United States alone, and courts in the United Kingdom, Germany, India, China and elsewhere have issued their first judgments. Discovery in these cases has forced developers to reconstruct, sometimes painfully, the history of datasets assembled years earlier. The second change is regulation. The European Union's AI Act now requires every provider of a general-purpose AI model placed on the EU market to maintain a copyright compliance policy, to respect machine-readable reservations of rights, and to publish a summary of the content used for training according to a template issued by the European Commission. Since 2 August 2026 the Commission's AI Office has had the power to enforce those obligations with fines. The third change is economic. A market for licensed training data now exists, with publishers, image libraries, music companies and online platforms selling access to their archives. A licence is only worth paying for if the licensee can later demonstrate that it used the licensed material and not something else, and only enforceable if the licensor can tell the difference. What this booklet covers The booklet moves from the ground up. It begins with how training corpora are actually built from the open web, because most confusion about the law of AI training comes from confusion about the facts of it. It then sets out the copyright mechanics of a training pipeline, stage by stage, identifying which acts plausibly implicate which rights. Two chapters follow on litigation: the first on the American fair use cases that have now produced substantive rulings, including the first appellate decision; the second on the wider map of disputes over news, music, images and books, and on the shift from lawsuits toward licensing settlements. A chapter on jurisdictions outside the United States follows, covering the European text and data mining framework, the AI Act, recent judgments in Germany and England, the Advocate General's November 2026 opinion in the first generative AI case before the Court of Justice of the European Union, and approaches in Japan, Singapore and India. The second half turns to the technical questions on which those legal disputes increasingly depend. One chapter examines what model weights actually retain about their training data, drawing on the research literature on memorisation and extraction. The next asks whether it is possible to trace a trained model's behaviour back to specific sources, surveying influence functions, membership inference, dataset inference, data watermarks and machine unlearning, and assessing honestly what each can and cannot establish. The final three chapters are practical. They cover the signalling systems through which website operators express consent or refusal, from the robots exclusion protocol to newer licensing standards; the architecture of a web collection pipeline designed to be compliant and auditable; and the governance, licensing and documentation practices that turn individual technical controls into an audit file that will survive scrutiny by a regulator, a licensor or a court. What "audit-proof" means here The phrase "audit-proof" can be misread, so it is worth being precise. In this booklet an audit-proof pipeline is one that withstands an audit because it was built to be audited: it respects the refusals expressed by site operators, it records the terms under which each item was collected, it honours licences and opt-outs, it can show its work, and it can remove material when required. It is not a pipeline designed to evade detection, disguise its identity, circumvent technical barriers or frustrate accountability. Those techniques exist, and some of the litigation discussed in later chapters concerns them. They are the opposite of what is described here, and recent case law suggests that they convert a contestable legal position into an indefensible one. A note on stance The legal questions covered here are genuinely contested, and serious people disagree about them. Authors, artists, journalists and musicians argue that their work has been taken without permission or payment to build products that compete with them. Developers and many academics argue that learning from publicly available material is what humans and machines alike have always done, that the outputs of a well-designed model do not reproduce the works it learned from, and that requiring permission for every item in a corpus of billions would make the technology impossible to build except by the largest incumbents. Both positions are presented as fairly as possible. Where courts have ruled, their rulings are described accurately and with dates; where questions remain open, they are described as open. Litigation status is stated as of mid-November 2026, and several of the cases discussed will have moved on by the time this is read. Nothing here is legal advice. Readers facing a specific decision should consult a lawyer qualified in the relevant jurisdiction. Chapter 1. How Training Corpora Are Built Public argument about AI training tends to picture a single act: a company "feeds" the internet into a model. The reality is a long chain of separate operations, often carried out by different organisations at different times, each of which leaves its own legal and documentary footprint. Understanding that chain is the precondition for understanding everything else in this booklet. A court deciding whether a use is fair, a regulator asking for a training data summary, and an engineer trying to remove a work from a future model are all, in effect, asking where in the chain something happened and who did it. From crawl to corpus Almost every large text corpus used for pretraining language models begins with a web crawl. A crawler is a program that requests web pages, stores what it receives, extracts the links on each page, and adds those links to a queue of pages to request next. Search engines have run crawlers since the 1990s. What changed with large-scale machine learning is that the archive of crawled pages became valuable in itself, independent of any search index built from it. The most important single source is Common Crawl, a non-profit founded in 2007 that has released regular snapshots of the web, free of charge, for well over a decade. Its archive now runs to hundreds of billions of pages and several petabytes of data. Common Crawl stores pages in a standard archival format, WARC, which preserves the raw HTTP response alongside metadata such as the URL and the time of retrieval. It also publishes derived files containing extracted text. Common Crawl's crawler identifies itself with the user-agent string CCBot and states that it follows the robots exclusion protocol, the convention by which websites tell crawlers which paths they may or may not visit. Common Crawl does not claim to own or license the content it archives; it describes itself as providing access to material that was publicly available, and leaves users to determine their own legal position. Raw crawl data is unusable for training without heavy processing. A typical pipeline extracts the main text from each page, discarding navigation menus, advertisements and boilerplate; identifies the language; removes pages that are too short, too repetitive or dominated by symbols; filters out content judged low-quality by heuristics or by a classifier trained to recognise "good" text; strips or masks personal information such as email addresses; removes exact and near-duplicate documents; and, increasingly, filters for toxicity or unsafe material. Each step changes what ends up in the corpus, and each step is a decision someone made and could, in principle, document. The lineage of named public datasets shows how these steps accumulate. Google's C4 corpus, released in 2019 alongside the T5 model, was built from a single month of Common Crawl, filtered with simple rules such as keeping only lines ending in terminal punctuation and discarding pages containing words from a list of offensive terms. EleutherAI's The Pile, released at the end of 2020, combined filtered Common Crawl text with twenty-one other sources, including academic papers, code, legal opinions, patents, subtitles, and a component called Books3. The Technology Innovation Institute's RefinedWeb, Allen Institute for AI's Dolma, and Hugging Face's FineWeb, released between 2023 and 2024, pushed web-only corpora to trillions of tokens with more sophisticated filtering and deduplication, and published more detailed documentation of how they were constructed. These later projects are, in part, a response to the earlier opacity: their creators understood that a dataset without a documented construction process was becoming a liability. The components that cause trouble Not every source in a corpus carries equal legal risk, and the litigation of the past four years has concentrated on a few specific categories. The first is books. Long, carefully edited prose is valuable for training models to sustain coherent reasoning over many pages, and books are much harder to obtain lawfully at scale than web pages. Books3 contained nearly two hundred thousand books, many of them in copyright, derived from a private ebook tracker. Other corpora drew on so-called shadow libraries such as Library Genesis, Z-Library and the Pirate Library Mirror, which host millions of pirated books and articles. Discovery in American litigation showed that at least two major developers downloaded large collections from such sources. In the case against Anthropic, the company had also bought millions of print books, cut off their bindings, scanned them, and destroyed the paper originals, and the court treated those two acquisition routes very differently, as Chapter 3 explains. The second is news. News articles are abundant on the open web but often sit behind paywalls or are governed by restrictive terms of service. Publishers argue that their archives represent decades of expensive reporting and that AI products which summarise or answer questions about current events compete directly with them. The lawsuit filed by The New York Times in December 2023 alleged that millions of its articles had been used to train OpenAI's models; by the time the parties moved for summary judgment in September 2026, the consolidated news plaintiffs were asserting more than ten million works. The third is images. Text-to-image models such as Stable Diffusion were trained on datasets built by the German non-profit LAION, whose LAION-5B collection consisted of roughly 5.85 billion pairs of image URLs and the alternative text attached to them on the web. LAION distributed links and captions, not the images themselves; anyone training on the dataset had to download the images from their original hosts. That design was intended to reduce LAION's own legal exposure, and a German court later accepted that LAION's compilation of the dataset fell within research and text and data mining exceptions. But it also meant that the images any given model was actually trained on depended on what was retrievable at the moment of download, and that the dataset contained whatever the web contained. In December 2023, researchers at the Stanford Internet Observatory reported finding more than a thousand verified instances of child sexual abuse material among the links in LAION-5B. LAION withdrew the dataset and in 2024 released a cleaned version. The episode showed that provenance failures are not only a copyright problem. The fourth is code and user-generated content. Code repositories carry explicit licences, many of which, such as the GNU General Public License or the MIT License, impose attribution or share-alike conditions. Forums, question-and-answer sites and social platforms host material created by millions of users under terms of service that grant the platform broad rights. Platforms such as Reddit and Stack Overflow have since restricted access to their data through their application programming interfaces and begun selling licensed access to AI developers, while suing those they accuse of taking it without permission. "Publicly available" is not a licence The phrase that recurs throughout developer documentation and public statements is "publicly available data." It is a factual description, not a legal category. A web page is publicly available in the sense that anyone with a browser can request it. That says nothing about whether its contents are protected by copyright, which, in most jurisdictions, they are by default from the moment of creation. It says nothing about whether the site's terms of service prohibit automated collection, or whether its robots.txt file asks crawlers to stay away. And it says nothing about whether the person who posted it had the right to do so; a pirated book on a public file-sharing site is publicly available in the same sense. Three legal regimes intersect at the point of collection. Copyright governs the reproduction of protected expression, and is the subject of most of this booklet. Contract governs the terms on which a site offers access, and can bind a crawler that has agreed to those terms, whether by clicking through or, in some jurisdictions and circumstances, merely by using the site with notice of them. Computer misuse and anti-circumvention laws govern access to systems and the bypassing of technical protections: the Computer Fraud and Abuse Act in the United States, the Computer Misuse Act in the United Kingdom, and, in copyright statutes on both sides of the Atlantic, prohibitions on circumventing technological measures that control access to works. American courts have narrowed the reach of the Computer Fraud and Abuse Act for scraping of public pages, notably in the Supreme Court's 2021 decision in Van Buren v. United States and the Ninth Circuit's rulings in hiQ Labs v. LinkedIn. But as Chapter 8 shows, anti-circumvention claims under copyright law have re-emerged as a significant risk for scrapers that work around anti-bot systems. The upshot is that the same act of downloading a page can be lawful or unlawful depending on facts the downloader may not have recorded: what the page said about permitted uses, what the robots.txt file said on the day of the crawl, whether the content was behind a login or paywall, and what technical measures were in place. A pipeline that does not capture those facts at collection time cannot reliably reconstruct them later. Websites change their terms and their robots.txt files constantly, and archived copies of those files are incomplete. The shape of the problem Three features of corpus construction make provenance hard and explain why it has become the hinge of the legal debate. The first is scale. A modern pretraining corpus contains billions of documents drawn from hundreds of millions of domains. No developer can individually assess the rights position of each item. Any compliance approach must therefore operate on rules and signals that can be applied automatically: domain-level policies, machine-readable opt-outs, licence detection, blocklists of known infringing sources. Where those automatic signals are absent or ambiguous, the developer is making a judgement under uncertainty, and the quality of its documentation determines whether that judgement can later be defended as reasonable. The second is layering. Datasets are built on datasets. A model developer may train on a corpus assembled by a second organisation from a crawl conducted by a third, mixed with a licensed archive from a fourth and synthetic data generated by a model trained by a fifth. Each layer can lose information about the layer beneath it. When the Data Provenance Initiative traced the lineage of popular fine-tuning datasets, it found that licences applied by aggregators often failed to reflect the more restrictive terms of the underlying sources. A developer that relies on an aggregator's licence label without checking the chain is inheriting someone else's mistakes. The third is time. Training runs that produced models now in commercial use were conducted years ago, on data collected years before that, under norms and rules that have since changed. The robots.txt directives aimed specifically at AI crawlers barely existed before August 2023, when OpenAI published the user-agent for its GPTBot crawler and many publishers moved quickly to block it. The EU's text and data mining opt-out has applied since 2021, but the means of expressing it have been contested in court as recently as December 2025, when a German appeal court held that a plain-language reservation in a website's terms was not machine-readable for a download made in 2021. A defensible provenance record must therefore capture not only what was collected but when, and what the applicable signals said at that moment. These features do not make compliance impossible. They make it an engineering problem. The rest of this booklet is about how the law has come to depend on the answers to provenance questions, how far technical methods can supply those answers after the fact, and how pipelines can be built to supply them from the start. Chapter 2. The Copyright Mechanics of a Training Pipeline Copyright law does not regulate "AI." It regulates specific acts done to protected works: reproducing them, distributing copies, communicating them to the public, adapting them, and, in some systems, making them available online. To ask whether training a model infringes copyright is therefore to ask a series of narrower questions about which of those acts occur, at which stage of the pipeline, by whom, and whether any exception, defence or licence covers them. This chapter works through those stages. It is the conceptual map the rest of the booklet relies on. What copyright protects, and what it does not Copyright protects original expression: the particular words, images, melodies or code that an author has chosen. It does not protect ideas, facts, methods, systems or styles. This distinction, often called the idea-expression dichotomy in American law, is central to the defence of machine learning. Developers argue that a model learns statistical regularities about language or images, which are closer to facts and ideas than to protected expression. A model that has learned how sonnets tend to be structured, or what the typical colour palette of an impressionist landscape looks like, has learned something copyright does not own. The counter-argument starts from the observation that, in order to learn those regularities, the training process must first make complete copies of the protected expression, and that the resulting model may, in some cases, retain enough of that expression to reproduce it. Both observations are factually correct. The legal question is what follows from them. Facts matter in a second sense as well. Many of the most valuable training sources, such as news reports, encyclopaedias and legal databases, are largely factual. The facts in a news story are free for anyone to use; the reporter's choice of words is not. The Third Circuit Court of Appeals, ruling in September 2026 on whether an AI developer infringed by copying Westlaw's case-law headnotes, accepted that the headnotes cleared copyright's "admittedly low bar" for originality because they reflected editorial choices about what points to draw from a judicial opinion and how to phrase them, but it also treated their factual character as weighing slightly in the copier's favour. That pattern, thin protection for largely factual material, recurs across the case law. Stage by stage The acts that occur in a training pipeline can be grouped into seven stages. Table 1 summarises them; the discussion that follows explains each. Table 1. Stages of a generative AI pipeline and the legal questions each raises. Stage What happens Main rights implicated Typical defences or limits Key provenance question Acquisition Pages, files or archives are downloaded or bought Reproduction; contract; anti-circumvention Fair use or TDM exception; implied licence; lawful access Where did each item come from, and on what terms? Storage and corpus building Copies retained, processed, combined into datasets Reproduction; possibly distribution if shared Same as above; research exceptions Which copies were kept, for how long, and who received them? Training Model processes copies and updates its weights Reproduction (transient and persistent) Fair use; TDM exception; temporary copies exception Which data went into which model version? The model Weights are stored, copied, distributed or imported Reproduction if works are memorised; secondary infringement Model is not a copy if no works are stored Does the model retain and reproduce specific works? Outputs Model generates text, images, audio or code Reproduction; adaptation; communication to the public No substantial similarity; user responsibility Do outputs reproduce protected expression? Retrieval and grounding Model fetches live content to answer a query Reproduction; communication to the public; press publishers' right Fair use; quotation; short extracts Which sources were fetched, and how much was shown? Management information Author names, titles or licence notices removed Removal of rights management information Lack of knowledge or intent Was attribution data preserved or stripped? Acquisition The first act is getting the data. Downloading a web page makes a copy of it, as does buying a digital archive, scanning a physical book, or receiving a dataset from another organisation. In copyright terms each of these is a reproduction. Whether it infringes depends on whether a defence applies, and here the source matters enormously. A copy made from a lawfully accessed web page, a purchased book or a licensed archive stands on different ground from one made from a pirate site, a paywalled article reached by circumventing the paywall, or a website whose operator has explicitly refused permission. The American cases have drawn this distinction sharply. In Bartz v. Anthropic, Judge William Alsup held in June 2025 that scanning purchased print books to create a digital library was fair use, but that downloading millions of pirated books to build a "central library" was not, even if some of those copies were later used for training. In the European framework, the general text and data mining exception applies only to works to which the miner has "lawful access," and the Advocate General's November 2026 opinion in the Like Company case read that condition as excluding pirate libraries and the circumvention of paywalls or bot-blocking measures. Acquisition, in short, is where many of the most serious legal risks sit, and it is also the stage that technical provenance controls can most directly govern. Acquisition also brings in non-copyright law. A crawler that agrees to a site's terms of service, or ignores a technical barrier, may face claims for breach of contract or under anti-circumvention provisions even where the copying itself would be defensible. Storage and corpus building Once acquired, copies are stored, cleaned, filtered, deduplicated and assembled into datasets. These operations involve further reproductions, and storage can be prolonged. A copy kept indefinitely in a "central library," available for future uses not yet decided, was precisely what troubled Judge Alsup. If the dataset is shared with other organisations or published, distribution and making available become relevant. LAION's choice to distribute links rather than images was a response to this risk. Storage also matters for compliance. The European text and data mining exceptions permit reproductions to be retained only "for as long as is necessary for the purposes" of the mining. A developer that keeps raw copies forever, with no documented retention policy, may be outside the exception even if its initial copying was within it. Training During training, the model processes copies of the data, typically many times over, and adjusts billions of numerical parameters to reduce its prediction error. The copies made in memory during this process are transient. Some jurisdictions have exceptions for temporary copies that are an integral part of a technological process and have no independent economic significance, such as Article 5(1) of the EU's InfoSoc Directive. Advocate General Szpunar suggested in November 2026 that this exception might cover the transient copies made in the course of training but not the stored training dataset itself. The central American question is whether training as a whole is fair use. The central European question is whether it falls within the text and data mining exceptions. Both are addressed in detail in later chapters. For present purposes the point is that training is usually analysed as a distinct act from acquisition and from output, although, as Chapter 5 notes, the Advocate General's questioning at the Like Company hearing raised the possibility that courts might eventually consider the whole chain together. The model The trained model is a large file of numbers. Whether that file is itself a copy of the works it was trained on is one of the most consequential questions in the field, because if it is, every distribution, download or import of the model becomes a potential infringement. The answer depends on facts about memorisation. A model that cannot reproduce any particular training work, and from which no work can be extracted, is difficult to describe as a copy of anything. A model from which substantial portions of a novel or the complete lyrics of a song can be reliably extracted looks different. In November 2025 the High Court of England and Wales held in Getty Images v. Stability AI that the Stable Diffusion model was not an "infringing copy," because Getty had accepted that the model weights did not store its images. In the same month, the Munich Regional Court reached a different conclusion on different facts in GEMA v. OpenAI, holding that song lyrics memorised in OpenAI's models were reproduced in the models themselves. Chapter 6 examines the technical evidence behind these conflicting outcomes. Outputs When a user prompts a model, it generates new material. If that material reproduces or closely imitates protected expression from a training work, the output can infringe regardless of whether training was lawful. Every major jurisdiction agrees on this, and even the strongest defenders of training as fair use accept that infringing outputs are a separate problem. The disputed questions are how often it happens, who is responsible when it does (the developer, the deployer or the user who wrote the prompt), and what safeguards a developer must adopt. The EU's General-Purpose AI Code of Practice commits signatories to implement "proportionate technical safeguards" against infringing outputs. In the Munich judgment against the AI music company Suno in July 2026, the court held Suno, not its users, liable for outputs generated in Germany, and found that simple, open-ended prompts were enough to produce outputs substantially similar to the claimant's works. Outputs can also raise questions outside copyright. Several news publishers have pleaded trademark claims where models attributed fabricated articles to them, and some of those claims have survived motions to dismiss. Retrieval and grounding Many deployed systems do not rely only on what the model learned during training. They fetch documents at the moment a question is asked, through web search or a database lookup, and feed those documents into the model as context for its answer. This technique, retrieval-augmented generation, raises distinct issues. The retrieved content is copied at query time, often in full, and the answer may summarise, paraphrase or quote it. Unlike training data, which is diffused across billions of parameters, retrieved content is directly present and directly traceable. Retrieval is where several of the most aggressive news-publisher claims now sit, including those against Perplexity and the "grounding" claims in the New York Times litigation. It is also the subject of the Like Company referral to the Court of Justice, where Advocate General Szpunar concluded that a chatbot retrieving a press article in real time and reusing it in an answer could, in certain circumstances, infringe the press publishers' right created by the 2019 Digital Single Market Directive. In India, by contrast, the Delhi High Court held at the interim stage in July 2026 that outputs generated through retrieval in ANI v. OpenAI were not substantially similar to the claimant's works and did not infringe. Management information Finally, copyright statutes protect "copyright management information": author names, titles, licence notices and similar data attached to a work. In the United States, Section 1202 of the Digital Millennium Copyright Act prohibits knowingly removing such information with reason to know that the removal will facilitate infringement. Text extraction pipelines routinely strip bylines, copyright notices and licence headers as boilerplate. Plaintiffs in several early cases pleaded Section 1202 claims on this basis. Many have been dismissed, often because plaintiffs could not identify specific outputs that reproduced their works without the management information, but the claim survives in some cases and the underlying engineering fact is real. A pipeline that preserves attribution data in its provenance records, even if it removes it from the text fed to the model, is in a stronger position. Who does what The stage-by-stage analysis also clarifies who is responsible for what. The organisation that crawled the web, the one that built the dataset, the one that trained the model, the one that deployed it, and the user who prompted it may all be different. Copyright law has well-developed doctrines for allocating responsibility: direct infringement for the party whose volitional conduct causes the copy; contributory and vicarious liability, or their equivalents, for those who knowingly facilitate or profit from another's infringement; and secondary infringement provisions for importing or dealing in infringing articles. These allocations matter for provenance because each party can only document what it controls. A developer that trains on a third-party corpus can record which corpus and which version it used, what due diligence it performed on that corpus, and what representations it obtained from the supplier, but it cannot record facts about the original crawl that the supplier never captured. This is why the provenance practices described in later chapters emphasise contractual representations and chain-of-custody documentation as well as technical controls. Licences and their limits Not all training data is used without permission. Much of the open web is published under licences that expressly permit reuse. Wikipedia text is available under a Creative Commons Attribution-ShareAlike licence. Many government publications are in the public domain or under open government licences. Open-source code comes with licences that permit copying subject to conditions. Using licensed material does not end the analysis, because licences have conditions. Creative Commons licences require attribution and, in their ShareAlike variants, require derivatives to carry the same licence. Whether a trained model is a "derivative" or "adaptation" of each Creative Commons work in its corpus is unsettled, and Creative Commons itself has taken the position that whether its licences' conditions apply to training depends on whether copyright permission was needed in the first place: where training is permitted by an exception or by fair use, the licence conditions do not bite. Non-commercial licences raise a different problem for commercial developers. Code licences with copyleft conditions raise the question of whether model outputs that reproduce licensed code must carry the licence. A provenance system that records licence terms at collection time lets a developer make these judgements deliberately and apply different treatment to different licence classes. The broader lesson of this chapter is that "is AI training legal?" is not one question. It is at least seven, attached to different acts at different stages, and the answer to each depends on facts about provenance: the source, the terms, the signals, the retention, the model's behaviour, and the output. The litigation discussed in the next two chapters can be read as a long, expensive process of establishing those facts. Chapter 3. Fair Use on Trial in the United States The United States has no statutory exception for text and data mining. American developers who train on copyrighted works without a licence rely instead on fair use, the flexible defence codified in Section 107 of the Copyright Act. Because fair use is decided case by case, on the facts, the law of AI training in the United States is being made by judges, one ruling at a time. By November 2026 there were three substantive federal decisions on training, one of them now affirmed on appeal, together with a government report, a statement of interest from the Department of Justice, and a large pending summary judgment motion in the consolidated news cases. Read together, they show a doctrine that is converging less on a single answer than on a set of factual distinctions, nearly all of them about provenance and market effect. The four factors Section 107 directs courts to weigh four factors: the purpose and character of the use, including whether it is commercial; the nature of the copyrighted work; the amount and substantiality of the portion used; and the effect of the use on the potential market for or value of the work. Courts treat these as considerations to be weighed together, not as a checklist, but two factors usually dominate. The first factor asks, above all, whether the use is "transformative": whether it adds something new, with a further purpose or different character, rather than merely superseding the original. The Supreme Court's 2023 decision in Andy Warhol Foundation v. Goldsmith narrowed this inquiry. The Court held that the question is whether the specific use at issue shares the purpose of the original, and that a use which serves substantially the same purpose, particularly a commercial one, will weigh against fair use even if it adds new expression. Developers rely on an older line of cases holding that copying entire works can be fair where the copy is used for a fundamentally different function: Authors Guild v. Google, in which the Second Circuit in 2015 upheld Google's scanning of millions of books to create a searchable index that displayed only snippets; and the Ninth Circuit's software cases, Sega v. Accolade and Sony v. Connectix, which permitted copying code as an intermediate step toward building compatible, non-infringing products. The fourth factor asks whether the use harms the market for the original, including markets the rights holder would be likely to develop. Rights holders argue that AI developers are displacing an emerging licensing market for training data and, more fundamentally, that AI outputs compete with the works they were trained on. Developers respond that a market for licensing is circular, since it exists only if the use requires a licence, and that outputs which do not reproduce a particular work cannot harm that work's market. Thomson Reuters v. Ross: the first ruling, and the first appeal The first substantive decision did not involve generative AI at all. Ross Intelligence was a start-up building a legal research tool that would answer natural-language questions with relevant passages from judicial opinions. It sought a licence to Westlaw content from Thomson Reuters, a competitor, and was refused. It then obtained, through a third-party contractor, around twenty-five thousand "bulk memos" consisting of legal questions paired with answers; the questions had been drafted with the help of Westlaw's headnotes, the short summaries of legal points that Westlaw editors attach to cases. Ross used the memos to train its search tool. Thomson Reuters sued in 2020, and Ross shut down its operations the following year, citing the cost of the litigation. Judge Stephanos Bibas, a Third Circuit judge sitting by designation in the District of Delaware, initially found that fair use raised questions for a jury. In February 2025 he revised his opinion and granted partial summary judgment to Thomson Reuters on its infringement claims covering more than two thousand headnotes. He held that Ross's use was commercial and not transformative: it used the headnotes to build a competing legal research product, serving the same purpose as the original. He distinguished the intermediate-copying cases on the ground that they concerned computer code, which must be copied to achieve compatibility, whereas Ross could have built its training materials without copying the headnotes. And he found that Ross's product was a market substitute for Westlaw and harmed a potential market for licensing headnotes as AI training data. The Third Circuit accepted an interlocutory appeal and, on 29 September 2026, affirmed in a sealed opinion, making it the first federal appellate ruling on fair use in AI training. According to published accounts of the decision, the panel found the headnotes sufficiently original to be protected, held that the first, third and fourth factors favoured Thomson Reuters, and accepted that the factual nature of the headnotes tilted the second factor slightly toward Ross. On the market factor, the court recognised harm both to the legal research market and to a developing market for licensing headnotes as training data. The court also rejected Ross's intermediate-copying argument for the same reason Judge Bibas had: copying the headnotes was the easy way to build training data, not a necessary one. The most important feature of the appellate decision, however, may be what it declined to decide. The court expressly distinguished Ross's system from the generative models at issue in other litigation, noting that Ross's platform could not generate original expression. That carve-out means the decision is binding authority for non-generative systems that compete with their sources, and persuasive authority on the market for training data, but it leaves the fair use status of generative training open. Commentators across the spectrum read it that way. For rights holders, its recognition of a training-data licensing market is valuable. For developers, its insistence on the generative distinction preserves the arguments that prevailed in San Francisco. Bartz v. Anthropic: training transformative, piracy not In June 2025, Judge William Alsup of the Northern District of California decided a summary judgment motion brought by Anthropic against a putative class of authors led by Andrea Bartz, Charles Graeber and Kirk Wallace Johnson. The record showed that Anthropic had acquired books in two ways. It had downloaded millions of pirated books from Books3, Library Genesis and the Pirate Library Mirror, and it had later bought millions of print books, many of them used, stripped their bindings, scanned the pages, and discarded the originals. Copies from both sources were retained in a central digital library, and subsets were used to train the company's Claude models. Judge Alsup analysed three uses separately. On training, he held that the use was fair, describing it as quintessentially transformative: the models learned from the works in the way a reader aspiring to be a writer learns from books, in order to create something different. Significantly, the authors did not allege that any Claude output reproduced their works, and the judge noted that if they had, the analysis would be different. On the purchased print books, he held that converting a lawfully purchased print copy into a digital copy for internal use, while destroying the original, was a fair use akin to format-shifting, since it did not multiply the number of copies. On the pirated library, he held that downloading pirated copies to build a permanent, general-purpose library was not fair use, even if some of those copies were later used for a transformative purpose. The fact that a developer might later make a fair use of a copy did not excuse the way it was obtained, particularly where lawful alternatives existed. The ruling reshaped the case. The judge certified a class of rights holders in books downloaded from Library Genesis and the Pirate Library Mirror in July 2025 and set a trial on the piracy claims for December. With statutory damages of up to $150,000 per work for wilful infringement in prospect, across hundreds of thousands of works, the potential exposure was existential. The parties announced a settlement in September 2025 of $1.5 billion, and the court granted preliminary approval later that month. On 20 July 2026, Judge Araceli Martínez-Olguín, to whom the case had been reassigned after Judge Alsup's retirement, granted final approval. The settlement covers 482,460 works on a defined list, at roughly $3,000 per work, four times the $750 statutory minimum. Anthropic must destroy the original files it torrented from Library Genesis and the Pirate Library Mirror and any copies derived from them. The release is limited to past conduct up to 25 August 2025 and expressly does not cover claims based on AI outputs or future conduct. The court rejected class counsel's request for 12.5 percent of the fund in fees and awarded about 6.8 percent, just over $101 million. The Bartz settlement is the largest copyright recovery in American history. Its significance for this booklet lies in what it paid for. Anthropic was not found to have infringed by training. It paid to resolve liability for acquiring and keeping pirated copies. The ruling and settlement together make acquisition provenance the single most financially consequential fact in American AI copyright litigation to date. Kadrey v. Meta: a win built on the wrong arguments Two days after Judge Alsup's ruling, Judge Vince Chhabria, in the same courthouse, granted summary judgment to Meta against thirteen authors, including Richard Kadrey, Sarah Silverman and Ta-Nehisi Coates, who alleged that Meta had trained its Llama models on their books, downloaded from shadow libraries. Meta won, but the opinion reads in places like a warning to developers. Judge Chhabria agreed that training Llama was highly transformative. But he wrote that the fourth factor, market harm, was the most important, and that in many circumstances copying protected works to train generative models without permission would be unlawful, because such models can generate a vast quantity of competing material and thereby dilute the market for the originals. He called this theory "market dilution" and suggested it could defeat fair use even where no output reproduces any particular work. The plaintiffs lost because they had not developed that argument or offered evidence to support it. They had argued instead that Llama could reproduce their books, but the evidence showed it could not do so in any meaningful amount, and that Meta had harmed a market for licensing their books for training, which the judge regarded as a market the authors were not entitled to as a matter of law. The ruling, he stressed, applied only to these thirteen plaintiffs and did not establish that Meta's use of copyrighted materials was lawful. Judge Chhabria also treated the use of shadow libraries differently from Judge Alsup. He did not regard downloading from pirate sources as dispositive in itself, but considered it relevant to the analysis, particularly insofar as Meta's torrenting might have contributed to the distribution of pirated works. Claims that Meta distributed copyrighted works through BitTorrent while downloading them survived the summary judgment ruling. Under the court's scheduling order, summary judgment briefing on those distribution claims runs from December 2026 into February 2027, with a hearing set for 25 February 2027. The contrast between the two San Francisco rulings has been widely discussed. On training, they agree. On acquisition, Alsup treated piracy as independently disqualifying, while Chhabria folded it into the overall weighing. On market harm, Chhabria offered a theory, market dilution, that would make fair use substantially harder for generative models if plaintiffs can prove it. Both emphasised the absence of evidence that the models reproduced the plaintiffs' works. Both, therefore, made the outcome turn on factual records about where data came from and what the models do. The Copyright Office and the executive branch In May 2025 the US Copyright Office released a pre-publication version of the third part of its report on copyright and artificial intelligence, addressing generative AI training. The report concluded that training a generative model on a large and diverse dataset will often be transformative, but that the degree of transformation depends on the use: training a model to generate outputs that share the purpose of the originals, such as music generated to compete with the recordings it was trained on, is less transformative than training for research or for non-expressive tasks. It accepted that market dilution was a cognisable harm under the fourth factor. It observed that the knowing use of pirated or illegally accessed works weighs against fair use without being determinative. It concluded that voluntary licensing markets were developing and that government intervention was premature. The day after the report appeared, the administration dismissed the Register of Copyrights, Shira Perlmutter. She challenged her removal, and the dispute reached the Supreme Court, which on 30 June 2026 declined to stay a lower court order allowing her to remain in office pending the litigation. The report itself remains in pre-publication form. It is not binding on courts, but it has been cited by parties on both sides. The executive branch has taken a more developer-friendly line elsewhere. In early September 2026, on the eve of summary judgment motions in the consolidated news cases against OpenAI and Microsoft, the Department of Justice filed a statement of interest supporting OpenAI's fair use defence. The President had earlier remarked publicly that AI developers could not be expected to pay for every article or book from which their systems learned. Statements of interest carry no legal force, and federal judges routinely disagree with the government, but the filing signalled that the developer position has institutional support. The pending test: the news cases The most closely watched pending matter is the consolidated litigation against OpenAI and Microsoft before Judge Sidney Stein in the Southern District of New York. It combines the New York Times case with suits by the Daily News and other newspapers, the Center for Investigative Reporting, Ziff Davis, and author classes including the Authors Guild. In March and April 2025, Judge Stein largely denied motions to dismiss the core infringement claims while dismissing some unfair competition and Digital Millennium Copyright Act claims. Discovery produced a dispute over the preservation of ChatGPT user logs that drew public attention; in January 2026 the judge upheld a magistrate judge's order requiring OpenAI to produce a de-identified sample of twenty million logs. On 4 September 2026 the parties in the news cases filed cross-motions for summary judgment. According to public accounts of the filings, OpenAI argued that pretraining and its browsing feature are fair use, that browse copies made before publishers blocked its crawler were impliedly licensed, and that verbatim regurgitation was extremely rare, with twenty-four instances identified in the twenty-million-log sample. The news plaintiffs sought judgment on liability for copying at several stages, including acquisition, training, grounding and output; argued that crawl-to-referral ratios showed AI products take far more than they return in traffic; and advanced a market dilution theory based on low-cost automated news sites. Unsealed filings reported in September 2026 also included evidence that the plaintiffs say shows paywall circumvention and terms-of-service violations, which, if credited, would bring the acquisition questions that proved decisive in Bartz to the centre of the news litigation. No ruling had been issued as of mid-November 2026. What the American cases have settled, and what they have not Several propositions now have meaningful judicial support in the United States. Training a generative model on lawfully acquired works has been held to be transformative by both judges who have squarely considered it, although neither ruling binds other courts and both are district-level decisions. Acquiring works from pirate sources to build a permanent library has been held not to be fair use by one court and treated as relevant but not decisive by another; either way, it has proved enormously costly. Training a non-generative system that competes directly with its source has been held not to be fair use at the appellate level. Market dilution has been recognised as a potential harm by a district judge and the Copyright Office, but no court has yet found it proved on an evidentiary record. And infringing outputs remain actionable regardless of the status of training. What remains open is at least as important. No appellate court has ruled on generative training. No court has ruled on whether retrieval-augmented generation is fair use. No court has decided whether a developer's honouring or ignoring of opt-out signals affects the fair use analysis, although the European framework makes that question central and American plaintiffs have begun to plead it. And no court has determined how much memorisation, or what rate of verbatim output, is enough to turn a transformative use into a substitutive one. What is clear is the direction of travel. American fair use doctrine is resolving the AI training question not through a single holding about machine learning but through a series of distinctions: lawful versus unlawful acquisition; generative versus competing non-generative use; models that do versus models that do not reproduce their training works; and markets that are versus are not harmed. Each distinction is a factual question, and the evidence that answers it is provenance evidence. A developer that cannot show where its data came from, or demonstrate what its model does and does not reproduce, cannot take advantage of the doctrine that has so far favoured it. Chapter 4. The Litigation Map: News, Music, Images, Books and Platforms The fair use rulings described in the previous chapter arose from a small number of cases about books. They sit within a much larger body of litigation that, by late 2026, covered nearly every creative and informational industry. The disputes differ in their legal theories and in what the plaintiffs want, and those differences reveal how the provenance questions vary by medium. A news publisher worries about real-time retrieval and market substitution; a record label worries about outputs that sound like its recordings; an image library worries about watermarks reappearing in generated pictures; a platform worries about scrapers that bypass its access controls. This chapter surveys the main fronts, then examines the most significant trend of the past eighteen months: the conversion of lawsuits into licences. News and reference publishers News organisations have been the most active institutional plaintiffs. The New York Times sued OpenAI and Microsoft in December 2023, after licensing talks failed. Its complaint was notable for its exhibits: a hundred examples in which GPT-4, prompted with the opening of a Times article, produced long passages closely matching the original. OpenAI responded that the examples were produced by deliberately manipulative prompting that violated its terms of use and did not reflect how ordinary users interact with its products. That exchange framed what has become a central evidentiary battle in the news cases: how often models reproduce articles in normal use, and whether the plaintiff or the defendant bears the burden of proof on that question. The Times case is now part of a multidistrict litigation in the Southern District of New York, consolidated in April 2025, that also includes the Daily News and other Alden Global Capital newspapers, the Center for Investigative Reporting, Ziff Davis, and several author class actions. As described in Chapter 3, cross-motions for summary judgment on fair use were filed in September 2026 and remain pending. A second group of cases targets answer engines that rely heavily on retrieval. Dow Jones and the New York Post sued Perplexity in October 2024, alleging that Perplexity copied their articles to generate answers that substituted for visiting their sites, and that it sometimes attributed fabricated text to them. Encyclopaedia Britannica and its subsidiary Merriam-Webster sued Perplexity in 2025 on similar theories; that case was in expert discovery as of late 2026. A coalition of magazine and newspaper publishers including Condé Nast, The Atlantic and Forbes sued the enterprise AI company Cohere in early 2025; in November 2025 Judge Colleen McMahon largely denied Cohere's motion to dismiss, allowing claims based both on training and on outputs, including trademark claims arising from articles the system allegedly fabricated and attributed to the publishers. The news cases stand out for the importance they place on retrieval and on crawler behaviour. A model trained on an article in 2021 and a system that fetches the same article when a user asks about it today present different provenance questions. For the first, the issues are whether the article was in the training set and whether the model can regurgitate it. For the second, they are whether the system's crawler honoured the publisher's robots.txt, whether it went behind a paywall, how much of the article it displayed, and whether it diverted traffic. Several publishers allege that answer engines used undisclosed crawlers or third-party scraping services to reach content they had blocked, which turns the question of crawler identity into a litigation issue. Music The music industry has pursued AI developers on two fronts. In October 2023, music publishers led by Universal Music Publishing Group, Concord and ABKCO sued Anthropic, alleging that its Claude models had been trained on song lyrics and could reproduce them on request. In January 2025 the parties agreed a stipulation under which Anthropic would maintain guardrails against outputting lyrics, and in March 2025 the court denied the publishers' request for a preliminary injunction. The case continues in the Northern District of California, alongside a second action by the same publishers and a separate suit by BMG filed in 2026. In June 2024, the three major record companies, Universal, Sony and Warner, sued the AI music generators Suno and Udio, alleging that they had trained on vast numbers of copyrighted recordings. The defendants did not deny training on recordings available on the open internet but argued fair use. The record companies then began to settle. Universal settled with Udio in October 2025 and announced a joint licensed platform. Warner settled with Udio and then with Suno in November 2025, again pairing settlement with licensing arrangements. As of late 2026, Universal's case against Suno and Sony's cases against both companies continued. In Germany, the collecting society GEMA has won two significant judgments in the Munich Regional Court. In November 2025, in GEMA v. OpenAI, the court held that OpenAI had infringed by memorising and reproducing the lyrics of nine songs, finding that the lyrics were reproduced in the model itself and that the text and data mining exception did not cover that reproduction. That judgment is under appeal to the Munich Higher Regional Court. In July 2026, in GEMA v. Suno, the same court largely granted GEMA's claims concerning six well-known works. It asserted jurisdiction over training conducted in the United States, applied American fair use law to that training and found the defence unavailable because simple prompts produced outputs substantially similar to the originals, held that the works were memorised in model versions hosted on German servers, and held Suno rather than its users liable for outputs generated in Germany. It also held that compliance with the EU AI Act's copyright obligations did not by itself shield a provider from copyright liability. The Suno judgment was not final as of November 2026. Chapter 5 returns to the German cases. Music cases illustrate the particular importance of output similarity. A song is short and highly recognisable; a model that generates a track with the same melody and a closely similar vocal style, or that reproduces a full set of lyrics, produces evidence of memorisation that a court can hear or read. That makes music an unusually hard medium in which to defend the claim that a model learns only general patterns. Visual art and images Visual artists filed one of the first generative AI suits, Andersen v. Stability AI, in January 2023, against Stability AI, Midjourney, DeviantArt and later Runway. Judge William Orrick dismissed several claims in 2023 but in August 2024 allowed the core copyright claims to proceed, including the theory that the models themselves might contain compressed copies of training images. The case has moved slowly; in early 2026 the court extended the schedule, and a summary judgment hearing was set for February 2027. Getty Images has litigated against Stability AI on two continents. In England, the High Court ruled in November 2025 after a trial in which Getty had dropped its primary infringement claims because it could not show that training had taken place in the United Kingdom. Mrs Justice Joanna Smith rejected the remaining secondary infringement claim, holding that Stable Diffusion was not an "infringing copy" because the model did not store Getty's images, while finding limited trademark infringement in respect of watermarks generated by early model versions. Getty's appeal on the secondary infringement point began in the Court of Appeal on 11 November 2026. In the United States, Getty dismissed its Delaware action in August 2025 and refiled in the Northern District of California; in April 2026 Judge Trina Thompson largely denied Stability's motion to dismiss, allowing copyright, trademark, unfair competition and dilution claims to proceed. The major film studios entered the field in June 2025, when Disney and Universal sued Midjourney in Los Angeles, alleging that it had trained on their libraries and generated images of characters such as Darth Vader, Elsa and the Minions, and that it had ignored requests to adopt safeguards. Warner Bros. Discovery filed a similar suit in September 2025. These cases emphasise output: the studios' complaints are full of generated images of their characters, and their theory does not depend on any particular finding about training. They also show how output safeguards, such as filters that refuse prompts naming protected characters, have become a litigation issue in their own right. Books after Bartz The Bartz settlement did not end book litigation. Because its release covered only past conduct and excluded output claims, and because it applied only to Anthropic, authors and publishers continued to pursue other developers and new theories. New suits filed in 2025 and 2026 included actions against Anthropic, Meta and others on behalf of authors and publishers outside the Bartz class, including one led by the journalist John Carreyrou; jury trials in the cases against Meta are set for May 2027. In the consolidated OpenAI litigation, author classes continue to press claims alongside the news plaintiffs. Publishers have also sued operators of shadow libraries directly; in October 2026 a group of academic publishers sought a $19.5 million default judgment against one such operator in New York. The book cases have become, above all, acquisition cases. Since Bartz, the question plaintiffs ask first is not "did you train on my book?" but "where did you get it?" Discovery has focused on download logs, torrent client records, internal discussions about the legal risks of shadow libraries, and the provenance of datasets obtained from third parties. Platforms and scraping A final category of litigation pits platforms against scrapers, and concerns access rather than copyright in particular works. Reddit sued Anthropic in June 2025 in California state court, alleging that Anthropic's crawlers had accessed Reddit more than a hundred thousand times after the company said it had stopped, in breach of Reddit's user agreement. In October 2025 Reddit sued Perplexity together with the search-scraping services SerpApi, Oxylabs and AWMProxy, alleging that they circumvented Google's anti-bot system, SearchGuard, to harvest Reddit content from Google's search results. Google itself sued SerpApi in December 2025 on similar anti-circumvention grounds. The two anti-circumvention cases have produced conflicting early rulings, discussed in Chapter 8. In July 2026, Chief Judge Yvonne Gonzalez Rogers in the Northern District of California dismissed Google's anti-circumvention claims against SerpApi, finding that Google had not adequately pleaded that SearchGuard controlled access to copyrighted works with the authority of their owners. In the same period, Judge Paul Engelmayer in the Southern District of New York allowed Reddit's anti-circumvention claim against Perplexity and SerpApi to proceed, holding at the pleading stage that SearchGuard qualified as an access control. These are not merits decisions, but they establish that circumventing technical measures to reach content is a distinct legal risk for anyone building a collection pipeline. From litigation to licensing The most consequential development of 2025 and 2026 may not be any judgment but the steady conversion of disputes into commercial arrangements. Licensing of training and display rights predates most of the litigation: the Associated Press licensed its archive to OpenAI in July 2023, Axel Springer followed in December 2023, Reddit agreed a deal with Google in early 2024 reported at around $60 million a year, and News Corp agreed a five-year deal with OpenAI in May 2024 that was reported to be worth more than $250 million. Since then the pattern has spread across media: Microsoft licensed non-fiction titles from HarperCollins; Amazon licensed New York Times content; image libraries and stock agencies sold training access; and music companies converted lawsuits against Suno and Udio into licensed platforms. Table 2 summarises the status of the principal disputes discussed in this chapter and the previous one as of mid-November 2026. Table 2. Status of selected AI copyright disputes as of mid-November 2026. Case Forum Main issue Status Thomson Reuters v. Ross US Court of Appeals, Third Circuit Training a competing non-generative tool on headnotes Not fair use; affirmed 29 September 2026 Bartz v. Anthropic US District Court, N.D. California Training on books; pirated library Training held fair use (June 2025); $1.5bn settlement approved 20 July 2026 Kadrey v. Meta US District Court, N.D. California Training on books from shadow libraries Summary judgment for Meta on training (June 2025); distribution claims pending, hearing February 2027 In re OpenAI (incl. NYT v. Microsoft and OpenAI) US District Court, S.D. New York Training, retrieval and outputs using news and books Cross-motions for summary judgment filed September 2026; pending Getty Images v. Stability AI High Court and Court of Appeal, England and Wales Whether an imported model is an infringing copy Claim failed November 2025; appeal heard from 11 November 2026 GEMA v. OpenAI Regional Court, Munich Memorised song lyrics Infringement found November 2025; on appeal GEMA v. Suno Regional Court, Munich Training and outputs for music Infringement found July 2026; not final ANI v. OpenAI Delhi High Court Training and retrieval using news Interim injunction refused 24 July 2026; suit continues Like Company v. Google Court of Justice of the European Union Chatbot reuse of press content; training Advocate General's opinion 3 November 2026; judgment awaited Settlements and licences have several effects on the provenance landscape. They establish that a market for training data exists, which strengthens plaintiffs' market-harm arguments in remaining cases, as the Third Circuit's recognition of a training-data licensing market illustrates. They create contractual obligations, such as using licensed content only for specified models or purposes, or removing it on termination, that can only be honoured by developers that can track which data went into which models. And they shift the burden of provenance onto the licensor as well: a publisher selling its archive must be able to show that it holds the rights to what it sells, including freelancers' contributions and syndicated content it does not own. Critics of the licensing turn raise two concerns. The first is that licensing at scale is available mainly to large developers and large rights holders, entrenching incumbents on both sides and leaving individual creators, small publishers and new entrants out of the market. The second is that licensing deals, by implying that permission is needed, may undermine fair use arguments that would otherwise benefit researchers, smaller developers and the public. Defenders respond that licensing is simply how markets for valuable inputs ordinarily work, and that collective licensing schemes, of the kind several countries are exploring, can extend the market to smaller rights holders. Whatever view one takes, licensing changes the nature of the provenance problem. Under a pure fair use regime, provenance records serve mainly to defend acquisition choices and demonstrate safeguards. Under a licensing regime, they become part of the commercial bargain itself, the evidence that each party has kept its side of the deal. Chapter 5. Beyond Fair Use: Europe, the United Kingdom and Asia Outside the United States, the law of AI training is mostly written in statutes rather than in an open-ended defence. Many jurisdictions have adopted specific exceptions for text and data mining, the automated analysis of works to extract information. Those exceptions are narrower than fair use in some respects and more predictable in others, and several of them make the rights holder's own choices, expressed through machine-readable signals, part of the legal test. That design makes provenance a statutory requirement rather than merely good evidence. This chapter surveys the European Union, Germany, the United Kingdom, Japan, Singapore, India and China, and closes with the problem that cuts across all of them: territoriality. The European Union: exceptions with an opt-out The EU's framework rests on two articles of the 2019 Directive on Copyright in the Digital Single Market. Article 3 permits research organisations and cultural heritage institutions to carry out text and data mining for scientific research on works to which they have lawful access, and this exception cannot be overridden by contract or reservation. Article 4 permits anyone, including commercial companies, to reproduce and extract lawfully accessible works for text and data mining, but only if the rights holder has not expressly reserved that use "in an appropriate manner, such as machine-readable means in the case of content made publicly available online." Copies made under Article 4 may be retained only as long as necessary for the mining. When the directive was drafted, generative AI was not its main concern, and for some years it was disputed whether training a generative model counted as text and data mining at all. That debate has largely been settled by legislation. The EU's Artificial Intelligence Act, adopted in 2024, expressly links general-purpose AI models to the Article 4 framework. Article 53 requires every provider of a general-purpose AI model placed on the EU market to put in place a policy to comply with EU copyright law, "in particular to identify and comply with" reservations of rights expressed under Article 4(3), and to draw up and make publicly available a "sufficiently detailed summary" of the content used for training, according to a template provided by the AI Office. The Commission published that template in July 2025. Recital 106 of the Act states that these obligations apply to any provider placing a model on the EU market regardless of the jurisdiction in which the training took place, an assertion of extraterritorial reach that has been criticised but not yet tested in court. The general-purpose AI obligations applied from 2 August 2025. To help providers comply, the Commission facilitated a voluntary General-Purpose AI Code of Practice, published in July 2025, whose copyright chapter sets out concrete measures. Signatories commit to maintain a written copyright policy; to refrain from circumventing effective technological measures such as paywalls when crawling; to exclude websites that courts or public authorities in the EU have identified as persistently infringing copyright on a commercial scale; to use crawlers that read and follow robots.txt in accordance with the Internet Engineering Task Force's RFC 9309 and to respect other appropriate machine-readable reservations; to implement proportionate safeguards against infringing outputs and prohibit infringing uses in their terms; and to designate a contact point and complaint mechanism for rights holders. Most major Western developers signed the code, though Meta declined. Since 2 August 2026 the Commission, acting through the AI Office, has had power to enforce the general-purpose AI obligations, including by requesting documentation, evaluating models, requiring corrective measures and imposing fines of up to three percent of worldwide annual turnover or fifteen million euros, whichever is higher. Models placed on the market before August 2025 have until August 2027 to comply. A separate legislative package adopted in 2026 to simplify parts of the AI Act postponed several high-risk requirements but left the general-purpose AI timetable intact. The practical consequence is that, for any developer serving European users, a documented compliance policy, a working opt-out system and a training data summary are now enforceable legal obligations. The Like Company opinion The first case on generative AI to reach the Court of Justice of the European Union arose from a dispute between a Hungarian news publisher, Like Company, and Google over Google's Gemini chatbot, formerly Bard. Like Company alleged that Gemini reproduced and summarised its articles in answers to users. A Budapest court referred a series of questions to the Court of Justice in April 2025, asking whether a chatbot's display of content that partly matches a press article is a communication to the public, whether training a model involves reproduction, whether such reproduction falls within the text and data mining exception, and whether the chatbot provider reproduces content when it outputs all or part of a press article in response to a user's instruction. Advocate General Maciej Szpunar delivered his opinion on 3 November 2026. Advocate General opinions are not binding, but the Court follows them in a majority of cases, and this one is the most detailed judicial analysis of generative AI and copyright in European law to date. According to published summaries, the Advocate General concluded that a chatbot which retrieves a publisher's article in real time, reuses it to generate an answer and presents that answer to the user may, in certain circumstances, infringe the press publishers' right; that outputs reproducing protected works can infringe in their own right, assessed case by case; and that the text and data mining exception does not extend to outputs. On training, he advised the Court that the referred questions were inadmissible, so that the Court may never decide them, but set out his views for completeness. He considered that training involves acts of reproduction; that the temporary copies exception might cover transient copies made during training but not the stored dataset; that training is in principle text and data mining and is not excluded by the international "three-step test"; that the press publishers' right does not cover training, because it is limited to online uses; and that a trained model is not itself a database. Most significantly for this booklet, the Advocate General suggested that the text and data mining exception should cover training only if three conditions are met. The developer must have lawful access, which excludes pirate libraries and shadow databases and the circumvention of paywalls or bot blocks. The rights holder must not have reserved its rights in a suitable machine-readable form, and he treated robots.txt as a valid method despite its limitations. And the developer must show that it took measures, such as deduplication and output filtering, to prevent the model from producing outputs that closely resemble specific training works; absent such measures, national courts should refuse the exception. He also indicated that EU law may apply to the scraping of works from servers located in the EU even if training takes place elsewhere. If the Court adopts this framework, every element of the exception will be a provenance question: where the data came from, what signals accompanied it, and what the developer did to prevent memorisation. Even if it does not, the opinion is likely to influence national courts and the AI Office. Germany: LAION, GEMA and the meaning of machine-readable German courts have produced more AI copyright judgments than any other European system. The first concerned LAION itself. A photographer, Robert Kneschke, sued LAION for downloading one of his images from a stock photography site whose terms prohibited automated collection, in order to compile the LAION-5B dataset. In September 2024 the Hamburg Regional Court dismissed the claim, holding that LAION's compilation fell within the scientific research exception in Section 60d of the German Copyright Act, which implements Article 3. In December 2025 the Hanseatic Higher Regional Court in Hamburg upheld the dismissal. It agreed that LAION was a non-commercial research organisation, but went further and held that the general text and data mining exception in Section 44b also applied, that it covers preparatory acts such as compiling datasets, and that it extends to the training of generative AI. Critically, it held that the plain-language reservation in the stock site's terms of use was not "machine-readable" for a download made in 2021. It applied a dynamic, technology-neutral standard, leaving open that natural-language reservations might become machine-readable once tools can reliably interpret them. The court granted leave to appeal to the Federal Court of Justice, so a final German ruling, or a further referral to the Court of Justice, is likely. The GEMA cases, introduced in Chapter 4, approach the question from the output side. In GEMA v. OpenAI, decided in November 2025, the Munich Regional Court held that song lyrics were reproduced in OpenAI's models because the models had memorised them and could output them in response to simple prompts. It held that the text and data mining exception covers the analysis of works but not a model that stores and reproduces them, and that responsibility for the outputs lay with OpenAI. In GEMA v. Suno, in July 2026, the same court extended that reasoning to music, held that memorisation in model versions hosted on German servers was an infringing reproduction outside the exception, and found jurisdiction over training conducted in the United States under a provision of German collecting society law. Taken with the Hamburg decisions, the German case law suggests a working distinction: collecting and analysing works for training can be lawful if lawful access and opt-out conditions are met, but a model that memorises and reproduces particular works falls outside that protection. The United Kingdom: a policy in suspension The United Kingdom's text and data mining exception, in Section 29A of the Copyright, Designs and Patents Act 1988, covers only non-commercial research. Commercial AI training on copyrighted works without a licence therefore has no statutory exception in the UK, and the country's fair dealing defences are narrower than American fair use. In December 2024 the government consulted on introducing a broad commercial exception with an opt-out, modelled loosely on the EU approach, and indicated that this was its preferred option. The proposal provoked an intense campaign by the creative industries and a series of parliamentary defeats in the House of Lords during the passage of the Data (Use and Access) Bill, as peers led by Baroness Beeban Kidron sought to add transparency obligations for AI developers. The Act, as passed in June 2025, instead required the government to publish an economic impact assessment and a report on copyright and AI. The government's report, published in March 2026, stated that the broad exception with an opt-out was no longer its preferred way forward, without naming a replacement. It declined to introduce mandatory transparency obligations, noting the split between creative industries, which strongly favoured them, and technology companies, which preferred industry-led commitments. It committed to gathering further evidence and monitoring developments abroad, including the outcome of the Getty Images appeal. Commentators of every persuasion described the result as a postponement. For practical purposes, commercial developers training in the UK need a licence or a defensible argument that no protected expression was reproduced, while the Getty appeal will determine whether models trained abroad and imported into the UK can be infringing copies. Japan, Singapore and other permissive regimes Japan has the most permissive statutory regime among major economies. Article 30-4 of its Copyright Act, introduced in 2018, permits the exploitation of works for purposes that do not involve "enjoying" the thoughts or sentiments they express, including data analysis, to the extent necessary, unless the use would unreasonably prejudice the interests of the copyright owner. In March 2024 the Agency for Cultural Affairs published guidance on how this applies to generative AI. It indicated that training generally falls within the provision, but that the exception may not apply where a purpose of the training is to enable outputs that reproduce the creative expression of particular works, as with training designed to imitate a specific work, and that the proviso may be engaged where, for example, a developer reproduces a database that the rights holder offers for AI training while circumventing technical measures intended to prevent such reproduction. Japanese law thus combines a broad permission with limits that again turn on purpose, output and the developer's respect for technical signals. Singapore's Copyright Act 2021 includes an exception for computational data analysis that applies to commercial and non-commercial uses alike, requires lawful access to the source material, and cannot be overridden by contract. Israel's Ministry of Justice issued an opinion in 2022 that training on copyrighted works would generally be fair use under Israeli law. Several other jurisdictions, including Hong Kong, have consulted on new exceptions. India and China India has no text and data mining exception, and the first significant Indian decision arose in a suit by the news agency ANI against OpenAI, filed in the Delhi High Court in November 2024. On 24 July 2026, Justice Amit Bansal refused ANI's application for an interim injunction. At this preliminary stage the court held that the suit could proceed in Delhi, but found that storing ANI's works to train OpenAI's models fell prima facie within the fair dealing provision in Section 52(1)(a) of the Copyright Act, that outputs generated through retrieval were not substantially similar to ANI's works, and that ANI had not shown memorisation or regurgitation. The court also weighed the harm an injunction would cause to the public. The findings are provisional and the suit continues. China regulates generative AI through administrative measures as well as copyright law. The Interim Measures for the Management of Generative Artificial Intelligence Services, in force since August 2023, require providers to use training data from lawful sources and not to infringe intellectual property rights. In February 2024 the Guangzhou Internet Court held a generative AI service provider liable for outputs that reproduced the Ultraman character, focusing on the provider's failure to adopt reasonable safeguards rather than on training. Chinese courts have so far concentrated on outputs and on the duty of care owed by service providers. Comparing the regimes, and the problem of territory Table 3 compares the principal approaches. Table 3. How selected jurisdictions treat AI training on copyrighted works, as of November 2026. Jurisdiction Main legal basis for unlicensed training Role of opt-out signals Key recent decision or instrument United States Fair use, decided case by case Not a statutory element; pleaded as evidence Bartz and Kadrey (2025); Thomson Reuters v. Ross affirmed (September 2026) European Union Text and data mining exception, Article 4 DSM Directive Central: valid reservation removes the exception AI Act Article 53; Like Company opinion (November 2026) Germany Sections 44b and 60d Copyright Act Must be machine-readable; plain text insufficient for 2021 Kneschke v. LAION (December 2025); GEMA v. OpenAI and v. Suno United Kingdom Non-commercial research exception only No statutory role; reform shelved Government report (March 2026); Getty appeal pending Japan Article 30-4 non-enjoyment exception Relevant to the unreasonable-prejudice proviso Agency for Cultural Affairs guidance (March 2024) Singapore Computational data analysis exception Cannot override by contract; lawful access required Copyright Act 2021 India Fair dealing, Section 52 Not addressed in law ANI v. OpenAI interim ruling (July 2026) The comparison reveals a striking convergence beneath the differences. Every regime that permits training does so on conditions that resolve into the same factual questions: was access lawful, did the developer respect the rights holder's expressed refusal, does the model reproduce particular works, and do the outputs substitute for them. The regimes differ in which of these questions is decisive and who bears the burden of answering it. They differ more sharply on territory. Copyright is territorial: each country's law governs acts done within it. Training is usually conducted in a small number of data centres, often in the United States, while models are deployed worldwide. Getty's English case lost its primary claim because training occurred outside the UK. The EU's AI Act asserts that its copyright obligations apply regardless of where training took place. The Munich court in GEMA v. Suno applied American fair use law to training in the United States and found it wanting. Advocate General Szpunar suggested that EU law may apply to scraping from EU servers. For developers, the consequence is that the most demanding regime in any market they serve effectively sets the standard for their global pipeline, because a single model cannot easily be trained one way for Europe and another way for everywhere else. That is a strong practical argument for building collection systems to the strictest applicable standard: lawful access, honoured opt-outs, documented provenance and memorisation controls. Chapter 6. What the Weights Remember The single most contested factual question in AI copyright is what a trained model retains about the works it learned from. If models retain nothing but general patterns, the case that training is a non-expressive, transformative use is strong, and the claim that a model is itself an infringing copy is weak. If models retain and can reproduce substantial portions of particular works, both arguments weaken. Courts in England and Germany reached opposite conclusions in November 2025 about whether a model can be a copy, largely because they were working from different facts. This chapter sets out what the research literature has established about memorisation, why the answer differs across models, works and media, and how memorisation can be measured and reduced. Why models cannot store everything, and why they store something A generative model is trained by repeatedly showing it examples and adjusting its parameters to make its predictions more accurate. A language model learns to predict the next token, a word or fragment of a word, given the preceding text. An image diffusion model learns to remove noise from images, conditioned on a text description. In both cases, the trained model consists of a fixed set of numerical parameters, its weights, whose number is set before training begins and does not grow with the amount of data. That fact supports a simple and widely used argument. A large language model might have seventy billion parameters, which at standard precision occupy roughly 140 gigabytes. It might be trained on fifteen trillion tokens of text, which in raw form occupy tens of terabytes. The model is far too small to contain a copy of its training data, even under aggressive compression. Stable Diffusion's early versions had around a billion parameters, a few gigabytes, and were trained on two billion or more images; that works out to a few bytes per image. Whatever the model has learned, it cannot be a compressed archive of its training set. But the argument proves less than it seems. It shows that a model cannot store most of its training data. It does not show that a model cannot store some of it. Training does not allocate capacity evenly. A passage that appears thousands of times across the web, such as a famous poem, the opening of a well-known novel, a popular song lyric, a standard software licence or a widely reposted news story, exerts far more influence on the weights than a passage that appears once. Models also have more capacity to spare as they grow. The research consensus, developed over several years, is that large models memorise a small but non-trivial fraction of their training data, that what they memorise is heavily skewed toward duplicated and distinctive content, and that the fraction grows with model size. How researchers define and measure memorisation The literature distinguishes several kinds of memorisation, and the differences matter for law. Extractable memorisation, studied by Nicholas Carlini and colleagues in a series of papers beginning in 2021, asks whether an adversary with access to the model, but not to the training data, can construct a prompt that causes the model to emit a training example verbatim. Their 2021 study of GPT-2 recovered hundreds of verbatim sequences, including personal contact details and code, by generating large amounts of text and ranking it by how confident the model was. In 2023, Milad Nasr and colleagues showed that a simple attack, asking ChatGPT to repeat a single word indefinitely, caused the model to "diverge" from its chat behaviour and emit memorised training data at a much higher rate; with a modest budget they extracted more than ten thousand unique memorised examples. Discoverable memorisation asks a narrower question: if the model is given the true prefix of a training example, does it complete it with the true suffix? This is easier to measure, because the researcher can sample examples from the known training set and test them. A 2022 study by Carlini and colleagues, "Quantifying Memorization Across Neural Language Models," found three regularities. Memorisation increases log-linearly with model size, so that larger models memorise more. It increases with the number of times an example is duplicated in the training data. And it increases with the length of the prompt: the more context the model is given, the more likely it is to reproduce what follows. Approximate memorisation recognises that copyright does not require verbatim copying. Daphne Ippolito and colleagues showed in 2022 that filters which block exact verbatim reproduction can be evaded by outputs that differ trivially, for example by changes in capitalisation or spacing, and that models often reproduce training text with small variations. A measure of memorisation that counts only exact matches will understate what a copyright lawyer would regard as reproduction. Counterfactual memorisation asks how much a model's behaviour on a particular example depends on that example having been in the training set, as opposed to the model having learned general patterns that let it predict the example anyway. A model may complete a common phrase correctly without having memorised any particular source. This distinction matters when a defendant argues that apparent reproduction reflects generic language rather than copying. What the evidence shows Four bodies of evidence are now prominent in litigation. First, deduplication matters enormously. Studies by Katherine Lee and colleagues in 2021 and Nikhil Kandpal and colleagues in 2022 found that training data deduplication substantially reduces the rate at which models emit memorised text, and that the risk of a sequence being regenerated rises sharply with the number of times it appears in the training data. This is both a scientific finding and a practical lever: a developer that deduplicates aggressively, at the level of documents and of long repeated passages, will produce a model that memorises less. Second, memorisation of books varies widely. A 2025 study by A. Feder Cooper, Mark Lemley and colleagues tested several open-weight language models for their ability to reproduce passages from books in the Books3 dataset. They found that most models memorised little of most books, but that some large models had memorised substantial portions of a few very popular works; one model could reproduce large parts of the first Harry Potter novel and of George Orwell's Nineteen Eighty-Four with high probability. The variation was extreme across both books and models. The authors noted that this cuts both ways legally: it undermines the claim that models never contain copies, but also the claim that every model contains copies of every book it was trained on. In Kadrey v. Meta, by contrast, the record showed that the Llama models at issue could not be prompted to reproduce meaningful amounts of the plaintiffs' books, which contributed to the court's rejection of the regurgitation theory. Third, image models memorise too, at lower rates. A 2023 study by Carlini, Jamie Hayes and colleagues extracted training images from Stable Diffusion and Google's Imagen, focusing on images that were heavily duplicated in the training data; the extracted images were near-identical to originals, including identifiable photographs of people. Gowthami Somepalli and colleagues showed in 2023 that diffusion models sometimes replicate substantial elements of training images in generated outputs. But the rate of exact replication for typical prompts is low, and the Getty case illustrates a subtler phenomenon. Early Stable Diffusion models sometimes generated images bearing distorted versions of Getty's watermark, because the watermark appeared on so many training images. The model had learned the watermark as a pattern, without storing any particular Getty photograph, which is why the High Court found trademark infringement for some early model versions while accepting that the model did not contain copies of Getty's images. Fourth, deployment-time evidence differs from attack-time evidence. In the OpenAI litigation, OpenAI pointed to twenty-four instances of verbatim regurgitation in a sample of twenty million ChatGPT logs. The news plaintiffs, like the Times in its original complaint, relied on examples produced by prompting the models with the opening of articles. These are answers to different questions. The first asks how often ordinary users receive reproductions in practice, which bears on market substitution. The second asks whether the model has retained the work in a form that can be retrieved, which bears on whether the model is a copy and on the risk that a determined user can obtain the work. Courts have not yet decided which question matters more, and the answer may differ between the training, model and output stages identified in Chapter 2. Is a model a copy? The English and German courts that ruled in November 2025 can be reconciled through these distinctions. In Getty Images v. Stability AI, Getty accepted at trial that the Stable Diffusion weights did not store its images, and the court held that an "infringing copy" under English law must contain or have contained a copy of the protected work. A model that had learned patterns from images but did not store them was not such a copy. In GEMA v. OpenAI, the evidence showed that the models could reproduce the full lyrics of the songs in suit in response to simple prompts. The Munich court reasoned, as many technologists would, that information which can be reliably reconstructed from a file is in some sense stored in it, just as a compressed archive contains the documents it can be unpacked into even though no human could read them in compressed form. On that reasoning, the model reproduced the lyrics, at least for those works. Both courts therefore focused on the same factual question: can this work be reconstructed from this model? They gave different answers because the facts were different. That suggests the legal status of a model as a copy is not a single question but a work-by-work and model-by-model one. A model may be a copy of a famous song lyric it has memorised while not being a copy of a million obscure blog posts it learned from. The Getty appeal, which began in the Court of Appeal on 11 November 2026, will address the English statutory question, but the factual distinction is likely to persist whichever way it is decided. For developers, the consequence is that memorisation is not merely a safety or privacy concern. It is the hinge on which the model's own legal status turns. Advocate General Szpunar's suggestion that the EU text and data mining exception should apply only where developers have taken measures against memorisation, discussed in Chapter 5, would make memorisation controls a condition of the exception itself. Reducing memorisation Developers have a range of tools for reducing memorisation, and their effectiveness is increasingly a matter of record. Deduplication is the first and most effective. Removing exact duplicate documents is standard; removing near-duplicates using techniques such as MinHash, which estimates the similarity of documents by comparing samples of their word sequences, catches reposted articles and lightly edited copies. Removing long repeated substrings across documents, rather than only whole documents, addresses passages such as song lyrics or famous quotations that appear embedded in many different pages. Targeted filtering removes or down-weights specific categories of content known to be both heavily duplicated and legally sensitive: lyrics sites, full-text book repositories, paywalled news reproduced on aggregator sites. This requires the developer to know what is in its corpus at the level of sources and domains, which is a provenance capability. Training-time techniques can reduce memorisation directly. Training for fewer passes over the same data reduces memorisation. Differential privacy, which adds calibrated noise to training updates to bound how much any single example can influence the model, offers formal guarantees but at a significant cost in quality for large models, and is rarely used for pretraining at scale. Researchers have proposed lighter-weight methods, such as the "goldfish loss" introduced by Abhimanyu Hans and colleagues in 2024, which excludes a pseudo-random subset of tokens from the training objective so that the model never learns to reproduce long sequences verbatim. Output-time safeguards catch reproduction that training-time measures miss. GitHub introduced a filter for its Copilot coding assistant that compares suggestions, together with surrounding code of about 150 characters, against public code and suppresses matches. Developers of language models have deployed filters that block the output of song lyrics and long passages of known copyrighted text; the Concord stipulation in January 2025 committed Anthropic to maintaining such guardrails. Image generators filter prompts naming protected characters or living artists. These safeguards depend on a reference index of protected content against which outputs can be checked, which again requires provenance infrastructure: hashes, fingerprints or n-gram indexes of the works the developer wants to avoid reproducing. Post-training alignment, the reinforcement learning and instruction tuning that turn a pretrained model into an assistant, can train a model to refuse requests for verbatim copyrighted text. But as the divergence attack against ChatGPT showed, alignment can suppress memorised content without removing it, and adversarial prompting can sometimes bring it back. Memorisation as a provenance problem Every one of these measures connects to provenance. A developer cannot measure memorisation of a work unless it knows the work was in the training set and retains a reference copy to test against. It cannot deduplicate effectively without content hashes and similarity indexes across the corpus. It cannot filter sensitive sources without knowing which documents came from which domains. And it cannot demonstrate to a court or regulator that it took measures against memorisation unless it recorded what it did, when, and with what effect. This is the first point at which the technical and legal halves of the problem meet directly. The question "does the model contain this work?" can only be answered by testing the model against the work, and the question "did you take reasonable steps to prevent memorisation?" can only be answered by records. Provenance infrastructure, in other words, is also memorisation infrastructure. The next chapter asks how far it is possible to go further, and trace a model's behaviour back to the specific sources that produced it. Chapter 7. Tracing Weights Back to Sources If a model's behaviour could be traced reliably back to the specific works that produced it, many of the hardest problems in AI copyright would become tractable. Rights holders could prove their works were used. Developers could show they were not. Licensing revenue could be shared according to actual contribution. Problematic sources could be identified and removed. The aspiration is often expressed as a kind of audit trail running backwards from output to input, and it has driven a substantial body of research. This chapter examines what that research can and cannot deliver. The short answer is that tracing a trained model back to its sources after the fact is possible only in limited ways, with significant uncertainty, and that the reliable route to provenance runs forward, through records made at the time of collection and training. Two different questions It helps to separate two questions that are often run together. The first is membership: was a particular work, or a particular collection of works, in the training data of a particular model? This is the question a rights holder asks when deciding whether to sue, and the question a regulator asks when checking a training data summary. It is binary for any single item, although evidence about it may be probabilistic. The second is attribution: how much did a particular training item contribute to a particular output or to the model's behaviour in general? This is the question that matters for revenue sharing, for identifying which sources caused a harmful output, and for some theories of market harm. It is a matter of degree, and in a model trained on trillions of tokens the contribution of any single item to any single output is usually tiny and diffuse. Neither question is answered by inspecting the weights directly. Weights are not an index of training documents. They encode the cumulative effect of billions of small updates, each influenced by many examples at once. Every method discussed below works indirectly, either by probing the model's behaviour or by approximating the counterfactual effect of removing data. Membership inference and its limits Membership inference attacks try to determine whether a given example was in a model's training set by observing how the model treats it. The intuition, set out in work by Reza Shokri and colleagues in 2017, is that models tend to be more confident on data they were trained on than on data they were not. For a language model, the simplest signal is the loss or perplexity the model assigns to a text: a document the model has seen should, on average, look less surprising to it. More refined methods compare a text's score against reference models trained without it, or, like the Min-K% Prob method proposed by Weijia Shi and colleagues in 2024, focus on the least likely tokens in a passage, which are thought to be more revealing than the average. For large language models, membership inference has proved much less reliable than early results suggested. A 2024 study by Michael Duan and colleagues evaluated a range of attacks against models trained on the Pile, a corpus whose contents are publicly known, and found that most attacks performed barely better than random guessing at distinguishing training documents from documents drawn from the same distribution but held out. Their explanation was that large models trained for a single pass over enormous datasets do not overfit individual documents much, and that the boundary between "seen" and "unseen" text is blurred by the overlap of language across documents. Several studies in 2024 also showed that many published successes were artefacts of distribution shift: when researchers compared training documents with "non-member" documents written later, the attacks were partly detecting differences in date and topic rather than membership. A simple classifier that never looked at the model at all could match or beat some published attacks on popular benchmarks. The consequence for litigation is significant. A single-document membership inference test is generally not reliable evidence that a particular article or book was in a large model's training data, and expert witnesses relying on it can expect to be challenged. The exception is where the model has memorised the work strongly enough that it can be extracted, as discussed in the previous chapter; extraction is much stronger evidence than a statistical membership score, because an exact or near-exact reproduction of a long passage is very unlikely to arise by chance. Dataset inference A more promising approach shifts from single documents to collections. Dataset inference, developed for language models by Pratyush Maini and colleagues in 2024, combines many weak membership signals across a set of documents from a single source and applies a statistical test to decide whether the set as a whole was used in training. The method requires the claimant to hold a comparable set of documents that were definitely not used, for example unpublished material from the same author or publisher, written in the same style and period, so that the comparison is not confounded by distribution shift. Under those conditions the authors reported that dataset inference could distinguish training sets from held-out sets with high confidence and few false positives. Dataset inference suits the position of large rights holders, such as a news organisation with millions of articles or a publisher with thousands of books, who want to know whether their archive was used. It is less useful for individual creators with small bodies of work, and it depends on the availability of a clean comparison set. It also answers only the membership question, not attribution. Planting evidence in advance: canaries, traps and watermarks If detection after the fact is difficult, rights holders can try to make it easier in advance by placing distinctive markers in their content before it is collected. The idea originates in privacy research. Carlini and colleagues proposed in 2019 to insert randomly generated "canary" sequences into training data and later measure how readily the model reproduces them, as a way of quantifying memorisation. Matthieu Meeus and colleagues adapted this in 2024 as "copyright traps": synthetic, unique sequences embedded in documents, invisibly or innocuously, so that their later reproduction by a model would show the documents were used. Their experiments, training a model of 1.3 billion parameters from scratch, found that short traps repeated a few times were not detectably memorised, but that longer sequences repeated many times, on the order of a thousand, could be detected reliably. That makes traps practical mainly for publishers that control large volumes of content and can repeat a trap across many pages. Johnny Wei and colleagues proposed in 2024 a related method of "data watermarks," such as inserting random character sequences or substituting visually identical Unicode characters, with a hypothesis test that gives a statistical guarantee on the false detection rate. Similar techniques, sometimes called radioactive data, have been studied for images. These methods have real limitations. Deduplication and quality filtering, which careful developers apply as a matter of course, may strip traps or downweight the pages that carry them. Unicode substitution can be normalised away. And markers placed today detect only collections made after they were placed. But they change the evidential position: a rights holder that has planted markers can make a far stronger claim than one relying on generic membership inference. There is an important asymmetry for developers. Dataset inference and traps can show that a collection was used; they cannot show that it was not. A developer seeking to demonstrate that it did not train on a rights holder's content must rely on its own records: the crawl logs, exclusion lists and dataset manifests that show the content was never collected or was removed before training. Attribution: influence functions and their successors The attribution question, how much a training example contributed to an output, has been studied under the heading of training data attribution. The most principled approach is to ask a counterfactual: how would the model's output change if this example had been removed from the training set and the model retrained? Retraining a large model once per training example is impossible, so researchers approximate the answer. Influence functions, introduced to modern machine learning by Pang Wei Koh and Percy Liang in 2017, use a mathematical approximation drawn from robust statistics to estimate the effect of up-weighting or removing a single training point, without retraining. The calculation requires information about the curvature of the model's loss function, which is prohibitively expensive for large models. In 2023 a team at Anthropic led by Roger Grosse scaled influence functions to language models with up to 52 billion parameters, using an approximation technique for the curvature. Their findings are instructive. For small models, the most influential training sequences for a given output often shared surface features such as overlapping words. For larger models, influential sequences were more often conceptually related, reflecting more abstract generalisation, and influence was spread thinly across many documents rather than concentrated in a few. Only in cases of memorisation did a single source dominate. Other methods trade accuracy for scale. TRAK, introduced by Sung Min Park and colleagues in 2023, uses random projections of model gradients to estimate attribution far more cheaply. Datamodels, from Andrew Ilyas and colleagues in 2022, train many models on random subsets of the data and fit a simple predictive model of how each training example affects outputs; this is more faithful to the counterfactual but requires many training runs. Data Shapley and its variants, drawing on cooperative game theory, attempt to assign each example a fair share of a model's value, but are expensive to compute exactly. Research on attribution for image generators, such as a 2023 study by Sheng-Yu Wang and colleagues, has built benchmarks by fine-tuning models on known exemplars so that ground-truth influence can be checked, an acknowledgement of how hard attribution is to validate on models trained from scratch. Researchers have also documented that attribution scores can be fragile. A 2021 study by Samyadeep Basu and colleagues found that influence estimates for deep networks are sensitive to architecture, regularisation and random initialisation, so that two training runs on the same data could yield different rankings of influential examples. A pragmatic alternative, widely used in practice, is similarity search: embedding outputs and training documents in a common vector space and retrieving the training documents most similar to a given output. This is cheap and useful for flagging possible reproduction, but it measures resemblance rather than causal contribution. An output that resembles a training document may have been produced because of it, or because of thousands of other documents saying similar things. Table 4 compares the main techniques. Table 4. Techniques for linking model behaviour to training data. Technique Question answered Access required Reliability for large generative models Practical use Extraction and regurgitation tests Is this specific work retrievable from the model? Query access to the model; copy of the work Strong evidence when long passages reproduce; silent when they do not Litigation evidence; memorisation audits Single-document membership inference Was this document in the training set? Query access; often model probabilities Weak; near chance in several 2024 evaluations Research; screening only Dataset inference Was this collection used? Model probabilities; clean held-out set from same source Promising for large collections Large rights holders assessing use Canaries, traps and data watermarks Was marked content used? Ability to mark content before collection Good if markers are long and repeated and survive filtering Publishers protecting future content Influence functions and related attribution How much did an example shape an output? Full model weights and training data Approximate, costly and sometimes unstable Research; debugging; possible revenue-sharing input Similarity search Which training items resemble this output? Training corpus index Measures resemblance, not causation Output review; takedown triage Unlearning: tracing in reverse The counterpart to tracing is removal. If a rights holder withdraws consent, a licence ends, or a court orders that particular material be removed, can a developer remove its effect from a trained model without retraining from scratch? Exact unlearning methods design training so that removal is cheap. The SISA approach, proposed by Lucas Bourtoule and colleagues in 2021, splits the training data into shards, trains separate model components on each, and combines them, so that removing an example requires retraining only the affected shard. This works for some model types but is difficult to apply to large monolithic generative models. Approximate unlearning methods instead adjust a trained model to make it behave as if it had not seen certain data, for example by training it to increase its loss on the material to be forgotten. Ronen Eldan and Mark Russinovich demonstrated in 2023 a method for making a model "forget" the Harry Potter books, after which it could no longer produce details of the series. Evaluations have been sobering. A 2024 benchmark, MUSE, developed by Weijia Shi and colleagues, tested several unlearning methods on news articles and books and found that most either failed to remove the targeted knowledge fully, as judged by memorisation and membership tests, or degraded the model's general capabilities, or both. Other work has shown that knowledge apparently removed can sometimes be recovered with light further training. Approximate unlearning is best understood as a way of suppressing a model's ability to reproduce or use particular content, comparable in effect to a strong output filter, rather than as a guarantee that the content's influence is gone. The legal significance of this is becoming clearer. The Bartz settlement required Anthropic to destroy the pirated files and their copies, but did not require it to destroy or retrain models trained on them. Licensing agreements increasingly include provisions about what happens to models trained on licensed data when the licence ends. And the EU framework, by tying the text and data mining exception to respect for opt-outs, raises the question of what a developer must do when a rights holder opts out after its content has already been used. The practical answer at present is that developers can exclude the content from future training runs, document that exclusion, and apply output-side safeguards to existing models. They cannot, with current techniques, verifiably remove it from a model already trained. Why provenance must run forward The lesson of this chapter is that the backward path from weights to sources is narrow and unreliable, except where memorisation is strong enough to make it obvious. A court asking whether a particular article was used to train a particular model will, in most cases, get its answer not from forensic analysis of the model but from discovery of the developer's records: the crawl logs, dataset manifests and training configurations. In the New York Times litigation, the parties' experts worked from training data that OpenAI made available for inspection under protective conditions. Where such records are incomplete, both sides are left arguing over probabilistic inferences. The forward path, by contrast, is straightforward in principle. If each item collected is recorded with its source, time, terms and signals; if each dataset is versioned with a manifest of its contents; and if each training run records which dataset versions it consumed, then the membership question can be answered by lookup, removal from future models can be implemented by filtering, and the developer can prove a negative as well as a positive. Attribution in the fine-grained sense will remain approximate, but membership, which is what most legal and regulatory questions actually turn on, becomes a matter of record rather than inference. There is also an architectural choice that makes attribution tractable. Systems that rely on retrieval-augmented generation fetch identifiable documents at query time, and can record and display which documents informed each answer. Their copying is more visible, which is why retrieval has attracted direct legal claims, but it is also far easier to license, attribute and pay for. Developers and publishers negotiating over attribution and revenue sharing are increasingly doing so around retrieval, where the link between source and output is direct, rather than around pretraining, where it is diffuse. The remaining chapters therefore turn from forensic tracing to the construction of systems that make tracing unnecessary: pipelines that respect consent signals at the point of collection, record what they collect, and carry that record forward into every dataset and model built from it. Chapter 8. Signals of Consent: Robots.txt, Opt-Outs and Licensing Protocols For most of the web's history, the relationship between websites and the automated programs that visit them was governed by a short text file and a gentleman's agreement. The robots exclusion protocol let site operators tell crawlers where not to go, and reputable crawlers complied. Generative AI has placed that arrangement under extraordinary strain, and in the process has turned it into something its creators never intended: a mechanism with legal consequences. Under the EU's text and data mining framework, a machine-readable reservation of rights determines whether an exception applies. Under the EU Code of Practice, signatories commit to honour robots.txt. In American litigation, a developer's response to blocking signals has become evidence of good or bad faith, and attempts to work around technical barriers have produced anti-circumvention claims. This chapter describes the signalling systems now in use, their weaknesses, and what their legal status means for anyone building a collection pipeline. The robots exclusion protocol The protocol dates from 1994, when the Dutch engineer Martijn Koster proposed it as a way for webmasters to keep early crawlers away from parts of their sites. A site places a file named robots.txt at its root. The file contains groups of rules, each introduced by a user-agent line naming the crawler to which the rules apply, followed by allow and disallow lines specifying paths. A crawler that honours the protocol fetches the file before crawling, finds the group matching its own user-agent token, or the wildcard group if none matches, and avoids disallowed paths. For nearly three decades the protocol existed only as an informal convention. In September 2022 the Internet Engineering Task Force published it as RFC 9309, standardising its syntax and parsing rules, including how crawlers should handle errors, redirects and caching. The protocol has three properties that matter for AI. It is voluntary: nothing in the protocol itself compels compliance, and a crawler that ignores it faces no technical barrier. It is crawler-based rather than use-based: it lets a site say which crawlers may visit, not what they may do with what they collect. And it is forward-looking: it governs future visits, not copies already made. The second property caused immediate difficulty. A site operator who wanted to be indexed by search engines but not used for AI training had no way to say so if the same crawler served both purposes. AI developers responded by publishing separate user-agent tokens for different purposes. OpenAI announced its GPTBot crawler for training data in August 2023 and later added OAI-SearchBot for its search features and ChatGPT-User for fetches made on behalf of users. Google introduced a token called Google-Extended in September 2023, which controls whether content crawled by Googlebot may be used for training and grounding its Gemini models, without affecting search indexing. Apple introduced Applebot-Extended in 2024. Anthropic, Meta, Perplexity, ByteDance, Common Crawl and others publish their own tokens. These tokens gave site operators finer control but did not resolve the underlying tension. Publishers have complained that Google-Extended does not cover the use of content in AI-generated summaries displayed in Google's search results, so that opting out of those uses would require opting out of search entirely, an unacceptable cost for most publishers. Several AI companies have also taken the position that fetches triggered by a user's specific request, such as asking an assistant to read a particular page, are not crawling and are therefore not governed by robots.txt in the same way, by analogy with a user's browser. Publishers dispute this. In August 2025 the network services company Cloudflare publicly accused Perplexity of using undeclared crawlers with generic browser user-agents to reach sites that had blocked its declared crawlers; Perplexity rejected the characterisation, arguing that the traffic reflected user-initiated requests and third-party services. Whatever the merits of that dispute, it illustrates that crawler identity, and whether it is honestly declared, has become a contested factual question with legal consequences. How widely sites are opting out The scale of the response has been measured. In July 2024 the Data Provenance Initiative published "Consent in Crisis," an audit of the robots.txt files and terms of service of the domains underlying three major public training corpora, C4, RefinedWeb and Dolma. It found that in a single year, from 2023 to 2024, restrictions had risen sharply: roughly five percent of all tokens in C4, and more than a quarter of tokens from the most actively maintained and widely used sources, had become fully restricted from AI crawlers through robots.txt. It also found widespread inconsistency between what sites said in robots.txt and what they said in their terms of service, and between the restrictions imposed on different AI developers' crawlers. The authors drew two conclusions. The first was that the open web available for training was shrinking quickly, with consequences for the diversity and freshness of data available to developers who honour opt-outs, and potentially an advantage for those who already hold large archives or are willing to ignore them. The second was that the signalling infrastructure was not fit for the purpose to which it was being put: it could not express the distinctions site operators actually wanted to make, such as permitting search and research while refusing commercial training. Beyond robots.txt: the newer signals Several initiatives have tried to fill the gaps. The TDM Reservation Protocol, developed by a World Wide Web Consortium community group and finalised in 2024, was designed specifically for the EU's Article 4 reservation. It lets a site declare, through an HTTP header, an HTML meta tag or a file at a well-known location, that text and data mining rights are reserved, and optionally point to a policy describing how a licence can be obtained. It is purpose-based rather than crawler-based, and it has been adopted particularly by European publishers. Media-specific metadata provides another layer. The IPTC photo metadata standard includes a field for data mining permissions that can be embedded in image files. The Coalition for Content Provenance and Authenticity, an industry body whose C2PA specification is best known for marking the origin of images and video, includes an assertion by which content creators can state that their work should not be used for training. Some platforms, such as DeviantArt, introduced "noai" directives in page metadata. These signals travel with the content itself rather than with the site that hosts it, which helps when images are copied across many sites, but they are easily stripped by ordinary processing. Opt-out registries take a different approach. The company Spawning built a registry, Do Not Train, where artists and rights holders could list works or domains they did not want used, and some developers, including Stability AI for later versions of its image models, said they would honour it. Registries let individual creators opt out without controlling the sites where their work appears, but they depend on developers choosing to consult them. The Internet Engineering Task Force chartered a working group on AI preferences, known as AIPREF, in January 2025, to standardise a vocabulary for expressing preferences about AI use and mechanisms for attaching those preferences to content. The vocabulary draft, revised most recently in September 2026, defines three categories of use: AI training, AI use, which covers using content as input to AI systems at the moment of use, and search. A companion draft defines how preferences are attached through robots.txt and HTTP headers, so that a site could, for example, allow crawling and search while stating a preference against training. As of November 2026 these remain drafts and not published standards. Importantly, the working group has framed its output as a way of expressing preferences, leaving their legal effect to the law of each jurisdiction. Two private initiatives have moved further toward licensing. Really Simple Licensing, a standard released in version 1.0 in December 2025 by a steering committee backed by publishers including Yahoo, Ziff Davis and O'Reilly Media and endorsed by infrastructure companies including Cloudflare and Akamai, extends robots.txt with a machine-readable description of licensing terms. It distinguishes uses such as all AI uses, use as input to AI answers, and indexing for AI search, and lets publishers specify payment, attribution or contribution terms. Cloudflare, which carries a large share of web traffic, began blocking known AI crawlers by default for new customer domains in July 2025, launched a "pay per crawl" scheme in private beta that uses the long-dormant HTTP 402 Payment Required status code to let sites charge crawlers per request, and later introduced a Content Signals Policy that adds purpose-based statements about search, AI input and AI training to robots.txt files. Cloudflare and others have also promoted cryptographic verification of crawler identity, by which bots sign their requests so that sites can confirm a crawler is who it claims to be. What the signals mean in law The legal weight of these signals differs by jurisdiction, and it is changing. In the European Union, a valid machine-readable reservation removes the commercial text and data mining exception, so a developer that collects content in the face of one has no exception to rely on. The questions are what counts as machine-readable and appropriate. The Hanseatic Higher Regional Court held in December 2025 that a plain-language reservation in a website's terms was not machine-readable for a download in 2021, while leaving open that natural-language reservations might qualify once technology can reliably interpret them. Advocate General Szpunar in November 2026 treated robots.txt as a valid method despite its limitations. The Code of Practice commits signatories to honour robots.txt under RFC 9309 and "other appropriate machine-readable protocols," and the Commission has indicated that it expects the set of recognised protocols to evolve. A careful developer serving European users should therefore treat robots.txt, the TDM Reservation Protocol, and emerging standards such as AIPREF as binding signals, and should keep records showing what each said at the time of collection. In the United States, robots.txt has no statutory force. But it plays three roles. First, it is evidence of good or bad faith in the fair use analysis, which considers the propriety of the defendant's conduct; plaintiffs in several news cases allege that defendants continued to collect content after being blocked. Second, ignoring robots.txt in combination with other conduct may support claims for breach of contract or trespass to chattels, although American courts have been sceptical of such claims for public pages. Third, and most sharply, working around technical barriers can support anti-circumvention claims under Section 1201 of the Digital Millennium Copyright Act, which prohibits circumventing technological measures that effectively control access to copyrighted works. The anti-circumvention question is now live. In July 2026 Chief Judge Yvonne Gonzalez Rogers dismissed Google's Section 1201 claims against the search-scraping company SerpApi, holding that Google had not adequately pleaded that its anti-bot system, SearchGuard, effectively controlled access to copyrighted works or operated with the authority of the copyright owners; claims relating to search results with no copyrighted content were dismissed without leave to amend, while claims relating to copyrighted images were dismissed with leave to replead. In the Southern District of New York, Judge Paul Engelmayer allowed Reddit's Section 1201 claim against Perplexity and SerpApi to proceed, holding at the pleading stage that SearchGuard qualified as an access control and that Reddit's injuries fell within the statute's zone of interests. The two courts were addressing different plaintiffs and pleadings, and neither ruling decides the merits, but together they establish that circumventing bot-detection systems to reach content carries real legal risk, and that the outcome may depend on whose content sits behind the barrier. In Japan, the Agency for Cultural Affairs' guidance suggests that circumventing technical measures designed to prevent the reproduction of a database offered for AI training may engage the proviso that removes the protection of Article 30-4. In every jurisdiction surveyed, then, the developer's response to a site's technical signals has some legal relevance, ranging from decisive to evidential. The critique of opt-out Opt-out regimes have critics on both sides. Creators and their representatives argue that opt-out reverses the ordinary logic of property, placing the burden on millions of rights holders to protect their work from use rather than on the user to seek permission, and that it is unworkable for individuals whose work is copied to sites they do not control. Many argue for opt-in consent or collective licensing instead. They also note that opt-outs are only forward-looking, offering no remedy for content collected before the signal was set. Developers and some researchers argue the reverse: that opt-out regimes, combined with fragmented and inconsistent signals, risk excluding vast swathes of valuable and lawful material from training, particularly for smaller developers and researchers who cannot afford licences, and that signals intended to block commercial training often end up blocking research, archiving and accessibility uses as well. The Consent in Crisis authors made a version of this point, noting that blunt robots.txt blocks frequently fail to distinguish between uses that site operators might accept and those they would not. Both critiques point toward the same practical conclusion. The signalling landscape is fragmented, evolving and legally significant. A pipeline that reads only one kind of signal, or reads signals without recording them, will not withstand scrutiny. The next chapter describes how to build one that does. Chapter 9. Building a Compliant and Auditable Collection Pipeline The preceding chapters converge on a practical conclusion. Whatever the eventual answers to the doctrinal questions, a developer's legal position depends heavily on facts about collection and use that only the developer can record: where each item came from, what terms and signals accompanied it, what was done to it, which models consumed it, and what happened when someone objected. This chapter describes how to build a web collection pipeline that records those facts as a matter of course. The aim is a pipeline that is audit-proof in the only sense worth pursuing: one that survives an audit because it behaves well and can prove it. Nothing here concerns evading detection, disguising crawler identity or defeating technical barriers. Those practices are excluded by design, for legal reasons that earlier chapters made clear and for the simpler reason that a pipeline built to hide what it does cannot also be built to show what it does. Principles Six principles organise the design. Identify honestly. The crawler declares who operates it and why, in a way site operators can verify, and never presents itself as something else. Ask before taking. Before collecting any resource, the pipeline evaluates every applicable signal of consent or refusal and collects only what those signals permit. Record what was seen. The pipeline captures not only the content but the signals, terms and context that governed its collection, at the moment of collection, because those facts change and cannot reliably be reconstructed later. Carry the record forward. Every transformation, dataset and training run preserves a link back to the original records, so that the provenance of any item in any model can be answered by lookup. Make removal possible. The pipeline can find and exclude any item, source or class of content from all future datasets and models, and can show that it has done so. Separate policy from mechanism. Decisions about what may be collected are expressed as explicit, versioned rules, reviewed by people with legal and ethical responsibility, and applied by software that logs which version of the rules governed each decision. Stage one: source policy Collection begins before the first request, with decisions about which sources are in scope. A well-governed pipeline maintains several lists, each with an owner and a change history. An exclusion list names domains that must never be collected. It should include known piracy and shadow-library sites; the EU Code of Practice commits signatories to exclude sites that courts or public authorities in the EU have identified as persistently infringing on a commercial scale, and to use a list maintained for that purpose. It should also include sites hosting categories of content the developer has decided to avoid for legal or ethical reasons, such as sites dominated by leaked material, sites known to host abusive imagery, and full-text lyrics or book repositories whose content is overwhelmingly reproduced without permission. A licensed-sources list records domains or feeds covered by licence agreements, with a pointer to the agreement and its scope: which uses, which models, what term, what attribution or reporting obligations. Content from these sources should be tagged so that it can be tracked through to the models it reaches, because licence compliance and payment calculations depend on it. A restricted-sources list identifies domains whose terms of service the developer has reviewed and decided to honour, for example by not collecting at all, or by collecting only for specified uses. For the long tail of the web, manual review of terms is impossible, but for the few thousand domains that contribute the most valuable content, it is both feasible and, in light of the Consent in Crisis findings about mismatches between robots.txt and terms, prudent. Each of these lists is policy, not code. It should live in a version-controlled repository, with every change attributed to a person and justified, so that the developer can later show what its rules were on any given date. Stage two: crawler identity and behaviour The crawler should identify itself with a stable, documented user-agent token, ideally separate tokens for separate purposes, such as one for collecting training data and another for fetching content at a user's request, so that site operators can make the distinctions they care about. The operator should publish a page explaining what each crawler does, how to block it, the IP address ranges it uses, and how to contact the operator. Where the site operator or its infrastructure provider supports cryptographic verification of bot identity, the crawler should sign its requests. Behaviour matters as much as identity. The crawler should limit its request rate to avoid burdening sites, respect crawl-delay instructions and HTTP status codes that signal overload, and back off when sites return errors. It should not collect content behind logins, paywalls, CAPTCHAs or bot-detection challenges. It should not route requests through residential proxy networks, rotate user-agents to avoid blocks, or use headless browsers configured to mimic human visitors in order to evade detection. Each of these practices has appeared in recent litigation as evidence of improper acquisition, and two of them, bypassing paywalls and bot-detection systems, are squarely within the circumstances in which the Advocate General in Like Company would deny the text and data mining exception and in which American courts have allowed anti-circumvention claims to proceed. Stage three: evaluating signals at the point of fetch Before each fetch, the pipeline evaluates every applicable signal and records the outcome. The robots.txt file is the first. The crawler should implement RFC 9309 exactly, including its rules for error handling: a robots.txt file that returns a client error in the 400 range may be treated as permitting access, while a server error or an unreachable file must be treated as a complete disallow. It should not rely on a cached copy for more than twenty-four hours under normal conditions, as the standard advises. It should evaluate the rules for each of its own declared tokens and for the wildcard group, and where the site uses purpose-based extensions, such as AIPREF content-usage statements or Cloudflare's content signals, it should parse and honour them. Every evaluation should record a hash of the robots.txt file as retrieved, with the retrieval time, so that the developer can later prove what the file said. The page-level signals come next: HTTP headers and HTML meta tags expressing robots directives, text and data mining reservations under the TDM Reservation Protocol, "noai" directives, and links to licence documents. For media files, the pipeline should read embedded metadata such as IPTC data mining fields and C2PA assertions. Where a site publishes machine-readable licensing terms using Really Simple Licensing or a similar standard, the pipeline should record those terms and route the content according to whether the developer has accepted them. Licence detection follows. Pages that declare a Creative Commons or other open licence, often through a link marked with rel="license", should have the licence recorded in a standard form such as an SPDX identifier, so that downstream processes can treat licence classes differently. Code should be scanned for licence headers and repository licence files. Opt-out registries should be checked against the URL and, for images and other media, against content fingerprints. The result of this stage is a decision for each resource, collect, skip, or collect for restricted use only, recorded together with the evidence on which it rested. If any signal is ambiguous, the default should be not to collect for training. A pipeline that errs toward exclusion loses some data; one that errs toward inclusion accumulates liabilities. Stage four: capture and the provenance record Content that passes the evaluation is captured in a form that preserves its context. The standard web archival format, WARC, already stores the HTTP request and response, including headers, alongside the retrieval time and target URL, and is used by Common Crawl and web archives. The pipeline should compute a cryptographic hash of each captured payload, typically SHA-256, which serves as a stable identifier for the content regardless of where it is later moved or copied. Alongside the archive, the pipeline creates a provenance record for each item. Table 5 sets out the fields such a record should contain and the purposes each serves. Table 5. Fields of an item-level provenance record and their purpose. Field group Contents Purpose in an audit or dispute Identity Record ID; content hash; URL; canonical URL Locating every copy and derivative of an item Acquisition context Retrieval timestamp; crawler token and version; IP address used; HTTP status Proving when and how the item was obtained, and by whom Signals at collection robots.txt hash and verdict; page-level directives; TDM reservation; AI preference statements; registry checks Demonstrating that refusals were honoured as they stood at the time Rights and terms Detected licence identifier; licensed-source agreement reference; terms-of-service classification Applying licence conditions and showing the basis for use Attribution Author, title, publisher, copyright notice as found Preserving management information and enabling credit or payment Processing lineage List of transformations applied, with tool versions and parameters Explaining how the training text was derived from the original Membership Dataset versions and training runs that included the item Answering whether the item reached a given model Status Active, excluded, tombstoned; reason and date of any change Showing that removals and opt-outs propagated Attribution data deserves particular attention. Text extraction routinely strips bylines, copyright notices and licence statements from the content that will be fed to the model, because they are noise for training. That is defensible engineering, but the stripped information should be preserved in the provenance record. Doing so supports any later obligation to credit or pay rights holders, and it answers allegations that management information was knowingly removed. Stage five: processing lineage and dataset versioning Between capture and training, content passes through many transformations: text extraction, language identification, quality filtering, deduplication, removal of personal information, toxicity filtering, tokenisation. Each transformation should be recorded as a step in the item's lineage, with the identity and version of the tool and its parameters. The World Wide Web Consortium's PROV data model, which describes provenance in terms of entities, the activities that generate and use them, and the agents responsible, offers a well-established vocabulary for such records, and several data versioning tools implement similar concepts. The output of processing is a dataset, and every dataset used for training should be immutable and versioned. A dataset version is defined by a manifest listing the content hashes and record identifiers of every item it contains, together with references to the policy versions and processing pipeline versions that produced it. The manifest itself should be hashed, so that any later claim about the contents of a dataset can be checked against a fixed fingerprint. Storage systems that address content by its hash make this natural: the same item appearing in several datasets is stored once and referenced many times, and the developer can always determine exactly which datasets contain it. Datasets should also be documented for human readers. Datasheets for Datasets, proposed by Timnit Gebru and colleagues in 2018 and published in Communications of the ACM in 2021, set out questions about motivation, composition, collection process, preprocessing, uses, distribution and maintenance that a dataset's creators should answer. The MLCommons Croissant format, released in 2024, provides a machine-readable metadata standard for datasets with an extension for responsible-AI information, including provenance. The Data & Trust Alliance published Data Provenance Standards in 2024 that define a common set of metadata fields for describing the source, lineage and legal basis of datasets exchanged between organisations. Adopting one of these formats makes documentation portable and gives auditors a familiar structure. Stage six: training lineage The link that most organisations historically failed to record is the one between datasets and models. Every training run, including pretraining, continued pretraining, fine-tuning and preference training, should record the hashes of the dataset manifests it consumed, the proportions in which they were mixed, and the identity of the resulting model checkpoint. Each released model version should carry, in its internal documentation, the complete list of dataset versions that contributed to it, directly or through earlier checkpoints. With that record in place, the membership question that Chapter 7 showed to be so hard to answer forensically becomes a database query. Was this article in the training data of this model? Look up the article's content hash, find the dataset versions that include it, and check whether any of them appear in the model's lineage. The same query, run in reverse, answers questions a licensor will ask: which models were trained on our content, and in what proportion? Training lineage is also what makes the EU training data summary feasible. The template issued by the Commission in July 2025 asks providers to describe the main data sources used, including the most relevant domain names for web-crawled data, and to explain how they respected reservations of rights. A developer with complete lineage can generate that summary from its records; one without it must reconstruct it from memory and partial logs, with obvious risks. Stage seven: handling change Signals change. A site that permitted crawling in 2024 may block it in 2026. A licence may expire. A rights holder may send a complaint, a court may order removal, or a developer may decide that a source it once used is no longer acceptable. A compliant pipeline treats change as normal and has a defined process for each kind. Revalidation before each dataset build is the first safeguard. Before assembling a new dataset version from previously collected material, the pipeline should re-check current signals for the source domains, at least for robots.txt and TDM reservations, and exclude material from sources that have since opted out. This goes further than the law strictly requires in some jurisdictions, since a reservation applies to future acts, but each new dataset build involves new reproductions, and building from the current state of consent is both defensible and simple to explain. A suppression list holds identifiers of content that must not be used: URLs, domains, content hashes and perceptual fingerprints for images and audio. Every dataset build filters against it. When an item is suppressed, its provenance record is marked with the reason and date, and the raw copy is deleted or quarantined according to the retention policy, leaving a tombstone that records that the item existed and was removed. A complaint process, of the kind the EU Code of Practice requires, routes rights holders' requests to a designated contact, verifies them, adds the relevant identifiers to the suppression list, and replies with a record of the action taken. Because provenance records link items to models, the response can say truthfully which existing models included the content and confirm that future models will not. For existing models, where removal from the weights is not feasible, the developer can add the content to the reference index used by output filters, so that the model is prevented from reproducing it. Retention policy completes the picture. The EU text and data mining exception permits copies to be retained only as long as necessary for the mining purpose. A pipeline should therefore define how long raw captures are kept, for example until the datasets derived from them have been built and validated, and should delete them on schedule, retaining the provenance records, hashes and manifests, which contain no protected expression, as the audit trail. Data from third parties and from models Much training data does not come from a developer's own crawler. Third-party datasets, whether open-source corpora or commercial supplies, should be subject to due diligence proportionate to their risk. A developer should require a supplier to describe how the data was collected, what signals were honoured, what licences apply, and what exclusions were made, ideally in a standard format such as the Data & Trust Alliance fields; should sample the data to check that the description is accurate, for example by testing whether it contains content from excluded domains; should obtain contractual representations about lawful collection and indemnities for breach; and should refuse datasets of unknown or undocumented origin. Books3 and similar collections circulated for years as open-source resources, and their use proved enormously costly. An unexplained dataset is not free. Synthetic data generated by other models raises a related issue. If a developer trains on text or images produced by another model, the provenance of that model's training data becomes part of the developer's own chain, and the terms of service of the generating model may restrict the use of its outputs to train competing models. The provenance record for synthetic data should identify the generating model, its version, the prompts or seeds used, and the terms under which the outputs were obtained. Personal data Web collection inevitably captures personal data, and in the European Union and many other jurisdictions data protection law applies alongside copyright. The European Data Protection Board's Opinion 28/2024, adopted in December 2024, addressed when AI models trained on personal data can be considered anonymous and the conditions for relying on legitimate interest as a lawful basis, emphasising measures such as filtering at collection, respect for robots.txt and similar signals, and transparency. The provenance infrastructure described here serves data protection compliance too: the ability to find every copy of an item, record why it was collected, and suppress it from future use is exactly what is needed to honour objections and erasure requests. Verifying that the pipeline works Controls that are not tested cannot be relied upon. A compliant pipeline should be checked continuously and audited periodically. Continuous checks can be automated. A developer can operate test domains whose robots.txt files block its crawlers in various ways, and confirm that the crawlers respect each configuration; any fetch from a blocked test path is a defect. Metrics should track the proportion of fetches for which signals were evaluated and recorded, the proportion of collected items with complete provenance records, and the time taken for suppressions to propagate into dataset builds. Dataset builds should fail if items without provenance records appear in them. Periodic audits should sample items from recent datasets and trace each back through its lineage to the original capture and signal evaluation, confirming that the record is complete and the decision was correct. They should sample suppressions and confirm that the suppressed content is absent from datasets built afterwards. And they should review the source policy lists, licence registry and complaint log for gaps. The findings of these audits, and the remediation that followed, are themselves part of the audit file discussed in the next chapter. A pipeline built this way does not guarantee that every legal question will be resolved in the developer's favour. It guarantees something more modest and more valuable: that whatever questions are asked, by a court, a regulator, a licensor or the public, the developer can answer them with evidence rather than reconstruction. Chapter 10. Licensing, Governance and the Audit File A well-built pipeline produces records. Records alone do not produce accountability. They must be governed by people with authority to make decisions, tied to the commercial agreements that increasingly determine what data may be used, and assembled into a body of evidence that can be produced when someone asks. This chapter addresses those three layers: the licensing market and the provenance obligations it creates; the internal governance that turns technical controls into defensible decisions; and the audit file, the organised record that a developer should be able to hand to a regulator, a court, a licensor, an insurer or an acquirer. The shape of the licensing market Licences for AI use of content now take several forms, and each carries different provenance obligations. Archive licences grant access to a body of existing content, such as a publisher's back catalogue, a stock image library or a platform's historical posts, for training. They are typically paid as a lump sum or annual fee and may restrict use to particular models or purposes. The early news deals, such as those between OpenAI and the Associated Press, Axel Springer and News Corp, combined archive access with other rights, and academic publishers including Taylor & Francis and Wiley disclosed significant AI licensing revenue in 2024. Feed and display licences grant continuing access to new content and, often, the right to display summaries, excerpts or links in AI products. They are the natural form for retrieval-augmented systems, where the link between source and output is direct, and they often include attribution requirements and reporting of usage. Per-access arrangements charge for each crawl or retrieval. Cloudflare's pay-per-crawl scheme and the payment terms expressible through Really Simple Licensing point toward a market in which site operators set prices for automated access and crawlers accept or decline them programmatically. Collective licences aggregate the rights of many holders and offer them to users through a single agreement. Collective management organisations have long licensed photocopying, broadcasting and public performance in this way. Several have extended their licences to AI uses: the Copyright Clearance Center in the United States added AI re-use rights to some of its licences in 2024, and collecting societies in Europe and the United Kingdom have announced or developed licences for training. Collective licensing is the most plausible route by which individual creators and small publishers, who cannot negotiate bilateral deals, could participate in the market, and it features in policy proposals in several countries. Advocate General Szpunar observed in his November 2026 opinion that member states could introduce statutory remuneration for creators. Every one of these forms requires the licensee to know what it used, and many require the licensor to know what it owns. A licence limited to particular models is meaningless unless the licensee tracks which models consumed the licensed data. A licence with reporting obligations requires usage records. A licence that terminates, with an obligation to stop using the content, requires the ability to exclude it from future training. On the licensor's side, an archive licence is only as good as the licensor's title to the archive. News organisations hold content contributed by freelancers, syndicated from wire services and licensed from photographers, often under agreements that predate AI and may not grant the publisher the right to sublicense for training. Academic publishers that sold training access to their catalogues faced complaints from authors who said they had not been consulted. A licensee should therefore ask for representations about the licensor's rights, and a licensor should audit its own chain of title before selling. Terms that matter Several licence terms deserve attention for their provenance implications. Scope defines which uses are permitted: pretraining, fine-tuning, evaluation, retrieval, display of excerpts, or some combination. Because the legal risks differ by stage, as Chapter 2 showed, licences increasingly distinguish them, and a developer's records need to distinguish them too. Model restrictions may limit the licensed content to particular models, model families or deployment contexts, or prohibit its use in models made available to competitors of the licensor. Compliance requires training lineage of the kind described in Chapter 9. Attribution and reporting provisions may require the licensee to credit the source in outputs, to report how often content was retrieved or displayed, or to provide periodic statements of which models include the licensed data. Audit rights allow the licensor to inspect the licensee's records to confirm compliance. A developer that has granted audit rights must be able to produce records that a third party can understand and verify. Termination provisions specify what happens when the licence ends. Because existing models cannot verifiably forget their training data, as Chapter 7 explained, licences typically address whether models already trained may continue in use, whether new models may include the content, and what the licensee must do with stored copies. Clear drafting here avoids later disputes that no technical measure could resolve. Warranties and indemnities allocate risk. The licensor warrants that it has the rights it purports to license; the licensee may warrant that it will use the content only as permitted. Indemnities shift the cost of third-party claims. Indemnities offered to customers A related market has developed in the other direction. Since 2023, several large AI providers have offered their commercial customers indemnities against copyright claims arising from use of their models' outputs: Microsoft's Customer Copyright Commitment, announced in September 2023, and similar undertakings by Google, OpenAI, Adobe and others. These commitments typically require the customer to use the provider's content filters and other safeguards and exclude cases where the customer deliberately sought infringing output. They are a commercial bet by the provider that its provenance practices and output safeguards make claims unlikely to succeed, and they place a premium on those practices: a provider offering indemnities has every incentive to know what is in its training data and what its models can reproduce. Enterprise customers, in turn, have begun to ask for provenance information as part of procurement. Questions about training data sources, opt-out compliance and output safeguards now appear routinely in vendor due diligence, alongside questions about security and data protection. A developer that can answer them with documentation rather than assurances is at a commercial advantage. Internal governance Technical controls implement decisions; governance makes them. A developer serious about provenance needs clear ownership of four kinds of decision. Source policy decisions determine which categories of data may be collected and used, which are prohibited, and which require specific approval. These are decisions with legal, ethical, reputational and commercial dimensions, and they should be made by a body that includes legal counsel, data protection expertise and senior technical leadership, with the authority to say no. The decisions should be recorded with their reasons. Risk tiering is the practical tool for applying source policy at scale. A common pattern distinguishes, at minimum, content that is licensed, in the public domain or openly licensed for the intended use; content collected from the open web with all applicable signals honoured; higher-risk categories, such as content from sites with restrictive terms, content predominantly created by individual artists, or personal data of special sensitivity, which require specific review and sign-off; and prohibited categories, such as content from piracy sources or content obtained by circumventing access controls. Each tier carries defined requirements for documentation and approval. Exception handling covers the cases that policy does not anticipate: a valuable dataset of uncertain origin, a licensor whose chain of title is unclear, an opt-out signal of ambiguous meaning. A defined route for escalation, with documented decisions, prevents such cases from being resolved informally by engineers under deadline pressure, which is how many of the problematic datasets of the past entered training pipelines. Incident response covers the discovery that something has gone wrong: that a dataset contains material from an excluded source, that a crawler ignored a block, that a model reproduces protected content at an unacceptable rate. The response should stop further use, quarantine affected material, assess which models are affected, involve counsel, and document every step. The Bartz litigation showed the cost of retaining problematic material after its nature was known; the record of a prompt, documented response is a significant mitigating fact. Preservation and retention Retention policies and litigation interact in ways that need planning. A developer applying the retention principle described in Chapter 9 will routinely delete raw captures once datasets are built. But once litigation is reasonably anticipated, the duty to preserve relevant evidence overrides ordinary retention schedules, and deletion of relevant material can lead to severe sanctions. In the New York Times litigation, a magistrate judge ordered OpenAI in May 2025 to preserve output logs it would ordinarily have deleted, an order that drew objections on privacy grounds and was later modified. Retention systems should therefore be designed so that deletion can be suspended for specific categories of material on instruction from counsel, with the suspension itself documented. Preservation also argues for keeping provenance records, hashes and manifests indefinitely, even when the underlying content is deleted. Those records contain little or no protected expression, so retaining them raises little copyright concern, and they are the evidence that answers most of the questions a litigant or regulator will ask. Legacy models and honest gaps Most organisations with models in production trained at least some of them before the practices described in this booklet were common. Their records for those models are incomplete. What should they do? The first step is an honest gap analysis: an assessment of what is known and unknown about the data behind each legacy model, prepared under the direction of counsel. Some gaps can be filled by reconstruction from surviving logs, configuration files, dataset archives and the recollections of the people involved. Others cannot. The second step is a risk-based decision for each model. Options include continuing to use the model with enhanced output safeguards; retraining it on documented data; restricting its use to lower-risk applications; or retiring it. The decision should take into account what is known about high-risk sources in its training data, its measured memorisation of protected content, and the jurisdictions in which it is deployed. Under the EU AI Act, models placed on the market before August 2025 must comply with the general-purpose AI obligations by August 2027, which gives the gap analysis a deadline. The third step is documentation of the decision and its reasons. What a developer must never do is manufacture retrospective records that purport to show controls that did not exist at the time. Fabricated provenance is worse than missing provenance: it converts a gap into misconduct and destroys the credibility of every genuine record the organisation holds. The audit file The culmination of the practices described in this and the previous chapter is an audit file: an organised, current body of documentation that can be produced on request. Its precise contents will depend on the developer and the jurisdictions in which it operates, but a comprehensive file includes the following. The copyright compliance policy required by Article 53 of the AI Act, with its version history and the identities of those responsible for it. The source policy: the exclusion, licensed and restricted lists, with their change history and the reasons for significant changes. Crawler documentation: the published description of each crawler, its user-agent tokens, its IP ranges, its contact details, and the history of changes. Signal evaluation records: for each collection campaign, the logs showing which signals were evaluated, the robots.txt snapshots and their hashes, and the decisions made. Provenance records and dataset manifests: the item-level records and the versioned manifests of every dataset used for training, with their hashes. Training lineage: for each released model, the list of dataset versions consumed and the configurations of the training runs that produced it. The public training data summary, where required, and the internal working papers from which it was prepared. Memorisation and output evaluations: the results of extraction tests, regurgitation measurements and red-team exercises, with the methodology used and the remediation that followed. Output safeguard documentation: descriptions of the filters and policies applied to outputs, with evidence of their effectiveness. The licence register: every licence under which training or retrieval content is held, with its scope, term, model restrictions, reporting obligations and the evidence of compliance. Third-party due diligence: for each external dataset, the supplier's description, the developer's verification results, and the contractual representations obtained. The complaint and suppression log: every rights holder complaint and opt-out request received, the action taken, and confirmation of propagation. Audit and incident reports: the findings of internal and external audits, records of incidents, and the remediation undertaken. Assembling this file is substantial work. But nearly all of it is a by-product of the pipeline and governance described above, and its value is felt in many settings: in technical compliance dialogues with the EU AI Office; in discovery, where a well-organised production can narrow disputes and reduce cost; in licence audits; in insurance underwriting; in enterprise sales; and in mergers and acquisitions, where buyers increasingly ask what liabilities lie in a target's training data. Transparency as strategy A final governance question is how much of this to publish. The EU requires a public training data summary for general-purpose models; beyond that, disclosure is voluntary. There are legitimate reasons for restraint: detailed disclosure of data sources can reveal commercially valuable information about a developer's methods, and can invite litigation by identifying potential plaintiffs. There are also strong reasons for openness. Public documentation of crawler behaviour, opt-out mechanisms and complaint processes is necessary for those mechanisms to work, since site operators can only block or license crawlers they know about. Published dataset documentation, of the kind pioneered by open projects such as Dolma and FineWeb, allows independent scrutiny that builds trust. And a developer that publishes clear commitments, and can be seen to honour them, accumulates credibility that will matter when courts and regulators assess its conduct. The trend of the past three years, from opaque corpora to documented ones, has been driven partly by regulation and litigation, but partly by the recognition that in a field where so much depends on trust, provenance that can be shown is worth more than provenance that is merely claimed. Conclusion The question that opened public debate about generative AI and copyright was simple and sweeping: is it lawful to train machines on the creative work of others without permission? Four years of litigation, legislation and research have not produced a simple answer. What they have produced is a clearer understanding of why the question resists a simple answer, and of what the answer will depend on. It will depend, first, on acquisition. The most consequential financial outcome in AI copyright to date, the $1.5 billion Bartz settlement approved in July 2026, paid for the way books were obtained, not for what was done with them. The Advocate General's opinion in Like Company would make lawful access a condition of the European exception. The paywall evidence unsealed in the New York Times litigation, and the anti-circumvention claims proceeding against scrapers, show the same theme from different angles. Where data comes from is no longer a background fact. It is often the decisive one. It will depend, second, on consent signals. In the European Union, a valid machine-readable reservation removes the exception on which commercial training relies, and the Code of Practice commits signatories to honour robots.txt. In the United States, a developer's response to blocking signals is evidence of its good or bad faith, and working around technical barriers creates independent liability. In Japan, circumventing such barriers may remove the protection of an otherwise broad exception. The crude and fragmented signalling infrastructure of the web has acquired legal weight that its designers never intended, and new standards are being built to carry it. It will depend, third, on memorisation. The English and German courts that disagreed in November 2025 about whether a model can be a copy were, on closer reading, agreeing about the test and disagreeing about the facts. A model from which a work can be reliably reconstructed contains that work in a sense the law can recognise; a model that has learned only patterns does not. The research literature confirms that both kinds of model exist, and that the same model may be one kind with respect to a famous song and the other with respect to a million obscure web pages. And it will depend, fourth, on markets. The Third Circuit's recognition of a market for training data, Judge Chhabria's theory of market dilution, the Copyright Office's acceptance of that theory, and the licensing deals and settlements that now span news, music, books and images together make the economic effect of AI on creative markets a central, and empirical, question. Each of these is a question about provenance. That is the argument of this booklet, and its practical consequence is the most important thing a reader should take from it. Why provenance is the robust strategy Consider the futures the law might reach. In one, American courts broadly hold that training on lawfully acquired data is fair use, Europe settles on a workable opt-out regime, and the main legal risks lie at the edges. In that world, provenance records are what let a developer show that its data was lawfully acquired, that it honoured opt-outs, and that it took measures against memorisation. In a second future, courts accept market dilution and fair use narrows for generative training, pushing the industry toward licensing at scale. In that world, provenance records are the substance of the licensing bargain, the evidence of what was used, by which models, and on what terms. In a third future, legislatures impose transparency or opt-in requirements, or create collective licensing schemes with statutory remuneration. In that world, provenance records are what compliance consists of. There is no plausible future in which a developer is better off not knowing what it trained on. The investment in provenance infrastructure is therefore robust to legal uncertainty in a way that few other strategic choices are. It pays off whichever way the doctrine turns. That is the strongest argument for making it now, before the doctrine is settled, rather than waiting to see what the courts require. The argument from technical capability points the same way. Chapter 7 showed that tracing a trained model back to its sources is possible only in limited ways. Membership inference for individual documents is unreliable at scale; dataset inference and planted markers help large rights holders in specific circumstances; attribution methods are approximate and costly; unlearning suppresses rather than removes. There is no forensic technique that will reliably reconstruct, after the fact, the provenance that was not recorded at the time. The only dependable path to answers about what went into a model is to record it on the way in. What remains unsettled Honesty requires acknowledging how much is still open. No appellate court has ruled on whether training a generative model is fair use. The Third Circuit's affirmance in Thomson Reuters v. Ross expressly reserved that question, and the summary judgment motions in the consolidated OpenAI litigation remain pending. Market dilution has been recognised in principle but never found proved on an evidentiary record, and it is not yet clear what such proof would look like. The Court of Justice has not ruled in Like Company, and the Advocate General has advised it not to reach the training questions at all. The German Federal Court of Justice may rule in the LAION case or refer further questions. The Court of Appeal in London is now hearing Getty's appeal on whether an imported model can be an infringing copy. The Munich decisions against OpenAI and Suno are subject to appeal. The territorial reach of national copyright law over training conducted elsewhere remains contested, with the EU AI Act, the Munich court and the Advocate General all pointing toward broader reach and the English courts so far adhering to strict territoriality. The legal effect of the newer consent signals, from AIPREF vocabularies to Really Simple Licensing terms, has not been tested. Nor has the question of what a developer owes a rights holder who opts out after its content has already been used, given that current techniques cannot verifiably remove content from trained models. And the distributional questions remain largely unaddressed: whether licensing markets will reach individual creators and small publishers or remain the preserve of large rights holders and large developers, and whether compliance costs will entrench incumbents. What readers should do For developers, the practical conclusions follow directly. Treat acquisition provenance as the first line of legal defence: exclude piracy sources and anything obtained by circumventing access controls, and document that you have done so. Identify crawlers honestly and honour every applicable signal of refusal, recording what each signal said at the time. Build item-level provenance records, versioned dataset manifests and training lineage, so that membership questions become lookups. Measure memorisation and deploy output safeguards, and keep the evidence. Govern exceptions and incidents through people with authority and record their decisions. Assess legacy models honestly and never manufacture records after the fact. For rights holders, the conclusions are different but complementary. Express refusals in every machine-readable form that may carry legal weight in the jurisdictions you care about, and keep records of when you did so. If you hold large collections, consider planting markers and preserving clean held-out sets that would support dataset inference. If you license your content, audit your own chain of title first, and negotiate for reporting, audit and termination terms that your counterparty can actually honour. And recognise that the strongest claims in current litigation are those built on specific evidence about acquisition, crawling behaviour and output reproduction. For policymakers, the lesson is that rules which reward provenance will be easier to comply with and to enforce than rules which depend on contested abstractions. Standardising consent signals, supporting shared registries and exclusion lists, clarifying the legal effect of machine-readable reservations, and considering safe harbours conditioned on documented good practice would all make the system more predictable for developers and more protective for creators. The law of generative AI will not be settled by a single grand ruling about whether machines may learn. It is being settled, case by case and rule by rule, through questions about what was taken, from where, with whose consent, and what came out the other side. Those who can answer such questions with evidence will be in the strongest position, whatever the courts eventually decide. Notes Litigation status throughout is stated as of mid-November 2026. Where a decision was reported but not yet publicly available in full, as with the sealed Third Circuit opinion in Thomson Reuters v. Ross, the account relies on published summaries by practitioners and the legal press. Introduction and Chapter 1 Shayne Longpre et al., "The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing and Attribution in AI," arXiv:2310.16787 (2023). Colin Raffel et al., "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer," Journal of Machine Learning Research 21 (2020), introducing the C4 corpus. Leo Gao et al., "The Pile: An 800GB Dataset of Diverse Text for Language Modeling," arXiv:2101.00027 (2020). Christoph Schuhmann et al., "LAION-5B: An Open Large-Scale Dataset for Training Next Generation Image-Text Models," NeurIPS Datasets and Benchmarks (2022). David Thiel, "Identifying and Eliminating CSAM in Generative ML Training Data and Models," Stanford Internet Observatory (December 2023). Van Buren v. United States, 593 U.S. 374 (2021); hiQ Labs, Inc. v. LinkedIn Corp., 31 F.4th 1180 (9th Cir. 2022). Chapters 2 and 3 Copyright Act of 1976, 17 U.S.C. §§ 106, 107, 1201, 1202. Andy Warhol Foundation for the Visual Arts, Inc. v. Goldsmith, 598 U.S. 508 (2023); Authors Guild v. Google, Inc., 804 F.3d 202 (2d Cir. 2015); Sega Enterprises Ltd. v. Accolade, Inc., 977 F.2d 1510 (9th Cir. 1992); Sony Computer Entertainment, Inc. v. Connectix Corp., 203 F.3d 596 (9th Cir. 2000). Thomson Reuters Enterprise Centre GmbH v. Ross Intelligence Inc., No. 1:20-cv-613 (D. Del.), memorandum opinion of 11 February 2025; affirmed by the US Court of Appeals for the Third Circuit, 29 September 2026 (opinion initially filed under seal). Bartz v. Anthropic PBC, No. 3:24-cv-05417 (N.D. Cal.), order on fair use of 23 June 2025; final approval of class settlement, 20 July 2026. Kadrey v. Meta Platforms, Inc., No. 3:23-cv-03417 (N.D. Cal.), order on summary judgment of 25 June 2025. In re OpenAI, Inc. Copyright Infringement Litigation, No. 1:25-md-03143 (S.D.N.Y.); cross-motions for summary judgment in the news cases filed 4 September 2026. US Copyright Office, Copyright and Artificial Intelligence, Part 3: Generative AI Training, pre-publication version (9 May 2025). Chapter 4 Concord Music Group, Inc. v. Anthropic PBC, No. 5:24-cv-03811 (N.D. Cal.). Andersen v. Stability AI Ltd., No. 3:23-cv-00201 (N.D. Cal.), order of 12 August 2024. Getty Images (US), Inc. v. Stability AI Ltd., [2025] EWHC 2863 (Ch), judgment of 4 November 2025. Disney Enterprises, Inc. v. Midjourney, Inc. (C.D. Cal., filed 11 June 2025). Advance Local Media LLC v. Cohere Inc. (S.D.N.Y.), decision on motion to dismiss, November 2025. Google LLC v. SerpApi, LLC, No. 4:25-cv-10826-YGR (N.D. Cal.), order of July 2026; Reddit, Inc. v. SerpApi, LLC et al. (S.D.N.Y.), decision on motions to dismiss, 2026. Chapter 5 Directive (EU) 2019/790 on copyright and related rights in the Digital Single Market, Articles 3, 4 and 15. Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 53 and Recital 106. European Commission, General-Purpose AI Code of Practice, Copyright Chapter (10 July 2025); Template for the Public Summary of Training Content (July 2025). Case C-250/25, Like Company v. Google Ireland Limited, Opinion of Advocate General Szpunar, 3 November 2026. Kneschke v. LAION e.V., Landgericht Hamburg, 310 O 227/23, judgment of 27 September 2024; Hanseatisches Oberlandesgericht Hamburg, judgment of December 2025. GEMA v. OpenAI, Landgericht München I, 42 O 14139/24, judgment of 11 November 2025; GEMA v. Suno, Landgericht München I, 42 O 763/25, judgment of 31 July 2026. UK Government, Copyright and Artificial Intelligence: report under the Data (Use and Access) Act 2025 (March 2026). Agency for Cultural Affairs (Japan), General Understanding on AI and Copyright in Japan (March 2024). ANI Media Pvt. Ltd. v. OpenAI OpCo LLC, Delhi High Court, order on interim injunction, 24 July 2026. Chapters 6 and 7 Nicholas Carlini et al., "Extracting Training Data from Large Language Models," USENIX Security Symposium (2021). Nicholas Carlini et al., "Quantifying Memorization Across Neural Language Models," ICLR (2023). Milad Nasr et al., "Scalable Extraction of Training Data from (Production) Language Models," arXiv:2311.17035 (2023). Nicholas Carlini et al., "Extracting Training Data from Diffusion Models," USENIX Security Symposium (2023). Gowthami Somepalli et al., "Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models," CVPR (2023). Daphne Ippolito et al., "Preventing Generation of Verbatim Memorization in Language Models Gives a False Sense of Privacy," INLG (2023). Katherine Lee et al., "Deduplicating Training Data Makes Language Models Better," ACL (2022); Nikhil Kandpal, Eric Wallace and Colin Raffel, "Deduplicating Training Data Mitigates Privacy Risks in Language Models," ICML (2022). A. Feder Cooper, Mark A. Lemley et al., "Extracting Memorized Pieces of (Copyrighted) Books from Open-Weight Language Models," arXiv (2025). Abhimanyu Hans et al., "Be Like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs," NeurIPS (2024). Reza Shokri et al., "Membership Inference Attacks Against Machine Learning Models," IEEE Symposium on Security and Privacy (2017). Weijia Shi et al., "Detecting Pretraining Data from Large Language Models," ICLR (2024). Michael Duan et al., "Do Membership Inference Attacks Work on Large Language Models?" Conference on Language Modeling (2024). Pratyush Maini et al., "LLM Dataset Inference: Did You Train on My Dataset?" NeurIPS (2024). Matthieu Meeus et al., "Copyright Traps for Large Language Models," ICML (2024). Johnny Tian-Zheng Wei, Ryan Yixiang Wang and Robin Jia, "Proving Membership in LLM Pretraining Data via Data Watermarks," Findings of ACL (2024). Pang Wei Koh and Percy Liang, "Understanding Black-box Predictions via Influence Functions," ICML (2017). Roger Grosse et al., "Studying Large Language Model Generalization with Influence Functions," arXiv:2308.03296 (2023). Sung Min Park et al., "TRAK: Attributing Model Behavior at Scale," ICML (2023). Andrew Ilyas et al., "Datamodels: Predicting Predictions from Training Data," ICML (2022). Samyadeep Basu, Phillip Pope and Soheil Feizi, "Influence Functions in Deep Learning Are Fragile," ICLR (2021). Sheng-Yu Wang et al., "Evaluating Data Attribution for Text-to-Image Models," ICCV (2023). Lucas Bourtoule et al., "Machine Unlearning," IEEE Symposium on Security and Privacy (2021). Ronen Eldan and Mark Russinovich, "Who's Harry Potter? Approximate Unlearning in LLMs," arXiv:2310.02238 (2023). Weijia Shi et al., "MUSE: Machine Unlearning Six-Way Evaluation for Language Models," arXiv:2407.06460 (2024). Chapters 8 to 10 M. Koster, G. Illyes, H. Zeller and L. Sassman, RFC 9309: Robots Exclusion Protocol, Internet Engineering Task Force (September 2022). Shayne Longpre et al., "Consent in Crisis: The Rapid Decline of the AI Data Commons," arXiv:2407.14933 (2024). W3C TDM Reservation Protocol Community Group, TDM Reservation Protocol (TDMRep), Final Community Group Report (2024). IETF AIPREF Working Group, "A Vocabulary for Expressing AI Usage Preferences," draft-ietf-aipref-vocab-08 (September 2026), and "Associating AI Usage Preferences with Content in HTTP," draft-ietf-aipref-attach. RSL Technical Steering Committee, Really Simple Licensing 1.0 Specification (December 2025). Timnit Gebru et al., "Datasheets for Datasets," Communications of the ACM 64, no. 12 (2021). MLCommons, Croissant Metadata Format and Croissant RAI extension (2024). Data & Trust Alliance, Data Provenance Standards (2024). W3C, PROV-O: The PROV Ontology, W3C Recommendation (2013). European Data Protection Board, Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models (December 2024). Further Reading US Copyright Office, Copyright and Artificial Intelligence, Part 3: Generative AI Training (pre-publication version, May 2025). The most thorough official analysis of how American fair use doctrine applies to training, covering transformativeness, market dilution and the role of licensing. Cited by both sides in current litigation. Bartz v. Anthropic, order on fair use (N.D. Cal., June 2025), and Kadrey v. Meta, order on summary judgment (N.D. Cal., June 2025). Read together, these two short opinions show how judges reason about training, acquisition and market harm, and where they diverge. Opinion of Advocate General Szpunar in Case C-250/25, Like Company v. Google Ireland (November 2026). The most detailed European judicial treatment of generative AI and copyright, including the three proposed conditions for applying the text and data mining exception to training. Shayne Longpre et al., "Consent in Crisis: The Rapid Decline of the AI Data Commons" (2024). An empirical audit of robots.txt and terms of service across the sources of major training corpora, and the clearest account of how and why the open web available for training is contracting. Shayne Longpre et al., "The Data Provenance Initiative" (2023). The audit that documented the scale of licence omission and misattribution in widely used datasets, with an accompanying tool for tracing dataset lineage. Nicholas Carlini et al., "Quantifying Memorization Across Neural Language Models" (2023). The foundational empirical study of how memorisation scales with model size, duplication and context, essential background for any argument about whether a model contains its training data. A. Feder Cooper, Mark A. Lemley et al., "Extracting Memorized Pieces of (Copyrighted) Books from Open-Weight Language Models" (2025). A careful study by computer scientists and a leading copyright scholar showing how memorisation varies across books and models, with explicit discussion of the legal implications. Roger Grosse et al., "Studying Large Language Model Generalization with Influence Functions" (2023). The most ambitious attempt to apply training data attribution at scale, valuable both for its method and for its evidence of how diffuse influence becomes in large models. Timnit Gebru et al., "Datasheets for Datasets" (2021). The short paper that established the practice of structured dataset documentation; still the best starting point for anyone designing provenance records. RFC 9309, Robots Exclusion Protocol (2022), and the IETF AIPREF working group drafts. The primary technical documents for the signals that now carry legal weight. Reading the standards themselves clarifies what robots.txt can and cannot express. European Commission, General-Purpose AI Code of Practice, Copyright Chapter (2025). A concise statement of what regulators now expect a responsible developer's copyright practices to look like, from crawler behaviour to complaint handling. Glossary Crawler. A program that automatically requests web pages, stores them and follows their links. Crawlers identify themselves to websites with a user-agent string. Dataset inference. A statistical method that combines many weak membership signals across a collection of documents to test whether the collection as a whole was used to train a model. Deduplication. The removal of exact or near-duplicate documents or passages from a training corpus. It substantially reduces memorisation. Extraction. Causing a model to output material from its training data, verbatim or nearly so. Successful extraction of a long passage is strong evidence that the passage was memorised. Fair use. The American defence to copyright infringement codified in Section 107 of the Copyright Act, decided by weighing four factors: purpose and character of the use, nature of the work, amount used, and market effect. General-purpose AI model. Under the EU AI Act, a model trained on large amounts of data that displays significant generality and can perform a wide range of distinct tasks. Providers face transparency and copyright obligations under Article 53. Influence function. A mathematical approximation of how a model's output would change if a particular training example were removed or up-weighted, used for training data attribution. Lawful access. A condition of the EU and Singapore text and data mining exceptions: the miner must have obtained the works through legitimate means, which excludes pirated sources and circumvention of access controls. Membership inference. An attempt to determine whether a specific example was in a model's training data by observing how the model treats it. Memorisation. The retention by a model of specific training examples in a form from which they can be reproduced, as opposed to learning general patterns. Provenance. The documented origin, history and chain of custody of data, including where it came from, under what terms, and what was done to it. Retrieval-augmented generation. A technique in which a system fetches documents at query time and supplies them to a model as context for its answer. Robots exclusion protocol. The convention, standardised as RFC 9309 in 2022, by which a website's robots.txt file tells crawlers which paths they may or may not visit. Text and data mining. The automated analysis of text and data in digital form to generate information such as patterns, trends and correlations. In EU law, the subject of exceptions in Articles 3 and 4 of the Digital Single Market Directive. Training lineage. The record linking each trained model version to the exact dataset versions and training runs that produced it. Unlearning. Techniques for removing or suppressing the influence of specific training data on a model that has already been trained. WARC. The Web ARChive file format, an international standard for storing crawled web content together with request and response metadata. Hashtags: #DataProvenance #GenerativeAI #AICopyright #CopyrightLaw #FairUse #IntellectualProperty #AILitigation #AIGovernance #ResponsibleAI #TrustworthyAI #ArtificialIntelligence #TrainingData #TrainingDataProvenance #DataLineage #DataTraceability #TrainingDataTracing #CopyrightCompliance #AICompliance #AIRegulation #EUAIAct #CopyrightInfringement #CopyrightLitigation #FairUseDoctrine #TextAndDataMining #TDMExceptions #DigitalCopyright #DataLicensing #ContentLicensing #LicensedTrainingData #WebScraping #EthicalWebScraping #WebCrawling #RobotsTxt #CrawlerCompliance #MachineReadableOptOut #RightsReservation #DataCollection #AuditableAI #AuditTrails #ComplianceAuditing #DataGovernance #DatasetDocumentation #DataSupplyChain #ChainOfCustody #ModelMemorization #TrainingDataExtraction #MembershipInference #MachineUnlearning #RetrievalAugmentedGeneration #AITransparency
- Feminist Legal Theory (Intersectionality, Patriarchy, and the Law)
Download the Book (PDF): Introduction In 1976, five Black women sued General Motors. Emma DeGraffenreid and her co-plaintiffs had been laid off from the company's St. Louis plant under a seniority system, and they argued that the system carried forward an older pattern: before 1964, General Motors had hired no Black women at all. When the plant finally began hiring them, they were the newest employees, and so they were the first to go when layoffs came. The federal district court did not reach the question of whether this was fair. It dismissed their claim on the ground that the law recognized discrimination against Black people and discrimination against women, but not a special category of discrimination against Black women. General Motors had hired women, the court noted; they were white women, and they worked in offices. It had hired Black workers; they were men, and they worked on the factory floor. Each axis of the claim, taken alone, failed. The women who stood at the point where the two axes met were, in the eyes of the law, nobody in particular. Thirteen years later, a young law professor named Kimberlé Crenshaw used DeGraffenreid v. General Motors as the opening illustration in an article that gave a name to what the court had missed. She called it intersectionality. The word has since escaped the law reviews and entered public argument, where it is praised, caricatured, and fought over. But its origin is worth remembering, because it was not born as a slogan or an identity politics. It was born as a technical criticism of how antidiscrimination law sorts claims, and as a demonstration that a rule which looks neutral on its face (treat race claims one way, sex claims another) can produce results that are anything but. That is the core insight of feminist legal theory as a whole, and it is the subject of this book. Law presents itself as neutral. Its rules are phrased in general terms: "any person," "the reasonable man," "a party to the contract," "the owner." Its judges are trained to decide cases by applying those terms to facts, without regard to who the parties are. Feminist legal theory begins from the observation that this neutrality is often formal rather than real. The general terms were written, and the doctrines built on them were developed, in a world where the presumptive legal subject was a man, usually a propertied one, and usually a white one. Rules designed around that subject can disadvantage everyone else without ever mentioning them. What this book argues The argument of this book is that feminist legal theory is best understood not as a single doctrine or a single political program, but as a set of methods for testing the law's claim to neutrality, and that those methods have been remarkably productive, sometimes self-correcting, and genuinely contested. The book tries to show all three things at once. Productive, because feminist legal scholars and advocates changed American law in ways that most people now take for granted. Sexual harassment became a legally recognized form of sex discrimination largely because Catharine MacKinnon argued that it should be, and the Supreme Court agreed in 1986. The marital rape exemption, which for centuries made it legally impossible for a husband to rape his wife, was dismantled state by state in the 1970s, 1980s and 1990s. The idea that a woman's pay, her credit, her jury service and her military eligibility could be set by statute according to her sex was taken apart in a sequence of Supreme Court cases, several of them argued by Ruth Bader Ginsburg. None of this was inevitable, and none of it happened because the law's own neutral logic demanded it. Self-correcting, because the most influential critiques of feminist legal theory came from inside it. Black feminist scholars, starting well before Crenshaw, showed that much early feminist analysis quietly took the experience of white, middle-class women as the experience of "women" in general. That criticism reshaped the field, and intersectionality is its best-known product. Other internal critics questioned whether the turn to criminal law as the main remedy for violence against women had made poor women and women of color less safe, not more. Still others asked whether feminism's focus on women's injury had made it blind to women's agency, or to harms suffered by men and by people who do not fit the binary categories on which much sex discrimination law depends. Contested, because the theory faces serious external critiques, which this book presents in their strongest form. Economists and law-and-economics scholars argue that many outcomes feminist theory attributes to discrimination are better explained by choices, preferences, and the costs of family life, and that legal intervention can make those costs worse. Civil libertarians argue that some feminist proposals, especially on pornography and campus discipline, sacrifice speech and due process. Conservative critics argue that the law's recognition of sex differences is often reasonable rather than oppressive. The Supreme Court itself has, in its most recent decisions on abortion, gender-affirming care for minors and school sports, rejected arguments that feminist and allied scholars had advanced. A fair account of the field has to explain why those arguments have force. How the book proceeds The book moves from theory to doctrine and back again. Chapter 1 explains what feminist legal scholars mean when they call a legal system patriarchal, using the old common law doctrine of coverture as the paradigm case, and introduces the public/private distinction that runs through everything that follows. Chapter 2 sets out the main schools of thought (formal equality, difference or cultural feminism, dominance theory, and the anti-essentialist and postmodern approaches) and the debates among them. Chapter 3 is devoted to intersectionality: its sources in Black feminist thought, its doctrinal home in employment discrimination law, and the ways courts have and have not absorbed it. The next four chapters apply those tools to particular fields of law. Chapter 4 takes property law, from coverture and the Married Women's Property Acts to the treatment of enslaved women as property and the continuing effects of marital property regimes. Chapter 5 examines the criminal law of rape, and Chapter 6 the criminal law of domestic violence and self-defense, including the argument that the feminist turn toward prosecution has had costs that fall hardest on the women it meant to protect. Chapter 7 addresses contract and work: the family as a site of unenforceable bargains, the labor market as a site of enforceable ones, and the doctrines of employment discrimination that sit between them. Chapter 8 turns to the Constitution, where the law of sex equality and the law of reproduction have run on parallel and sometimes intersecting tracks, and where the Supreme Court's decisions since 2022 have altered the ground. Chapter 9 gathers the principal critiques of feminist legal theory, from within and without, and assesses them. The conclusion asks what survives. The book is written primarily about American law, with comparative notes where other systems handle the same problem in instructive ways. It is an account of a body of legal thought and the doctrines it has shaped; it is not legal advice. A note on terms Some words in this field carry heavy freight, and it helps to be clear about how they are used here. "Patriarchy" is used in its analytical sense: a social and legal order in which authority, property and public power are structured to be held primarily by men, and in which that allocation is reproduced over time by rules and institutions rather than by individual intention alone. One need not accept every claim made under that heading to find the concept useful for describing, say, the common law of the eighteenth century. "Sex" and "gender" are used as the relevant legal materials use them, which is not always consistently; where the distinction matters to a case or an argument, the text says so. "Intersectionality" is used in Crenshaw's original legal sense, as a claim about how discrimination and legal categories interact, rather than in the looser sense it has acquired in public debate. Finally, a word on tone. This book is written by someone who takes feminist legal theory seriously as scholarship, which means neither treating it as settled truth nor dismissing it as ideology. Its best practitioners have always insisted that the proper response to a claim of neutrality is not to reject neutrality as an ideal but to ask, concretely and case by case, whether a particular rule actually delivers it. That is a question any reader can ask. The chapters that follow try to show how. Chapter 1: The Myth of Neutral Law: Patriarchy as a Legal Structure The clearest way to understand what feminist legal theorists mean when they call a legal system patriarchal is to look at a system that announced its own patriarchy openly. The English common law, as received in the American colonies and carried into the law of the early republic, did exactly that. Its central doctrine for married women was coverture, and William Blackstone's description of it in his Commentaries on the Laws of England, published in the 1760s, is still the standard starting point. By marriage, Blackstone wrote, the husband and wife are one person in law; the very being or legal existence of the woman is suspended during the marriage, or at least incorporated into that of the husband, under whose wing, protection and cover she performs everything. That single idea generated a whole architecture of rules. A married woman, a feme covert, could not own personal property in her own right; what she brought into the marriage and what she earned during it belonged to her husband. She could not make a binding contract, because she had no independent legal capacity to be bound. She could not sue or be sued without joining her husband. Her husband controlled and took the income from her real property, though he could not sell it outright without her consent. He was legally responsible for some of her conduct, and in turn was permitted a power of "moderate correction" over her, a subject Chapter 6 takes up. She could not, under Sir Matthew Hale's influential seventeenth-century formulation, be raped by him, because by marriage she had given a consent she could not retract. Coverture matters to feminist legal theory for three reasons. First, it demonstrates that sex-based hierarchy was not merely a matter of social custom that the law passively tolerated. It was written into the law itself, enforced by courts, and defended by the leading jurists of the age as rational and even benevolent. Second, it shows how a hierarchy can be justified in the language of protection: Blackstone insisted that the disabilities a wife lay under were for the most part intended for her protection and benefit, so great a favourite was the female sex of the laws of England. Third, and most important for everything that follows, coverture did not disappear in a single stroke. It was dismantled piece by piece over more than a century, and feminist scholars have argued persuasively that some of its logic survived by changing form. From explicit hierarchy to formal neutrality The American legal system inherited coverture and, for most of the nineteenth century, kept it. What changed first was not the principle but its edges. Beginning with Mississippi in 1839 and New York in 1848, states passed Married Women's Property Acts that allowed wives to hold separate property. New York's 1860 act went further and allowed wives to keep their own earnings. Chapter 4 traces these reforms in detail. Their significance here is that they left the underlying idea of marital unity largely intact while carving out exceptions to it, and courts frequently read the exceptions narrowly. At the same time, the law developed a new justification for treating women differently, one that did not depend on marriage at all. This was the doctrine of separate spheres: the idea that men and women were suited by nature for different domains, men for the market and politics, women for the home. The canonical judicial statement came in Bradwell v. Illinois (1873), in which the Supreme Court upheld Illinois's refusal to admit Myra Bradwell to the bar. The majority opinion rested on a narrow reading of the Privileges or Immunities Clause. But Justice Joseph Bradley's concurrence went further. The natural and proper timidity and delicacy of the female sex, he wrote, evidently unfits it for many of the occupations of civil life, and the paramount destiny and mission of woman are to fulfil the noble and benign offices of wife and mother, which he described as the law of the Creator. Bradwell was followed by Minor v. Happersett (1875), which held that the Fourteenth Amendment did not give women a right to vote, and later by Muller v. Oregon (1908), which upheld a maximum-hours law for women workers at a time when the Court was striking down similar laws for men. Muller is instructive because it was, in its day, a progressive victory. Reformers such as Florence Kelley and the lawyer Louis Brandeis sought protective legislation for working women, and Brandeis's famous brief assembled social science to show that long hours harmed women's health and, through them, the health of the next generation. The Court accepted the argument precisely because it treated women as a class apart. Woman's physical structure and the performance of maternal functions, the opinion said, place her at a disadvantage in the struggle for subsistence, and her physical wellbeing becomes an object of public interest in order to preserve the strength and vigor of the race. Feminist legal historians have spent a great deal of effort on this period because it complicates any simple story. Protective labor laws did protect some women from brutal conditions. They also excluded women from night work, overtime and better-paid trades, and they wrote into constitutional law the premise that a woman's body belonged in some sense to the public and to the future. Whether Muller was a gain or a loss for women depends on which women one asks and over what period, a question that anticipates the later debates between formal equality and difference feminism described in Chapter 2. The twentieth century eventually swept away explicit sex classifications almost entirely. The Nineteenth Amendment was ratified in 1920. The Civil Rights Act of 1964 prohibited sex discrimination in employment. Beginning with Reed v. Reed (1971), the Supreme Court began striking down statutes that treated men and women differently, and by the 1990s it required an "exceedingly persuasive justification" for any sex-based classification. Today it is difficult to find a statute that openly allocates rights by sex. This is where feminist legal theory's distinctive contribution begins. If the law no longer says that women may not own property, practice law or serve on juries, is it now neutral? The field's answer is that formal neutrality is not the same as substantive neutrality, and that the gap between them is where the interesting questions lie. Why formal neutrality is not enough There are several distinct ways in which a legal rule can be formally neutral and still reproduce a hierarchy. Feminist scholars have identified at least four, and it is worth separating them, because critics sometimes treat the argument as a single vague claim that "the law is biased." The first is the problem of the default subject. Many legal standards are built around an imagined typical person, and if that person was imagined as a man, the standard can disadvantage women even when applied to them evenhandedly. The most famous example is the "reasonable man" of tort law, now generally rendered the "reasonable person." Critics argued that the change of name did little if the content of reasonableness was still drawn from male experience. The question became concrete in sexual harassment law. In Ellison v. Brady (1991), the Ninth Circuit adopted a "reasonable woman" standard for judging whether conduct created a hostile environment, reasoning that a sex-blind reasonable person standard tends to be male-biased and to systematically ignore the experiences of women. The Supreme Court did not adopt that standard; in Harris v. Forklift Systems (1993) it spoke of a reasonable person, and later cases refer to a reasonable person in the plaintiff's position. The dispute between those formulations is a small window onto a large question: whether neutrality means applying one standard to everyone or applying a standard that takes the claimant's situation into account. The second is the problem of disparate effect. A rule may apply to everyone but fall predominantly on one group because of the group's different circumstances. A seniority system that uses last-hired, first-fired treats every worker identically, but if one group was excluded from hiring until recently, the rule concentrates layoffs on that group, which is exactly what happened in DeGraffenreid. Constitutional law has largely refused to treat disparate effect alone as discrimination. In Personnel Administrator of Massachusetts v. Feeney (1979), the Supreme Court upheld Massachusetts's lifetime preference for veterans in civil service hiring, even though, because the military had excluded or capped women for decades, the preference went overwhelmingly to men. Discriminatory purpose, the Court held, implies that the legislature selected a course of action at least in part "because of," not merely "in spite of," its adverse effects on women. Title VII, by contrast, recognizes disparate impact claims in employment, a doctrine that originated with race in Griggs v. Duke Power Co. (1971) and was applied to sex in Dothard v. Rawlinson (1977), where height and weight requirements for prison guards were held to screen out women disproportionately. The third is the problem of the unmarked baseline. Some rules seem neutral only because they take an existing distribution for granted. The standard workday and the ideal of the unencumbered worker, available for overtime and relocation, assume that someone else is caring for children and elderly parents. A rule that rewards that pattern treats everyone the same only if one ignores who, in practice, does the caring. Joan Williams made this argument central to her book Unbending Gender (2000), and it underlies the feminist critique of family and medical leave law discussed in Chapter 7. The fourth, and the most ambitious, is the problem of the constructed category. Some feminist theorists argue that the law does not merely respond to sex differences but helps produce them, by defining what counts as a woman's harm, a woman's work or a woman's consent. On this view, which Chapter 2 associates particularly with Catharine MacKinnon, the neutrality of the law is not a partial failure but a mechanism: by treating as natural a social order that men built, it makes that order invisible and therefore unchallengeable. These four claims are of increasing ambition and decreasing consensus. Nearly all legal scholars accept that default-subject problems exist in some doctrines; nearly all accept that disparate effects occur, while disagreeing about whether law should respond to them. The third and fourth claims are far more contested, and much of the criticism discussed in Chapter 9 is aimed at them. The public and the private One distinction runs through almost every chapter of this book and deserves separate treatment here: the line the law draws between public and private life. Classical liberal legal thought treats the public sphere of the state and the market as the proper domain of law, and the private sphere of the family and the home as a zone into which law should intrude only reluctantly. Feminist legal theorists have argued that this distinction, far from being neutral, historically served to shield the exercise of male power over women and children from legal scrutiny. The argument has two branches. The first is historical and concerns violence. When nineteenth-century American courts abandoned the husband's formal right of chastisement, they often did not replace it with vigorous enforcement of the ordinary law of assault. Instead, as Reva Siegel has shown in her study of wife-beating law, they invoked the privacy of the family. In State v. Rhodes (1868), the North Carolina Supreme Court declined to punish a husband who had struck his wife with a switch, reasoning that however great the evils of domestic violence, they were comparatively trivial next to the evils of exposing the family circle to public view. Siegel's term for this pattern, "preservation through transformation," describes how a status hierarchy survives the repudiation of its explicit rules by acquiring a new and more respectable justification. The husband no longer had a right to hit his wife; but the state would, in the name of privacy, decline to stop him. The second branch is economic and concerns work. The law of the market governs paid labor; the law of the family has historically treated unpaid domestic labor as outside the domain of contract and exchange. Courts refused to enforce agreements between spouses to pay for household services, on the theory that such services were already owed as a duty of marriage. The result, in feminist analysis, was that the work predominantly done by women was simultaneously essential and legally invisible. Chapter 7 examines this doctrine and its survival. It would be a mistake to conclude that feminist theory simply rejects privacy. The relationship is more complicated, and the complications are important. Privacy was the constitutional foundation for the right to use contraception in Griswold v. Connecticut (1965) and Eisenstadt v. Baird (1972), and for the abortion right recognized in Roe v. Wade (1973). Many feminists welcomed those decisions while worrying about their basis. MacKinnon argued that grounding abortion in privacy protected women's choices only where they already had the power to make them, and left the private sphere's inequalities untouched. Ruth Bader Ginsburg, before she joined the Court, argued that abortion rights would have rested on firmer ground had they been grounded in sex equality rather than privacy. When the Supreme Court overruled Roe in Dobbs v. Jackson Women's Health Organization (2022), it reasoned from history and tradition rather than from either privacy or equality, and it relied in part on the very common law tradition, including Hale, that feminist historians had spent decades criticizing. Chapter 8 returns to this. A method rather than a doctrine Because feminist legal theory spans such different claims, it is sometimes more useful to think of it as a set of methods than as a body of conclusions. Katharine Bartlett's article "Feminist Legal Methods," published in the Harvard Law Review in 1990, identified three that remain the best short description of how the field works. The first is asking the woman question: examining whether a rule or practice that appears neutral in fact disadvantages women, and if so, how. The question is diagnostic, not presumptive. Sometimes the answer is that a rule has no gendered effect at all. The point is that the question must be asked, because legal reasoning that never asks it will never notice the effects. The second is feminist practical reasoning: an approach to legal decision that attends to the particular facts and relationships of a case rather than subsuming them under general rules as quickly as possible. It is related to, but not identical with, the "ethic of care" associated with Carol Gilligan's psychology, discussed in Chapter 2. The third is consciousness-raising: the practice, borrowed from the women's movement of the 1960s and 1970s, of building knowledge from shared personal experience. In legal scholarship this has taken the form of narrative, personal testimony and attention to how legal events feel to the people subject to them. It is also the method most criticized by those who think legal scholarship should rest on generalizable evidence rather than individual stories. Intersectional scholars later added a fourth method, which Mari Matsuda called looking to the bottom: asking how a rule works for those at the margins of every relevant category, on the theory that a rule which works for them will work for everyone above them, while the reverse is not true. That method is the subject of Chapter 3. None of these methods tells a judge how to decide a given case. What they do is change the questions that get asked, and the history of feminist legal reform suggests that changing the questions has often been enough to change the answers. The next chapter looks at the competing theories that grew up around those questions, and at why their practitioners disagreed so sharply about what equality for women would require. Chapter 2: Equality, Difference, Dominance: The Schools of Feminist Legal Thought Feminist legal theory emerged as a recognizable academic field in the United States in the 1970s and 1980s, when the first large generation of women entered law schools and law faculties. It arrived with a practical agenda already under way. Litigators at the American Civil Liberties Union's Women's Rights Project, led by Ruth Bader Ginsburg, were bringing test cases against sex-based statutes; scholars were documenting the treatment of women in casebooks, courtrooms and the legal profession. The theory grew out of that practice, and its first great debate was a practical one: what does equality between the sexes actually require of the law? Four broad answers emerged. They are often presented as a chronological sequence, with each school replacing the last, but that is misleading. All four remain active, practitioners borrow from more than one, and many of the field's sharpest arguments are between them rather than with outsiders. The differences among them are summarized in Table 1 at the end of this chapter; the sections below explain the reasoning behind each. Formal equality: treat likes alike The first answer, usually called liberal or formal equality feminism, holds that the law should treat women and men the same wherever they are similarly situated, and that nearly always they are. Its central target was the sex-based classification: a statute or rule that explicitly assigned different rights to men and women. Its central tool was the Equal Protection Clause of the Fourteenth Amendment and, in employment, Title VII of the Civil Rights Act of 1964. The litigation strategy associated with this approach was remarkably successful. In Reed v. Reed (1971), Ginsburg's brief persuaded a unanimous Supreme Court to strike down an Idaho statute preferring men over women as administrators of estates. Frontiero v. Richardson (1973) invalidated a military rule under which servicemen automatically received dependency benefits for their wives while servicewomen had to prove their husbands' dependency; a plurality of four Justices would have made sex a suspect classification like race, though that position never gained a fifth vote. Craig v. Boren (1976) established the intermediate scrutiny standard that still governs: sex classifications must serve important governmental objectives and be substantially related to achieving them. United States v. Virginia (1996), in which Justice Ginsburg wrote for the Court, struck down the Virginia Military Institute's exclusion of women and required an "exceedingly persuasive justification" for sex-based government action. A distinctive feature of the formal equality strategy was its deliberate use of male plaintiffs. Craig v. Boren involved an Oklahoma law that let women buy low-alcohol beer at eighteen but required men to wait until twenty-one. Weinberger v. Wiesenfeld (1975) involved a widower denied Social Security survivor's benefits available to widows. The point was strategic and principled at once. Strategically, male plaintiffs made it easier for male judges to see sex classifications as arbitrary. In principle, the argument was that laws premised on women's dependency hurt everyone, because they locked both sexes into roles. The formal equality approach has a clear logic and clear virtues. It is administrable, since courts can readily identify an explicit sex classification. It is consistent with the broader liberal commitment to individual treatment rather than group stereotypes. And it avoids the danger, vividly illustrated by Muller v. Oregon, that any legal recognition of women's differences will be turned against them. Its limits became apparent in two ways. First, it had little to say about rules that were neutral on their face but harmful in effect, the Feeney problem described in Chapter 1. Second, and more sharply, it seemed to require that women be compared to men in order to claim equal treatment, which raised the question of what happens when women and men are not similarly situated. The paradigm case was pregnancy. The pregnancy problem and the equal treatment debate In Geduldig v. Aiello (1974), the Supreme Court upheld California's disability insurance program for workers, which excluded disabilities arising from normal pregnancy. The Court reasoned that the program did not classify by sex at all. It divided potential recipients into two groups, pregnant women and nonpregnant persons, and while the first group was entirely female, the second included members of both sexes. Two years later, in General Electric Co. v. Gilbert (1976), the Court applied the same reasoning to Title VII. Congress responded quickly, passing the Pregnancy Discrimination Act of 1978, which amended Title VII to provide that discrimination on the basis of pregnancy is sex discrimination and that pregnant women must be treated the same as others similar in their ability or inability to work. The Pregnancy Discrimination Act settled the statutory question for employment, but it opened a theoretical one. Some states had gone further than equal treatment. California, for example, required employers to provide up to four months of unpaid pregnancy disability leave with reinstatement, whether or not they provided comparable leave to other disabled workers. Was that permitted, or even required, by the principle of equality? Or was it exactly the kind of special treatment that had justified Muller? Feminist lawyers divided sharply. On one side, scholars such as Wendy Williams argued for an "equal treatment" approach. In her 1982 article "The Equality Crisis," Williams warned that special protections for pregnancy would reinforce the idea that women workers were uniquely costly and uniquely defined by motherhood, and that the history of protective legislation showed where that led. On the other side, scholars such as Linda Krieger and Patricia Cooney argued for a "positive action" or "special treatment" approach, contending that a rule that treats pregnant workers like men treats them like people who never get pregnant, which is not neutrality but a male norm. The Supreme Court resolved the specific dispute in California Federal Savings & Loan Ass'n v. Guerra (1987), holding that California's law was consistent with the Pregnancy Discrimination Act, which Justice Thurgood Marshall's opinion described as a floor beneath which pregnancy disability benefits may not drop, not a ceiling above which they may not rise. The underlying debate did not end. It resurfaced in Young v. United Parcel Service (2015), where the Court developed a framework for deciding when an employer that accommodates some workers must accommodate pregnant ones, and again in the Pregnant Workers Fairness Act, enacted in December 2022, which requires reasonable accommodation of pregnancy-related limitations directly rather than by comparison with other workers. That statute, notably, takes the accommodation side of the old debate; its constitutional validity has itself been litigated on a procedural ground concerning Congress's pandemic-era proxy voting, and a challenge brought by Texas remained pending before the full Fifth Circuit in 2026. Difference and relational feminism The second school takes the pregnancy debate as its starting point and generalizes it. If women differ from men in some respects, the argument goes, then a legal order built entirely on the male pattern will disadvantage them whenever they differ. The solution is not to pretend the differences away but to make the law take account of them without penalizing them. The most influential intellectual source for this position lay outside law, in the developmental psychologist Carol Gilligan's book In a Different Voice (1982). Gilligan argued that the dominant models of moral development, notably Lawrence Kohlberg's, had been built on studies of boys and men and ranked an abstract "ethic of justice," reasoning from rights and rules, as the highest stage of moral thought. Her research with girls and women suggested a different but not inferior pattern, an "ethic of care" that reasoned from relationships, responsibilities and context. Legal scholars seized on the idea. If law embodied the ethic of justice, perhaps a feminist jurisprudence would bring the ethic of care into it. Robin West's "Jurisprudence and Gender," published in the University of Chicago Law Review in 1988, pushed the argument into legal theory. West argued that modern jurisprudence, both liberal and critical, rested on a picture of the human being as fundamentally separate from others. That picture, she contended, described men's experience better than women's, since women's lives are materially and existentially connected to others through pregnancy, breastfeeding and the social expectations of care. A jurisprudence that took women seriously would have to recognize connection as a basic human condition and would value intimacy and care as highly as autonomy. Applications of relational feminism appeared across the law. Leslie Bender argued in tort law for an expanded duty to rescue grounded in care rather than the classical rule that one owes no duty to aid a stranger. Scholars in dispute resolution advocated mediation over adversarial litigation, especially in family law, though other feminists soon warned that mediation could be dangerous for women facing abusive partners, a debate that continues in the design of family courts. The difference approach attracted strong criticism from inside feminism almost immediately. Its critics argued that "women's voice" as Gilligan described it might be the voice of subordination rather than of nature, the characteristic response of people who have had to survive by attending to the needs of the powerful. They also argued that valuing care in the abstract risked romanticizing the very domestic role that had excluded women from public life, and that the "women" in these theories were implicitly white and middle class. Joan Williams, who argued that the "different voice" risked reproducing Victorian ideals of domesticity, called for attention to how work and family were structured rather than to supposed differences in women's character. Dominance theory The third school, associated above all with Catharine MacKinnon, rejects the whole framework of sameness and difference. Its central claim is that the question "are women the same as or different from men?" is the wrong question, because it takes men as the measure either way. Under the sameness approach, women must be like men to be treated equally. Under the difference approach, women are measured by how they deviate from men. In both, MacKinnon argued, "man has become the measure of all things." The alternative is to ask about power. On MacKinnon's account, gender is not a set of natural differences that the law then treats well or badly. It is a social hierarchy of male dominance and female subordination, and the differences we observe are largely produced by that hierarchy. The central mechanism of the hierarchy, in her early work, is sexuality: the social organization of sexuality around male desire and female availability. "Man fucks woman; subject verb object," as she put it in a deliberately jarring passage of her 1982 article in Signs, "Feminism, Marxism, Method, and the State." Equality, on this view, requires not identical treatment or accommodation of difference but the dismantling of the hierarchy itself. MacKinnon's name for the resulting test is anti-subordination: a practice is discriminatory if it contributes to the subordinate status of women as a group. The dominance approach has had more concrete legal influence than any other feminist theory, principally through sexual harassment law. In Sexual Harassment of Working Women (1979), MacKinnon argued that unwanted sexual advances and a sexualized hostile environment at work were not private misconduct but sex discrimination under Title VII, because they reflected and enforced women's subordinate status in employment. Lower courts had begun moving toward similar conclusions, and in Meritor Savings Bank v. Vinson (1986) the Supreme Court unanimously agreed that a hostile work environment could violate Title VII. MacKinnon helped write the brief for Mechelle Vinson in the Supreme Court. Sexual harassment law is now so embedded in American workplaces that it is easy to forget it was a theoretical innovation. The approach's other major campaign was less successful. With Andrea Dworkin, MacKinnon drafted model ordinances treating pornography as a civil rights violation, defined as the graphic sexually explicit subordination of women, and giving those harmed by it a cause of action. Indianapolis adopted a version in 1984. In American Booksellers Ass'n v. Hudnut (1985), the Seventh Circuit struck it down under the First Amendment. Judge Frank Easterbrook's opinion accepted the ordinance's premises about pornography's harms and held that this was precisely the problem: the ordinance regulated speech on the basis of its viewpoint about women, and the First Amendment does not permit the state to establish an approved view of women. The Supreme Court summarily affirmed. Canada took a different path. In R. v. Butler (1992), the Supreme Court of Canada interpreted the obscenity provision of the Criminal Code in terms that drew on harm to women, an outcome widely attributed to the influence of the dominance approach, and one whose later enforcement against gay and lesbian bookstores became a cautionary tale for its critics. Dominance theory's critics are numerous and serious, and they include many feminists. They argue that it treats women primarily as victims and denies their sexual agency; that its account of sexuality is totalizing and unfalsifiable, since any apparent consent can be recharacterized as the product of domination; and that its campaigns against pornography allied feminism with social conservatives and the censorious state. The "sex wars" of the 1980s, in which pro-sex or sex-positive feminists opposed the anti-pornography movement, were a defining split in the field and are revisited in Chapter 9. Anti-essentialism and postmodern feminism The fourth set of approaches is less a single school than a critique of the other three. It arose in the late 1980s and 1990s from two sources: Black feminist and other women-of-color scholarship, and postmodern philosophy. The anti-essentialist critique holds that all three earlier schools spoke of "women" as if women shared a single experience of gender, and that the experience they described was that of white, middle-class, heterosexual women. Angela Harris's article "Race and Essentialism in Feminist Legal Theory," published in the Stanford Law Review in 1990, made this case with particular force against MacKinnon and West. Harris argued that MacKinnon's claim to describe women's experience "as women" treated race as an add-on, so that Black women's experience appeared only as a more intense version of white women's experience. Yet for Black women, Harris pointed out, the history of rape in America was bound up with the history of slavery, in which enslaved women could not legally be raped at all, and with lynching, in which false accusations of rape against Black men were used as instruments of racial terror. A theory that treated rape as the paradigm of male power over women without accounting for that history could not describe Black women's experience; it might even obscure it. Harris's broader point was that gender is always experienced in combination with race, class and sexuality, and that a theory which strips those away does not find women's true essence. It finds the experience of the women who were not stripped of anything. This critique converged with Crenshaw's work on intersectionality, discussed in the next chapter, and with the writing of Patricia Williams, whose book The Alchemy of Race and Rights (1991) used personal narrative to challenge the critical legal studies movement's dismissal of rights as illusory. Williams argued that for Black Americans, who had been excluded from rights altogether, the formal status of rights-holder carried a meaning that white critics could not see. Postmodern feminist legal theory, represented by scholars such as Mary Joe Frug, went further, questioning the stability of the category "woman" itself. In "A Postmodern Feminist Legal Manifesto," published posthumously in the Harvard Law Review in 1992 after Frug's murder the previous year, she argued that legal rules help to produce the meaning of the female body, in ways that are multiple and sometimes contradictory, and that feminist legal strategy should aim to disrupt rather than fix those meanings. Later work in this vein connected feminist legal theory with queer theory and with the legal treatment of transgender people. The anti-essentialist turn has been the most transformative internal development in feminist legal thought, and it produced its own critics. Some feared that if "women" were endlessly subdivided, feminism would lose the collective subject in whose name it made claims. Janet Halley, in Split Decisions (2006), proposed that legal scholars sometimes "take a break from feminism," arguing that its insistence on analyzing everything through male dominance and female subordination blinded it to other forms of power and to harms that did not fit that structure. Others worried that postmodern skepticism about categories was incompatible with antidiscrimination law, which requires categories to function. The schools compared It would be wrong to present these approaches as a menu from which a reader chooses one. The history of the field shows constant borrowing. Sexual harassment law, the triumph of dominance theory, was won in a Title VII framework built by formal equality litigators. The Pregnant Workers Fairness Act reflects the accommodation arguments of difference feminism, but it passed with bipartisan support as a matter of workplace fairness rather than theory. Intersectional analysis now appears in all of the schools. Still, the approaches start from different questions and tend toward different remedies, and those differences recur in every chapter that follows. Table 1 sets out their central commitments side by side. Table 1. Principal schools of feminist legal theory compared. School Central question Key figures Characteristic remedy Principal criticism Formal equality Are women and men treated the same? Ruth Bader Ginsburg, Wendy Williams Strike down sex classifications Takes men as the baseline; ignores effects Difference / relational Does law value women's distinct experience? Carol Gilligan, Robin West Accommodation; valuing care Risks reinforcing stereotypes Dominance Does the practice sustain male power? Catharine MacKinnon, Andrea Dworkin Anti-subordination; harassment law Denies agency; unfalsifiable Anti-essentialist / intersectional Which women does the analysis describe? Kimberlé Crenshaw, Angela Harris Attention to multiple, combined axes May fragment the category "women" Postmodern How does law construct "woman"? Mary Joe Frug, Janet Halley Disrupting fixed categories Hard to square with rights claims The next chapter examines the fourth row of that table in detail, because intersectionality is both the most influential contribution of recent feminist legal thought and the one most often misunderstood. Chapter 3: Intersectionality: The Single-Axis Problem Intersectionality is now one of the most widely used words in the humanities and social sciences, and one of the most heavily contested in public life. Its legal origin is narrower and more precise than either its admirers or its critics usually acknowledge. Kimberlé Crenshaw introduced the term in "Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics," published in the University of Chicago Legal Forum in 1989. The article's subtitle announces three targets, and the first of them is doctrine. Crenshaw's claim was that American antidiscrimination law had been built on a "single-axis framework" that treated race and sex as separate and mutually exclusive categories of experience, and that this framework systematically failed the people whose experience of discrimination was shaped by both at once. The metaphor and the argument Crenshaw's best-known image is a traffic intersection. Discrimination, like traffic, may flow in one direction or another. If a Black woman is harmed because she is standing in the intersection, her injury could have been caused by cars traveling from any direction, and sometimes from all of them. The law's response, however, was to ask her to prove which single car hit her, and to deny relief if she could not show that she was harmed exactly as a white woman or a Black man would have been. The metaphor is often quoted and less often carried to its conclusion, which is that the problem is not merely one of missing categories. Crenshaw argued that the single-axis framework took as its paradigm of discrimination the experience of the most privileged members of each group. Sex discrimination was understood through the experience of white women, who were disadvantaged by sex but not by race. Race discrimination was understood through the experience of Black men, who were disadvantaged by race but not by sex. Because the law's picture of each form of discrimination was drawn from those whose experience was "pure," people who were burdened on more than one axis appeared to the law as imperfect or complicated claimants. They were marginalized not in spite of antidiscrimination law but, in part, by its structure. This is why intersectionality is properly understood as a critique of neutrality and not merely as an argument for adding more protected categories. The single-axis framework looked neutral: it treated every race claim alike and every sex claim alike. What it concealed was that the definition of a "race claim" and a "sex claim" was itself built around particular people. The cases Crenshaw used Crenshaw built her argument around three federal employment discrimination cases, and they remain the clearest illustrations of the problem. The first was DeGraffenreid v. General Motors, decided by the federal district court in the Eastern District of Missouri in 1976 and described in the introduction to this book. The court refused to allow the plaintiffs to combine race and sex claims, writing that they should not be allowed to combine statutory remedies to create a new "super-remedy" that would give them relief beyond what the drafters of the relevant statutes intended. The plaintiffs could bring a sex claim, which failed because General Motors had hired white women before 1964, or a race claim, which the court suggested be consolidated with another pending race discrimination suit against the company. What they could not do was claim that they had been harmed as Black women. The second was Moore v. Hughes Helicopters, Inc., decided by the Ninth Circuit in 1983. Tommie Moore, a Black woman, alleged that the company discriminated against women and Black workers in promotion to supervisory and upper-level craft jobs. She sought to represent a class of female employees. The court upheld the denial of class certification in part because she had claimed discrimination as a Black woman rather than as a woman, and so, the court reasoned, it was not clear she could adequately represent white women. Her statistical evidence was also compared against all women rather than Black women specifically, which obscured the disparity she was trying to prove. Crenshaw's point was that the case treated Black women's claims as a subset too particular to stand for women generally, while DeGraffenreid had treated them as a combination too novel to be recognized at all. Either way they lost. The third was Payne v. Travenol Laboratories, decided by the Fifth Circuit in 1982, in which Black women plaintiffs were not permitted to represent Black men in a race discrimination class action, apparently because their claims were seen as too distinct from those of men. The result, Crenshaw argued, was that Black women could not speak for women, could not speak for Black people, and could not speak for themselves as a combined group. There was, however, a contrary line. In Jefferies v. Harris County Community Action Ass'n, decided in 1980, the Fifth Circuit held that Black women were a protected subgroup under Title VII and that an employer could be liable for discriminating against Black women even if it did not discriminate against Black men or white women. The court reasoned that the use of the word "or" in the statute's list of protected characteristics, race, color, religion, sex or national origin, evidenced Congress's intent to prohibit employment discrimination based on any or all of those characteristics. The Ninth Circuit reached a similar conclusion in Lam v. University of Hawaii (1994), involving a woman of Vietnamese descent denied a law school position, and expressly warned against the tendency to bisect a person's identity at the intersection of race and gender, which it said often distorts or ignores the particular nature of the plaintiff's experience. The principal cases on both sides of the doctrinal question are set out in Table 2. Table 2. Leading federal cases on combined race and sex discrimination claims. Case Court and year Plaintiff's claim Outcome on the intersectional question DeGraffenreid v. General Motors E.D. Mo., 1976 Seniority layoffs of Black women Combined claim not recognized Jefferies v. Harris County Community Action Ass'n 5th Cir., 1980 Discharge and promotion of a Black woman Black women a protected subgroup Payne v. Travenol Laboratories 5th Cir., 1982 Class action by Black women Could not represent Black men Moore v. Hughes Helicopters 9th Cir., 1983 Promotion of a Black woman Could not represent all women Lam v. University of Hawaii 9th Cir., 1994 Hiring of an Asian woman Race and sex must be considered together Precursors: Jane Crow and the Combahee River Collective Crenshaw named the idea, but she did not originate the insight, and she has always been clear about that. Two precursors deserve particular attention because they show that the intersectional critique was present at the beginning of modern sex discrimination law, not grafted onto it later. The first is Pauli Murray. A lawyer, poet and later Episcopal priest, Murray coined the phrase "Jane Crow" to describe the combined effect of racism and sexism on Black women. In an article co-authored with Mary Eastwood, "Jane Crow and the Law: Sex Discrimination and Title VII," published in the George Washington Law Review in 1965, Murray argued that the Equal Protection Clause should be applied to sex classifications by analogy to race, a strategy that Ginsburg's litigation would later adopt. In White v. Crook (1966), a case in which Murray helped prepare the American Civil Liberties Union's arguments, a three-judge federal court struck down Alabama's exclusion of women from jury service in a case that also challenged the exclusion of Black citizens from juries in Lowndes County. The case joined race and sex in a single challenge. Ginsburg later listed Murray, along with Dorothy Kenyon, as co-authors of her brief in Reed v. Reed, in acknowledgment of their earlier work. The legal historian Serena Mayeri's book Reasoning from Race (2011) traces how the analogy between race and sex that Murray developed shaped, and also constrained, feminist constitutional litigation. The second is the Combahee River Collective, a group of Black feminists in Boston whose 1977 statement argued that the major systems of oppression, racial, sexual, heterosexual and class, are "interlocking," and that the synthesis of these oppressions creates the conditions of Black women's lives. The Combahee statement was a political manifesto rather than a legal argument, but its idea that oppressions combine rather than add up is the idea Crenshaw later translated into the language of antidiscrimination doctrine. Earlier still, Black women writers and activists, including Anna Julia Cooper in the 1890s and Sojourner Truth in the antebellum period, had pointed out that the public cause of "women" and the public cause of Black people each tended to forget Black women. Structural, political and representational intersectionality In a second article, "Mapping the Margins: Intersectionality, Identity Politics, and Violence Against Women of Color," published in the Stanford Law Review in 1991, Crenshaw extended the concept beyond employment law and distinguished three forms of intersectionality. Structural intersectionality describes how the position of women of color at the intersection of race, gender and class makes their experience of violence, and of the institutions that respond to it, qualitatively different from that of white women. Crenshaw's field research in battered women's shelters in Los Angeles found that many of the women seeking help were poor, unemployed or underemployed, responsible for children and, in some cases, limited in their English. Interventions designed on the assumption that the main barrier facing an abused woman was the abuse itself did not address the barriers of poverty, housing and language that shaped these women's options. Her most pointed example concerned immigration law. The Immigration Marriage Fraud Amendments of 1986 required a foreign spouse to remain "properly" married for two years before applying for permanent residence. An immigrant woman married to an abusive citizen or resident was therefore faced with a choice between remaining with her abuser or risking deportation. Congress created a waiver for battered spouses in 1990, but, as Crenshaw documented, its evidentiary requirements and the absence of language access meant that many women could not use it. The Violence Against Women Act of 1994 later created a self-petitioning process that allowed abused immigrant spouses to apply for status without their abuser's cooperation, a reform that intersectional advocacy helped to produce. Political intersectionality describes how women of color are situated within at least two subordinated groups that frequently pursue conflicting political agendas. Crenshaw gave the example of the Los Angeles Police Department's reluctance to release data on domestic violence arrests broken down by race, a reluctance shared by some advocates, who feared that the data would be used to reinforce stereotypes about violence in minority communities. The result, she argued, was that the particular interests of women of color were suppressed by both feminist and antiracist politics, each of which found it strategically inconvenient to acknowledge them. Representational intersectionality concerns cultural images of women of color and the way debates about those images can marginalize the women themselves. Crenshaw's example was the 1990 obscenity prosecution of the rap group 2 Live Crew, in which, she argued, both the prosecution and the group's defenders discussed race and sexuality in ways that left the misogyny directed at Black women in the lyrics largely out of view. How courts have (and have not) absorbed intersectionality More than three decades after Crenshaw's articles, the doctrinal picture remains mixed. Most federal courts of appeals that have considered the question accept that a plaintiff may bring a claim of discrimination on the basis of a combination of protected characteristics under Title VII, following Jefferies. The Equal Employment Opportunity Commission's compliance guidance has recognized "intersectional discrimination" for years. In practice, however, intersectional claims face difficulties. The first is proof. Many employment discrimination cases rely on comparator evidence, showing that a similarly situated person outside the protected class was treated better. For a Black woman, who is the right comparator? A white man, a white woman, a Black man? If an employer promotes white women and Black men, a claim framed on either axis alone may fail, and statistical evidence broken down to the level of Black women in a particular job category may involve samples too small to be meaningful. The second is the problem of "sex-plus" and characteristics that are not themselves protected. Courts developed the sex-plus doctrine to handle cases such as Phillips v. Martin Marietta Corp. (1971), in which the Supreme Court held that an employer could not refuse to hire women with preschool-age children while hiring men with such children. But when the "plus" characteristic is something like hair, the doctrine has been less hospitable. In Rogers v. American Airlines (1981), a federal district court in New York upheld the airline's ban on all-braided hairstyles against a Black woman's challenge, reasoning that the policy applied equally to all races and both sexes and that hairstyle was an easily changed characteristic. In EEOC v. Catastrophe Management Solutions (2016), the Eleventh Circuit rejected a challenge to an employer's rescission of a job offer to a Black woman who declined to cut her locs, holding that Title VII protected immutable characteristics of race, not cultural practices associated with it. These cases prompted legislative rather than judicial responses. California enacted the first CROWN Act, prohibiting discrimination based on natural hair texture and protective hairstyles, in 2019, and many other states and localities followed. Proposals for a federal CROWN Act passed the House of Representatives more than once but did not become law. The third difficulty is that the Supreme Court has never squarely addressed intersectional claims. Its most important modern Title VII decision on the meaning of sex, Bostock v. Clayton County (2020), adopted a textualist but-for test: an employer discriminates because of sex if it would have treated the employee differently had the employee's sex been different, all else being equal. Some scholars have argued that this logic supports intersectional claims, since discrimination against a Black woman that would not have occurred had she been a Black man is, under a but-for test, discrimination because of sex. In Muldrow v. City of St. Louis (2024), the Court held that a plaintiff challenging a job transfer under Title VII need show only "some harm" to a term or condition of employment, not a significant one, a ruling that lowered a barrier affecting many discrimination claims. And in Ames v. Ohio Department of Youth Services (2025), the Court held unanimously that plaintiffs who are members of a majority group need not meet a heightened "background circumstances" standard, emphasizing that Title VII protects individuals rather than groups and applies the same test to all. That emphasis on the individual claimant cuts in more than one direction for intersectional theory, a point taken up in Chapter 9. Comparative notes Other legal systems have faced the same problem and handled it differently, which is instructive for American readers. The United Kingdom's Equality Act 2010 included a section, section 14, that would have allowed claims of "combined discrimination" on the basis of two protected characteristics together. The government chose not to bring it into force, citing the burden on business, and it has remained dormant. Canada's approach under section 15 of the Charter of Rights and Freedoms, beginning with Andrews v. Law Society of British Columbia (1989), embraced a substantive rather than formal conception of equality, and Canadian courts have been somewhat more willing to consider the combined effect of grounds. South Africa's Constitution of 1996 prohibits unfair discrimination "on one or more" listed grounds, language that explicitly contemplates multiple and intersecting grounds. At the international level, the United Nations Committee on the Elimination of Discrimination against Women, which oversees the Convention on the Elimination of All Forms of Discrimination against Women, has described intersectionality as a basic concept for understanding the scope of states' obligations. The United States signed that Convention in 1980 but has never ratified it. What intersectionality claims and does not claim Because the word has traveled so far from its origin, it is worth stating what the legal concept does and does not assert. It asserts that discrimination may operate on the basis of combined characteristics in ways that analysis of each characteristic separately will miss. It asserts that legal categories built around the experience of the most advantaged members of a group may fail the less advantaged. And it asserts, as a methodological matter, that looking at how a rule affects those at the intersection of several disadvantages is a good test of whether the rule is working. It does not, in its legal form, assert a ranking of oppressions, a claim that identity determines truth, or a proposition that members of some groups should be treated worse than others. Critics who attack those ideas may be attacking positions that some people hold, but they are not attacking what Crenshaw argued in the law reviews. Equally, defenders of intersectionality sometimes claim more for it than the legal concept can deliver. As a critique of the single-axis framework, it has been powerful and partly successful. As a guide to how courts should actually adjudicate claims involving several characteristics, or how legislatures should design remedies, it leaves many questions open. Those questions are among the most important in the field, and Chapter 9 returns to them. The chapters that follow apply the intersectional method to the core fields of private and criminal law. They begin with property, where the law's treatment of women and its treatment of race were entangled from the beginning. Chapter 4: Property: Coverture, Its Undoing, and What Survived Property law is the natural place to begin applying feminist method to particular fields, because property was where the common law's subordination of women was most explicit and where its dismantling was most gradual. It is also where the entanglement of sex and race in American law is starkest. For more than two centuries, American property law determined not only what women could own but, for enslaved women, whether they were owners or owned. A feminist account of property that looks only at married white women's rights tells half the story, and the intersectional method developed in the previous chapter insists on both halves. Coverture and marital property Under the English common law system of marital property that most American states inherited, marriage transferred control of a wife's property to her husband. Her personal property, her money, goods and chattels, became his outright. Her real property, her land, remained hers in title, but he controlled it and took its rents and profits during the marriage. If a child was born alive, the husband acquired a life interest in her land, called curtesy, that continued after her death. The wife, for her part, received dower: a life interest in one-third of the land her husband had owned during the marriage, which took effect only if she outlived him. Her earnings from any work she did belonged to him. Two features of this system deserve emphasis. First, it was not a system of shared ownership. The husband did not hold property on behalf of the marital unit in any legally enforceable sense; he owned or controlled it, and his creditors could reach it to pay his debts. Second, wealthy families had long devised ways around it. Courts of equity would enforce trusts under which property was held for a married woman's "separate use," shielding it from her husband and his creditors. These arrangements were available to the propertied classes and their lawyers; for the great majority of women, who had no trust settlements, coverture operated in full. The Married Women's Property Acts began to change this. Mississippi's act of 1839, often described as the first, was narrow and was shaped in part by concerns about slaveholding families: it allowed married women to hold property, including enslaved people, separately from their husbands, protecting family wealth from a husband's creditors. New York's act of 1848 was broader, allowing a married woman to hold real and personal property as if she were single, and its 1860 Earnings Act allowed her to keep her own wages and to carry on a trade on her own account. By the end of the nineteenth century, every state had some version of these reforms. Feminist legal historians have argued that the acts were less transformative than their text suggests, and the most influential version of that argument comes from Reva Siegel. In "The Modernization of Marital Status Law: Adjudicating Wives' Rights to Earnings, 1860–1930," published in the Georgetown Law Journal in 1994, Siegel examined how courts interpreted the earnings statutes. She found that they drew a sharp line between work a wife performed for third parties, which the statutes now let her keep, and work she performed in the household or on the family farm or in the husband's business, which courts continued to treat as owed to him under the marriage relationship. A wife who took in boarders, nursed a neighbor or sold butter might find that a court treated the income as her husband's, because the work was done in the home and so was not "on her separate account." The upshot was that the reform freed the wages of wives who worked outside the home, disproportionately poorer women, while leaving the far larger volume of household labor, including much income-generating labor, within the husband's domain. The logic of coverture was not abolished; it was relocated into the distinction between market work and family work that Chapter 7 examines. Common law and community property The United States has two basic systems of marital property, and the contrast between them illustrates how structural choices that look technical can have large distributive effects. Most states follow the common law "separate property" model, in which each spouse owns what he or she earns or acquires in his or her own name. Nine states, including California, Texas, Arizona and Louisiana, follow a community property model derived from Spanish and French civil law, in which earnings and acquisitions during the marriage belong to both spouses equally, regardless of who earned them. On its face, the common law separate property system is sex-neutral: each spouse keeps his or her own. In a society where husbands typically earned wages and wives typically did unpaid household work, however, "each keeps his own" meant that the husband owned the family's accumulated wealth and the wife owned very little. The community property system recognized, at least in principle, that a wife's household labor contributed to the family's acquisitions. Yet community property states long gave husbands sole management and control of community property. Louisiana's "head and master" statute, which gave a husband unilateral authority to dispose of jointly owned property without his wife's consent, survived until the Supreme Court held it unconstitutional in Kirchberg v. Feenstra (1981). Joan Feenstra's husband had mortgaged their jointly owned home without her knowledge to pay his lawyer after she filed a criminal complaint against him. The Court held that the statute's sex-based classification could not survive heightened scrutiny. The divorce reforms of the 1970s and 1980s brought the two systems closer together. Common law states adopted "equitable distribution," under which courts divide marital property at divorce according to fairness rather than title, and most consider non-monetary contributions such as homemaking and childcare. Community property states moved to equal management. By the late twentieth century, the formal rules looked roughly neutral and roughly similar. Table 3 summarizes the main stages of the transformation. Table 3. Stages in the legal treatment of married women's property in the United States. Period Governing regime Wife's control of property Wife's control of earnings Colonial era to c. 1840 Common law coverture Husband controls land; owns personal property Belong to husband c. 1840 to 1900 Married Women's Property Acts Separate property recognized, read narrowly Market wages increasingly hers; household work not 1900 to 1970s Separate property; community property with husband as manager Formally separate; husband often manages jointly held assets Her own, but household work uncompensated 1970s onward Equitable distribution; equal management Formally equal Equal; homemaking counted at divorce in most states What formal equality left behind Feminist analysis of contemporary property law focuses on what the formal equality of the last row conceals. Three problems have received particular attention. The first is the treatment of human capital at divorce. For many families, the most valuable asset acquired during a marriage is not a house or a retirement account but the earning capacity of one spouse, built up through education, experience and career advancement while the other spouse took on a larger share of childcare and household work. Most states do not treat professional degrees or enhanced earning capacity as divisible property. New York was an exception for a time: in O'Brien v. O'Brien (1985), its Court of Appeals held that a medical license acquired during the marriage was marital property subject to equitable distribution, but the legislature later amended the statute, effective in 2016, to provide that enhanced earning capacity is not to be treated as marital property. Feminist scholars such as Joan Williams have argued that the typical divorce leaves the spouse who invested in the family, usually the wife, without a share of the family's most valuable asset, while spousal support has been sharply curtailed in many states. Critics respond that treating a person's future earnings as property raises problems of valuation and of personal liberty, and that alimony and child support are the proper tools. The second is the economic consequences of divorce more broadly. Lenore Weitzman's book The Divorce Revolution (1985) claimed that women's standard of living fell by 73 percent in the first year after divorce while men's rose by 42 percent. Those figures were widely cited, including in court decisions and legislative debates, before a reanalysis of her data by Richard Peterson, published in the American Sociological Review in 1996, found a calculation error and estimates closer to a 27 percent decline for women and a 10 percent increase for men. The episode is worth noting for two reasons. The corrected figures still show a substantial gender gap in the consequences of divorce, which later studies have broadly confirmed. And the original error became, for critics of feminist legal scholarship, an example of advocacy running ahead of evidence, a charge Chapter 9 considers. The third is Martha Fineman's argument, developed in The Neutered Mother, the Sexual Family and Other Twentieth Century Tragedies (1995) and The Autonomy Myth (2004), that the reforms replaced a hierarchical family with a formally egalitarian one that assumed away the unequal burden of dependency. Children, the elderly and the ill need care; someone provides it; and a property regime that treats spouses as equal, autonomous economic actors at divorce does not account for the fact that one of them has spent years in a role that the market does not reward. Fineman's later "vulnerability theory" generalizes the argument, contending that all people are dependent at some point and that the law should be organized around that universal vulnerability rather than around the fiction of the autonomous individual. Women as property: slavery and the law of descent The intersectional method requires attention to a different and darker chapter of American property law. Under the law of slavery, enslaved women were not subject to coverture, because they could not legally marry. They were themselves property, and the law of property determined the status of their children. In 1662, the Virginia General Assembly enacted a statute providing that children born in the colony would be bond or free according to the condition of the mother. This departed from the English common law rule, under which a child's status followed the father's. The rule, known by the Latin phrase partus sequitur ventrem, meant that the children of enslaved women were enslaved, regardless of who their fathers were. Because white men who fathered children by enslaved women were thereby adding to their own property rather than acknowledging heirs, the law turned sexual exploitation into a mechanism of wealth accumulation. Every slaveholding colony and state adopted the rule. Legal scholars including Adrienne Davis have argued that the rule made enslaved women's reproductive capacity itself an asset of the slaveholding economy, and that the law's refusal to recognize the rape of enslaved women was of a piece with it. The case of State v. Celia, tried in Missouri in 1855, in which an enslaved teenager named Celia was convicted and executed for killing the man who owned her and had repeatedly raped her, shows how completely the law denied enslaved women the protections it offered to others. The court refused to instruct the jury that the statute permitting a woman to use force to defend herself against rape could apply to her. Cheryl Harris's article "Whiteness as Property," published in the Harvard Law Review in 1993, generalized the connection between race and property law. Harris argued that American law had not only treated Black people as property but had come to treat whiteness itself as a form of property: a status with legally protected value, including the right to exclude. Her argument has been influential well beyond property law, and it has been contested, but it is relevant here for a specific reason. It shows that the property regime which feminist historians describe as the vehicle of male dominance over wives was also, and simultaneously, the vehicle of white dominance over enslaved people of both sexes. A white woman under coverture was deprived of property rights by her marriage; an enslaved Black woman was deprived of the capacity to marry and of any property in herself. The two positions were not points on a single scale of disadvantage. They were structurally different, and a theory that described only the first would misdescribe the law. Property after formal equality: housing and wealth Contemporary feminist and intersectional analysis of property focuses less on title rules, which are now formally neutral, than on the distribution of property and the legal processes by which people gain or lose it. Housing is the most studied area. Matthew Desmond's ethnography Evicted (2016), based on fieldwork in Milwaukee, found that women from Black neighborhoods were evicted at strikingly higher rates than other tenants, and Desmond concluded that eviction played a role in the lives of poor Black women comparable to the role incarceration played in the lives of poor Black men. His research also documented how nuisance ordinances, which penalize landlords for repeated police calls to a property, could lead to the eviction of domestic violence victims who called the police for help. Those ordinances became a focus of litigation and reform; the American Civil Liberties Union challenged several, and the 2022 reauthorization of the federal Violence Against Women Act added protections for the right of residents to seek police or emergency assistance from their homes without being penalized for it. A second area is credit. Until the Equal Credit Opportunity Act of 1974, lenders routinely refused credit to married women in their own names, required a husband's signature for a wife's application, or discounted a wife's income on the assumption that she would leave work to have children. The Act prohibited discrimination in credit on the basis of sex and marital status, and was amended in 1976 to add race and other characteristics. It is a clear case where legislation addressed a disparity that neutral market practice had produced. The later subprime lending crisis, however, showed how neutral-seeming credit markets could still produce concentrated harm: studies by researchers at the Consumer Federation of America and elsewhere found that Black women were disproportionately likely to receive subprime mortgages, including when their credit profiles would have qualified them for better terms. A third is heirs' property, land passed down without a will and held by many descendants as tenants in common. Heirs' property is concentrated among Black landowners in the rural South, and its legal structure made it vulnerable to forced partition sales, in which any co-owner, including a speculator who bought a small share, could force the sale of the whole parcel. The Uniform Partition of Heirs Property Act, promulgated by the Uniform Law Commission in 2010 and adopted by a substantial number of states since, gives co-owners rights of first refusal and requires courts to consider non-economic factors before ordering a sale. Its relevance to feminist analysis is that the problem often fell on older women who had inherited a share of family land and lived on it without clear title. The lesson of property The history of women's property rights is the clearest demonstration in American law of Siegel's "preservation through transformation." Each reform removed an explicit disability, and each time a new mechanism, sometimes narrow judicial interpretation, sometimes a formally neutral distinction between market work and household work, sometimes simple inequality in what each person brought to the table, preserved much of the old distribution. The intersectional view adds that the history was never one history. For white women, the story is of a gradual emancipation from a status that made them their husbands' dependents. For enslaved Black women, the story begins with a status that made them other people's property and made their children part of that property. Any feminist account of property that tells only the first story reproduces, in its own analysis, the single-axis error that Crenshaw identified in employment law. Chapter 5: Criminal Law I: Rape, Consent, and the Architecture of Disbelief No area of criminal law has been more thoroughly reshaped by feminist argument than the law of rape, and none shows more clearly how a body of rules can look neutral while encoding a particular point of view. The traditional common law of rape did not say that women were not to be believed. It did not need to. It achieved that result through a set of evidentiary and substantive requirements, each of which could be defended in isolation as a safeguard for the accused, and which together made conviction depend on the victim's conformity to a narrow picture of how a "real" rape happens and who a credible victim is. The traditional structure The common law defined rape as carnal knowledge of a woman, not the defendant's wife, forcibly and against her will. Each element carried weight. "Not his wife" encoded the marital exemption, whose classic statement is Sir Matthew Hale's in his History of the Pleas of the Crown, published posthumously in 1736. A husband could not be guilty of rape upon his wife, Hale wrote, because by their mutual matrimonial consent and contract the wife had given herself up in this kind unto her husband, which she could not retract. Hale gave no legal authority for the proposition, but it was repeated by treatise writers and courts for three centuries. "Forcibly" and "against her will" were interpreted to require not only that the woman did not consent but that the man used force, and that she resisted it. Some courts required "utmost resistance." In Brown v. State, decided by the Wisconsin Supreme Court in 1906, a sixteen-year-old testified that she had screamed and tried to get away as hard as she could; the court reversed the conviction, finding her resistance insufficient. Later courts softened the standard to "reasonable" or "earnest" resistance, but the requirement that the victim demonstrate her unwillingness through physical struggle persisted well into the twentieth century. Around these substantive elements the law built evidentiary safeguards. Many jurisdictions required corroboration of the victim's testimony by independent evidence, a requirement not imposed for other crimes. Many allowed the defense to introduce evidence of the victim's prior sexual history on the theory that an unchaste woman was more likely to have consented. Many admitted evidence of whether she had made a "prompt complaint," treating delay as evidence of fabrication. And judges commonly instructed juries in words derived from Hale that rape is an accusation easily to be made and hard to be proved, and harder to be defended by the party accused, though never so innocent, and that the victim's testimony should therefore be examined with caution. Feminist scholars did not argue that each of these rules was irrational. The criminal law rightly protects defendants against wrongful conviction, and rape cases often turn on the conflicting testimony of two people. Their argument was structural. Taken together, the rules reflected a specific fear, that women would lie about rape, and a specific image of rape, a violent attack by a stranger on a chaste woman who fought back. Cases that fit the image could be prosecuted. Cases that did not, which is to say the great majority, because most rapes are committed by acquaintances without weapons or visible injuries, could not. Estrich and "real rape" The most influential statement of this critique came from Susan Estrich, a Harvard law professor who opened her 1986 article "Rape," published in the Yale Law Journal and expanded into the book Real Rape (1987), with an account of her own rape by a stranger in 1974. Her case, she observed, was taken seriously by police because it fit the paradigm. Her argument was that the law recognized an "aggravated" rape, involving a stranger, weapons or beatings, while treating the "simple" rape by an acquaintance, date or partner, without extrinsic violence, as something less than rape or not rape at all. Estrich's central case was Rusk v. State, decided by Maryland's courts in 1979 through 1981. The victim met Edward Rusk at a bar, gave him a ride home, and went up to his apartment after he took her car keys. She testified that she was afraid, that she asked him to let her leave, and that she submitted after he began to choke her lightly. Maryland's intermediate appellate court reversed the conviction, finding insufficient evidence that her fear was reasonable. The Court of Appeals, Maryland's highest court, reinstated the conviction by a divided vote, but the dissenters argued that she should have resisted more. Estrich's point was that the debate over whether her resistance was adequate was itself the problem. The law was asking whether she had behaved as a reasonable man would in a fight, rather than asking whether Rusk had had sex with her without her consent. The reform era Between the 1970s and the 1990s, nearly every American jurisdiction reformed its rape law, often under direct pressure from feminist organizations. The reforms took several forms. Rape shield laws restricted the use of a victim's prior sexual history. Michigan enacted the first comprehensive shield statute in 1974, and by the early 1980s nearly every state had one. Federal Rule of Evidence 412, adopted by Congress in 1978, bars evidence of a victim's other sexual behavior or sexual predisposition in federal cases, subject to narrow exceptions including where exclusion would violate the defendant's constitutional rights. Shield laws have been criticized by some defense lawyers and civil libertarians as limiting the right to present a defense, and courts have occasionally found their application unconstitutional in particular cases; but their core is now settled. Abolition of the corroboration requirement and the cautionary instruction followed in most states. California's Supreme Court, in People v. Rincon-Pineda (1975), disapproved of the Hale instruction, observing that it was a product of a different era and that no comparable caution was given for other crimes. Redefinition of the offense often replaced the single crime of rape with graded offenses of "sexual assault" or "criminal sexual conduct," written in gender-neutral terms, covering a wider range of penetrative and non-penetrative acts and removing or reducing the emphasis on resistance. Michigan's 1974 statute was again the model. The marital exemption was the slowest to fall. The Model Penal Code of 1962, the most influential American criminal law reform project of the century, retained it. States began abolishing or narrowing it in the late 1970s, and courts did the rest. The leading judicial decision is People v. Liberta (1984), in which New York's Court of Appeals held the state's marital exemption unconstitutional under the Equal Protection Clause, finding no rational basis for distinguishing marital from non-marital rape, and rejecting each of the traditional justifications, including Hale's implied consent theory, which it called irrational and absurd. By 1993, marital rape was a crime in all fifty states. Some states, however, continued to treat it differently, with shorter reporting periods or lesser penalties in some circumstances, and a number of those distinctions persisted into the twenty-first century. Estrich's and others' critique of the force requirement proved harder to translate into law. Table 4 shows how three well-known state supreme court decisions of the reform era approached the relationship between force and consent. Table 4. Force and consent in three reform-era state decisions. Case Court and year Facts in brief Holding on force Rusk v. State Md. Ct. App., 1981 Victim submitted after defendant took keys and choked her lightly Reasonable fear can substitute for resistance State in the Interest of M.T.S. N.J., 1992 Teenage acquaintance; no force beyond penetration Force satisfied by penetration without affirmative permission Commonwealth v. Berkowitz Pa., 1994 Victim said "no" repeatedly; no threats or physical restraint Verbal refusal alone did not establish "forcible compulsion" The contrast between the New Jersey and Pennsylvania decisions is instructive. New Jersey's statute defined sexual assault as penetration using "physical force or coercion." In M.T.S., the New Jersey Supreme Court interpreted the physical force element to be satisfied by the act of penetration itself, provided the victim had not given what the court called "affirmative and freely-given permission." The decision effectively converted a force requirement into a consent requirement, and it is often cited as the origin of the "affirmative consent" standard in American criminal law. In Berkowitz, by contrast, the Pennsylvania Supreme Court reversed a rape conviction where a college student repeatedly said "no" to a classmate in his dormitory room, holding that the absence of consent, without more, did not establish the "forcible compulsion" the statute required. The Pennsylvania legislature responded in 1995 by creating a new offense of sexual assault that does not require force, only the absence of consent. Consent: affirmative, contextual and contested The debate over what consent means is now the central question of rape law, and it divides feminists as well as their critics. The affirmative consent position holds that sexual activity is lawful only with an affirmative expression of willingness, by words or conduct, and that silence or passivity does not suffice. Its defenders argue that it places responsibility where it belongs, on the person initiating sexual activity, and removes the question of whether the victim resisted or protested enough. California adopted an affirmative consent standard for campus disciplinary proceedings at colleges receiving state funds in 2014, and New York followed in 2015. Several states have incorporated elements of it into their criminal law. Its critics include many scholars sympathetic to feminist goals. They argue that affirmative consent misdescribes how many people actually negotiate sexual encounters, that it risks criminalizing ambiguous but consensual conduct, and that it places a burden on defendants that sits uneasily with the presumption of innocence. The dispute came to a head in the American Law Institute's lengthy project to revise the sexual assault provisions of the Model Penal Code, which began in 2012. An early draft proposed an affirmative consent standard; after intense debate among members, the Institute moved away from it toward a definition of consent as a person's willingness to engage in the act, which may be expressed or inferred from behavior, including words and conduct, considered in context. The revision also finally eliminated the marital exemption that the 1962 Code had retained. The dominance theory position is different again. MacKinnon has argued that consent itself is an inadequate concept for a society structured by sexual inequality, because the conditions under which women say yes are shaped by men's power, economic and physical and social, and that a standard asking only whether the woman agreed treats as free a choice that often is not. On her view, the law should focus on coercion, including the use of unequal power, rather than on consent. Critics reply that this position cannot distinguish between sex that is wanted and sex that is not, and denies women the capacity to make sexual choices at all. Comparative law shows the range of options. England and Wales, under the Sexual Offences Act 2003, define consent as agreeing by choice with the freedom and capacity to make that choice, and require that the defendant's belief in consent be reasonable. Canada's Supreme Court, in R. v. Ewanchuk (1999), rejected any defense of "implied consent" and held that consent must be actually present in the complainant's mind. Sweden adopted a consent-based definition of rape in 2018, including a new offense of negligent rape. Spain's 2022 law, popularly called the "only yes is yes" law, redefined sexual assault around consent, and its merger of offense categories produced an unintended wave of sentence reductions for previously convicted offenders, prompting an amendment in 2023. The Spanish episode is sometimes cited by critics as a caution about reform driven by principle ahead of drafting. Race and the law of rape The intersectional critique of rape law, advanced by Angela Harris, Kimberlé Crenshaw, Dorothy Roberts and others, adds a dimension that the "real rape" critique largely omitted. In American history the law of rape was not only a mechanism for disbelieving women. It was also, and simultaneously, a mechanism for terrorizing Black men and for denying Black women any protection at all. The denial of protection to Black women was, under slavery, formal. As Chapter 4 noted, enslaved women could not legally be raped by the men who owned them, and in some jurisdictions the rape of an enslaved woman by anyone was not a crime against her but at most a trespass against her owner's property. After emancipation, the formal rule disappeared, but the practice of non-enforcement did not. The case of Recy Taylor, a Black woman abducted and raped by a group of white men in Abbeville, Alabama, in 1944, drew national attention and an investigation by the NAACP, whose Montgomery branch sent Rosa Parks to investigate; two all-white grand juries declined to indict. The Alabama legislature formally apologized in 2011. Meanwhile, accusations of rape against Black men, true and false, were among the principal justifications offered for lynching. The journalist Ida B. Wells documented in the 1890s how often the charge was invented or used to justify killings for other reasons. Formal legal processes were not much better. In Powell v. Alabama (1932), the Scottsboro case, the Supreme Court reversed the death sentences of young Black men convicted of raping two white women after trials without meaningful assistance of counsel; one of the alleged victims later recanted. The death penalty for rape was imposed overwhelmingly on Black defendants; federal statistics show that of the 455 men executed for rape in the United States between 1930 and 1972, 405 were Black. When the Supreme Court held in Coker v. Georgia (1977) that the death penalty was unconstitutional for the rape of an adult woman, the plurality relied on proportionality rather than race, but the racial history was part of the background briefing. For feminist legal theory, this history poses a genuine dilemma, not merely a gap to be filled. The reform program aimed to make rape easier to prosecute. The intersectional critique points out that in a system where prosecution has historically fallen most heavily on Black men, and protection has historically been withheld most completely from Black women, making prosecution easier will not affect all men and all women alike. Crenshaw, in "Mapping the Margins," discussed the widely publicized rape of a white woman jogger in Central Park in 1989, which led to the conviction of five Black and Latino teenagers, and contrasted it with the scant attention paid to the rapes of women of color in New York in the same period. The convictions of the five were vacated in 2002 after another man confessed and DNA evidence confirmed his involvement, a result that strengthened the argument that intersectional analysis must attend to wrongful accusation as well as to non-prosecution. Campus proceedings and the due process critique The most intense recent debates about sexual assault law have concerned not criminal courts but university disciplinary processes under Title IX of the Education Amendments of 1972. In 2011, the Department of Education's Office for Civil Rights issued a "Dear Colleague" letter directing institutions to use a preponderance of the evidence standard in sexual misconduct proceedings. Critics, including a group of Harvard Law School faculty members, several of them prominent feminists, argued in a 2014 open letter that the resulting procedures lacked basic fairness for accused students. The Department withdrew the guidance in 2017 and in 2020 issued formal regulations requiring live hearings with cross-examination by an advisor in higher education, among other procedural protections. The Biden administration issued new regulations in 2024 that revised those procedures and expanded the definition of sex discrimination; a federal district court in Kentucky vacated those regulations nationwide in January 2025, and the Department of Education directed schools to follow the 2020 regulations. The swing of the pendulum reflects an underlying disagreement that no set of regulations can resolve. From one side, the concern is that campus processes had long protected institutions and accused students at the expense of victims, many of whom never reported. From the other, the concern is that procedures designed to correct that failure sometimes abandoned the protections against wrongful accusation that the criminal law, for good reasons, provides. The intersectional point applies here too: some critics, notably Janet Halley, argued that accused students who were Black men might be disproportionately vulnerable to weakened procedures. Evidence on that question remains limited, and it is best treated as a live concern rather than an established fact. What the reforms achieved It is difficult to measure the effect of rape law reform on the behavior it was meant to change. Studies of the early reforms, including work by Cassia Spohn and Julie Horney, found that the reforms had more modest effects on reporting, prosecution and conviction than advocates had hoped, in part because the attitudes of police, prosecutors and juries changed more slowly than the statutes. What the reforms plainly did accomplish was symbolic and normative: the law now says, almost everywhere, that sex without consent is a crime, that a victim need not fight to prove her unwillingness, that marriage is no defense, and that a woman's sexual history is not evidence of her credibility. Whether a society's legal statements change its conduct is a large question. The feminist critique of the old law was that its statements had said the opposite, and that the law was teaching what it claimed only to record. Chapter 6: Criminal Law II: Domestic Violence, Self-Defense, and the Carceral Turn If rape law shows how neutral rules can encode suspicion of women, the law of domestic violence shows how neutral rules can encode indifference. For most of American history, violence by husbands against wives was either expressly privileged or treated as a private matter beyond the reach of the criminal law. Feminist advocacy transformed that position within a single generation, and the transformation is among the most consequential legal changes of the late twentieth century. It also produced the most searching self-criticism the feminist legal movement has undertaken, because the tools it chose, arrest and prosecution, turned out to have costs that fell unevenly. From chastisement to privacy English common law recognized a husband's right to "correct" his wife, by moderate means, as he might correct a servant or child. American courts inherited the doctrine and, in the early nineteenth century, sometimes applied it. In Bradley v. State (1824), the Mississippi Supreme Court acknowledged a husband's right to moderate chastisement in cases of great emergency, while holding that he could not claim it in the case before it. Later in the century, courts repudiated the chastisement right. In Fulgham v. State (1871), the Alabama Supreme Court declared that the privilege, ancient though it be, to beat her with a stick, to pull her hair, choke her, spit in her face or kick her about the floor, or to inflict upon her like indignities, is not now acknowledged by our law. Yet as Reva Siegel showed in "'The Rule of Love': Wife Beating as Prerogative and Privacy," published in the Yale Law Journal in 1996, the repudiation of the right did not bring an end to legal tolerance. Courts and legislatures shifted to a different rationale. Rather than saying husbands had a right to beat their wives, they said the state should not intrude into the privacy of the home. State v. Rhodes, the North Carolina decision described in Chapter 1, declined to punish a husband's beating because doing so would require the courts to examine the family's private life. Siegel also found that in the late nineteenth and early twentieth centuries, prosecutions for wife beating, and proposals to punish it by whipping, were directed disproportionately at Black and immigrant men, while middle-class white households were protected by privacy. The legal response to domestic violence was shaped by race and class from the start. Through much of the twentieth century, the practical result was a pattern of non-enforcement. Police were trained to "mediate" domestic disputes and to avoid arrest. Prosecutors dropped cases when victims recanted. Civil protective orders, where they existed, were difficult to obtain and weakly enforced. The legal revolution The change began in the 1970s, driven by the battered women's movement, which founded shelters, documented the scale of the problem, and litigated. Several legal developments stand out. The first was litigation over police non-enforcement. Class actions in New York and Oakland in the late 1970s challenged police policies of non-arrest. The most dramatic case was Thurman v. City of Torrington (1984). Tracey Thurman had repeatedly asked the Torrington, Connecticut, police for protection from her estranged husband, Charles Thurman, who had threatened her. They did little. In June 1983 he stabbed her repeatedly while a police officer was present, and continued to attack her as more officers arrived. She survived with permanent injuries. A federal district court held that a police department's failure to protect women from domestic assault could violate the Equal Protection Clause if it reflected a policy of treating domestic violence differently from other assaults, and a jury awarded her $2.3 million. The case prompted Connecticut to adopt a mandatory arrest law in 1986. The second was social science. In the early 1980s, the Minneapolis Domestic Violence Experiment, conducted by Lawrence Sherman and Richard Berk and published in 1984, randomly assigned police responses to misdemeanor domestic assault calls and found that arrest was associated with lower rates of repeat violence than advice or separation. The study was enormously influential in the spread of mandatory and preferred arrest policies. Its replications, conducted in six other cities under federal sponsorship later in the decade, produced mixed results: in some cities arrest reduced repeat violence, in others it had no effect, and in some it appeared to increase violence among unemployed suspects over the longer term. Sherman himself later urged caution about mandatory arrest. By then, however, many states had adopted such laws. The third was federal legislation. The Violence Against Women Act of 1994, championed by then-Senator Joseph Biden, funded training and services, created federal crimes of interstate domestic violence, required states to give full faith and credit to protective orders issued in other states, and created a civil rights remedy allowing victims of gender-motivated violence to sue their attackers in federal court. The Act has been reauthorized in 2000, 2005, 2013 and 2022, with the later reauthorizations extending protections to immigrant, Native American and LGBTQ victims. The 1996 Lautenberg Amendment prohibited people convicted of misdemeanor crimes of domestic violence from possessing firearms. The limits of federal and constitutional protection Two Supreme Court decisions defined the limits of the constitutional response. In United States v. Morrison (2000), the Court struck down VAWA's civil rights remedy. Christy Brzonkala had alleged that she was raped by two football players at Virginia Tech, which imposed minimal discipline. The Court held, five to four, that Congress lacked power to create the remedy under either the Commerce Clause, because gender-motivated violence was not economic activity, or Section 5 of the Fourteenth Amendment, because that provision reaches state action and not private conduct. The decision was a major moment in the Court's federalism jurisprudence, and feminist scholars criticized it for treating violence against women as a local matter of private conduct, echoing the old public/private distinction. In Town of Castle Rock v. Gonzales (2005), the Court held that Jessica Gonzales had no constitutionally protected property interest in the enforcement of a restraining order against her estranged husband. She had repeatedly called the Castle Rock, Colorado, police over several hours to report that he had taken their three daughters in violation of the order. The police did not act. Later that night he arrived at the police station and opened fire, and was killed; the three girls were found dead in his truck. Colorado's statute said officers "shall" arrest for violations of protection orders, but the Court held that this language did not create an individual entitlement enforceable under the Due Process Clause, citing the deep-rooted nature of law enforcement discretion. The decision followed DeShaney v. Winnebago County Department of Social Services (1989), which had held that the state has no general constitutional duty to protect individuals from private violence. Gonzales later took her case to the Inter-American Commission on Human Rights, which in 2011 found that the United States had violated her rights under the American Declaration of the Rights and Duties of Man. The finding had no binding domestic effect. By contrast, the Court has upheld federal firearms restrictions tied to domestic violence. In United States v. Rahimi (2024), the Court upheld, by a vote of eight to one, the federal statute prohibiting people subject to domestic violence restraining orders from possessing firearms. The challenge arose under the historical-tradition test for Second Amendment claims announced in New York State Rifle & Pistol Ass'n v. Bruen (2022). Chief Justice Roberts's opinion held that the nation's historical tradition of disarming individuals who pose a credible threat to the physical safety of others supported the law. The case was closely watched by domestic violence advocates, who pointed to research showing that access to a gun substantially increases the risk that an abuser will kill a partner. Self-defense and the battered woman The law of self-defense presents one of the clearest examples of the default-subject problem described in Chapter 1. Traditional self-defense doctrine allowed deadly force only against an imminent threat of death or serious bodily injury, and required that the force be proportionate. The paradigm was a sudden confrontation between two men of roughly equal size. A woman who killed an abusive partner, often when he was not actively attacking her, because she believed she could not survive a direct confrontation, did not fit. In State v. Wanrow (1977), the Washington Supreme Court reversed the conviction of Yvonne Wanrow, a Native American woman who had shot a man she believed had molested her child, because the jury instructions directed the jurors to assess her conduct against the standard of an altercation between two men. The court held that the jury should consider the circumstances as she perceived them, including her size and the disparities in physical strength, and that the instructions as given had denied her the right to have her conduct judged in light of her own perceptions. The decision anticipated the "reasonable woman" debate in sexual harassment law. Lenore Walker's book The Battered Woman (1979) introduced the concept of "battered woman syndrome," describing a cycle of tension, violence and reconciliation, and a condition of "learned helplessness" that could explain why abused women did not leave. Expert testimony on the syndrome was admitted in an increasing number of jurisdictions to support self-defense claims, often to explain why a defendant's fear was reasonable and why she had not left. Its use has limits. In State v. Norman (1989), the North Carolina Supreme Court held that Judy Norman, who shot her husband while he slept after years of severe abuse, was not entitled to a self-defense instruction because there was no evidence that she faced an imminent threat. The case remains a standard teaching example of the tension between the imminence requirement and the reality of long-term abuse. Norman's sentence was later commuted by the governor. Feminist scholars have been divided about the syndrome itself. Elizabeth Schneider, in Battered Women and Feminist Lawmaking (2000), argued that its emphasis on helplessness and pathology misdescribed many women's experience and risked portraying battered women as psychologically impaired rather than as reasonable people responding to real danger. It also fit poorly with women who had fought back, and some scholars argued that it fit poorly with the experiences and stereotypes of Black women, who were less likely to be perceived by juries as helpless. Many jurisdictions and experts now prefer broader testimony about "battering and its effects" to the syndrome label. Coercive control A newer development shifts the focus from discrete acts of physical violence to patterns of domination. Evan Stark's book Coercive Control (2007) argued that the core of much abuse is not episodic assault but a sustained campaign of isolation, intimidation, surveillance and control over money and daily life, and that criminal law's focus on incidents of physical injury misses it. England and Wales created an offense of controlling or coercive behavior in an intimate or family relationship in section 76 of the Serious Crime Act 2015; Scotland's Domestic Abuse (Scotland) Act 2018 went further, creating an offense of engaging in a course of abusive behavior. In the United States, a few states, including California and Connecticut, have incorporated coercive control into the definitions of abuse used in family law and protective order proceedings, and a small number have begun to consider criminal offenses. The idea has broad support among advocates, but some feminist scholars who are critical of criminalization worry that a vague offense will be enforced selectively and, as with other criminal laws, most heavily in poor communities and communities of color. The critique of carceral feminism The strongest internal critique of the feminist response to domestic violence concerns the reliance on criminal law itself. The critique draws on intersectional analysis and has been developed by scholars including Beth Richie, Leigh Goodmark and Aya Gruber. Richie's Arrested Justice (2012) argued that the anti-violence movement, in allying with law enforcement, had built a response that served relatively privileged women and failed, or actively harmed, poor Black women, who were more likely to be arrested themselves, more likely to be disbelieved, and more likely to see their partners, children and communities damaged by incarceration. She described how the movement's early vision of transforming social conditions had narrowed into a program of criminalization. Goodmark, a former domestic violence advocate, argued in A Troubled Marriage (2012) and Decriminalizing Domestic Violence (2018) that mandatory arrest and "no-drop" prosecution policies, under which prosecutors proceed regardless of the victim's wishes, took decisions out of victims' hands in the name of protecting them. Mandatory arrest also produced a rise in "dual arrests," in which both partners were arrested, and in arrests of women who had used violence in self-defense. Goodmark proposed a public health and economic approach, including direct financial support for victims. Gruber's The Feminist War on Crime (2020) offered a broader historical account, arguing that feminist campaigns on rape and domestic violence had contributed to the expansion of the American carceral state in the 1980s and 1990s, and that feminists should reconsider their alliance with punitive criminal justice. Several concrete cases became touchstones for this critique. Marissa Alexander, a Black woman in Florida, was sentenced in 2012 to a mandatory twenty years for firing what she described as a warning shot toward her abusive husband, under a mandatory minimum statute for crimes involving firearms; her conviction was overturned on appeal, and she later accepted a plea deal and was released. The contrast with self-defense claims by others under Florida's "stand your ground" law was widely noted. Advocates also point to the prosecution of pregnant women for drug use, another area where criminal law was presented as protective. In Ferguson v. City of Charleston (2001), the Supreme Court held that a public hospital's policy of testing pregnant patients for cocaine without their consent and reporting positive results to police violated the Fourth Amendment. The policy had been applied almost entirely to Black women. Dorothy Roberts's work, discussed in Chapter 8, placed such prosecutions within a long history of state control over Black women's reproduction. The anti-carceral critique has its own critics. Many advocates and survivors argue that the criminal law, whatever its flaws, is the only mechanism that can physically separate a dangerous abuser from a victim, and that the pre-reform world of police indifference was far worse. They point out that the reforms were demanded by women who had been left unprotected, and that the most dramatic failures, such as Thurman and Castle Rock, were failures to enforce the law rather than excessive enforcement. Some also argue that the anti-carceral critique understates the extent to which women of color have themselves sought criminal justice protection. The debate is unresolved, and it now shapes policy discussions about specialized domestic violence courts, restorative justice programs and the allocation of funding under the Violence Against Women Act. The larger pattern The legal history of domestic violence shows each of the feminist methods described in Chapter 1 at work. Asking the woman question revealed that the privacy doctrine and the imminence requirement, both neutral on their face, operated to protect abusers. Attention to the bottom, in Matsuda's phrase, revealed that the remedies designed in response worked differently for poor women and women of color. And the persistence of the debate among feminists themselves shows that the field's critique of neutrality does not dictate a single answer. Identifying that a rule is not neutral is the beginning of the inquiry, not the end. Chapter 7: Contract, Work, and the Family Bargain Contract law is, in its self-description, the most neutral field in private law. Its central premise is that the law enforces the agreements of free and equal parties, and that it does not ask who they are, only what they agreed. That premise is a genuine achievement of liberal legal thought; Sir Henry Maine's famous nineteenth-century formula described the movement of progressive societies as a movement from status to contract, from positions assigned by birth to relations chosen by agreement. Feminist legal theory's engagement with contract begins by observing that, for most of legal history, women were on the status side of that line. Under coverture a married woman could not contract at all. When the disability was removed, the question became whether the law of contract, designed around the arm's-length bargain between strangers in the market, could accommodate the relationships in which most women's economic lives were actually embedded. Marriage as status and contract The political theorist Carole Pateman's The Sexual Contract (1988) offered the boldest version of the critique. Pateman argued that the social contract of classical liberal theory, the agreement by which free individuals create political society, rested on a prior and unacknowledged "sexual contract" by which men secured access to and authority over women. Marriage was her central example. It was called a contract, but its terms were fixed by law rather than negotiated by the parties, it could not be ended by agreement, and it placed one party under the authority of the other. It was, in her account, a contract that created a status of subordination. American law's treatment of marriage long reflected this dual character. Couples could not alter the essential incidents of marriage by agreement. In particular, courts refused to enforce contracts between spouses for payment for domestic services. The reasoning was that a wife already owed such services to her husband as a legal duty of marriage, so a promise to pay for them lacked consideration and was contrary to public policy. The rule has proved durable. In Borelli v. Brusseau (1993), a California appellate court refused to enforce a husband's promise to leave his wife certain property if she would care for him at home during his final illness rather than placing him in a nursing home. She did so for more than a year. The court held that nursing care of a spouse was part of the marital duty of support, so her promise could not serve as consideration. A dissenting judge argued that the rule rested on an outdated conception of marriage in which the wife's labor belonged to the husband. Feminist scholars have used the case to show how the logic of coverture, the idea that a wife's household work belongs to the marriage rather than to her, persisted in contract law long after the Married Women's Property Acts. At the same time, courts became more willing to enforce agreements about marriage's end. Premarital agreements, once viewed with suspicion as contracts in contemplation of divorce, are now broadly enforceable, and the Uniform Premarital Agreement Act of 1983, since adopted in a majority of states, sets out their conditions. Some feminist scholars welcomed the change as an expression of women's autonomy; others argued that the parties to such agreements are frequently unequal in wealth, information and bargaining position, and that enforcement tends to protect the wealthier spouse, usually the husband. The case of Simeone v. Simeone (1990), in which the Pennsylvania Supreme Court enforced a premarital agreement signed by a twenty-three-year-old nurse on the eve of her wedding to a thirty-nine-year-old neurosurgeon, is frequently used to illustrate both sides. The majority treated the agreement as an ordinary contract between equals, explicitly rejecting paternalistic assumptions about women's bargaining capacity. The concurrence and later critics asked whether treating the parties as equals, when they were not, was itself a form of neutrality that favored the stronger party. Cohabitation and the problem of the unmarried partner The flip side of marriage's special status was the legal position of those outside it. In Marvin v. Marvin (1976), the California Supreme Court held that unmarried cohabitants could enforce express or implied contracts between them, including agreements to share property, so long as the agreement did not rest on sexual services as consideration. Michelle Triola Marvin had lived with the actor Lee Marvin for about six years and claimed he had promised to share his earnings and support her for life. On remand she recovered nothing on her contract claims; a modest rehabilitation award by the trial court was reversed on appeal. Clare Dalton's article "An Essay in the Deconstruction of Contract Doctrine," published in the Yale Law Journal in 1985, used the cohabitation cases to show how contract doctrine's core distinctions, between express and implied agreements, between public and private, between the objective and subjective understanding of the parties, operated in ways that consistently disadvantaged the partner, usually a woman, who had contributed domestic labor rather than money. The decision whether to "find" an implied agreement was not dictated by doctrine, Dalton argued, but by the judge's picture of what the relationship was. Courts that saw cohabitation as a relationship of affection tended to find no contract; courts that saw it as an economic partnership might find one. The same labor could be valuable consideration or a gift of love, depending on how the relationship was characterized. Unconscionability and the poor contracting party The intersectional method finds a significant case in the contract canon itself. Williams v. Walker-Thomas Furniture Co. (1965), decided by the U.S. Court of Appeals for the D.C. Circuit, is taught in nearly every American contracts course as the leading case on unconscionability. Ora Lee Williams was a single mother of seven, supported by public assistance, who over several years purchased household goods from the Walker-Thomas store in Washington on installment contracts. A clause in the contracts spread each payment across all the items she had ever bought, so that until she paid off everything, the store retained a security interest in all of it. When she defaulted on a stereo, the store sought to repossess everything she had purchased since 1957. Judge J. Skelly Wright's opinion held that a court could refuse to enforce a contract that was unconscionable, meaning one involving an absence of meaningful choice by one party together with terms unreasonably favorable to the other. The case is typically taught as a doctrinal question about the limits of freedom of contract. Feminist and critical race scholars, including Muriel Morisey Spence and Blake Morant, have pointed out that the opinion and the later casebooks largely erased who Williams was: a Black woman, a single mother, a welfare recipient, contracting in a segregated retail market in which stores that sold to poor Black customers in the city charged higher prices and used harsher credit terms than those serving suburban white customers. Those facts were not incidental. They were what made her choice not "meaningful" in the sense the doctrine required. Yet the doctrine that emerged was framed in neutral terms, and courts applying it have often been reluctant to find unconscionability on the basis of the characteristics of the weaker party. The case thus shows both the possibility and the limit of neutral contract doctrine as a remedy for structural disadvantage. Mary Joe Frug, in "Re-reading Contracts: A Feminist Analysis of a Contracts Casebook," published in the American University Law Review in 1985, performed a similar exercise across an entire casebook, examining how the leading teaching materials presented women, typically as either dependents or unreliable parties, and how the language of doctrine presented the market as the domain of male actors. A related line of empirical work asks whether markets that rely on bargaining treat people differently. Ian Ayres's study "Fair Driving: Gender and Race Discrimination in Retail Car Negotiations," published in the Harvard Law Review in 1991, sent trained testers of different races and sexes, using identical scripted bargaining strategies, to negotiate for new cars at Chicago-area dealerships. White men received significantly better final offers than white women, Black men and Black women, with Black women and Black men receiving the worst offers. The study was important not only for its findings, which were contested in the literature and later partly qualified in follow-up work, but for its demonstration that ostensibly neutral bargaining in a free market could produce systematically different outcomes by race and sex. Work: from protection to equality The labor contract is where the tension between status and contract has been most consequential for women. Chapter 1 described Muller v. Oregon (1908), which upheld maximum hours for women at a time when Lochner v. New York (1905) forbade them for men. In Adkins v. Children's Hospital (1923), the Supreme Court struck down a District of Columbia minimum wage law for women, reasoning in part that the Nineteenth Amendment had reduced the legal differences between the sexes to the vanishing point. When the Court reversed course in West Coast Hotel Co. v. Parrish (1937), it upheld Washington State's minimum wage law for women, and with it the modern regulatory state. The plaintiff was Elsie Parrish, a hotel chambermaid. The pivotal decisions in the constitutional history of labor regulation thus involved women workers, and women's supposed special need for protection was the wedge by which regulation entered. The Equal Pay Act of 1963 required equal pay for equal work regardless of sex. The Civil Rights Act of 1964 added "sex" to the prohibited grounds of employment discrimination in Title VII through an amendment offered on the House floor by Representative Howard Smith of Virginia, a segregationist opponent of the bill. Historians still debate his motives; some have argued that the amendment was intended to make the bill harder to pass, others that Smith had long supported the Equal Rights Amendment and meant the amendment seriously. Its supporters in the House, notably Representative Martha Griffiths, argued forcefully that without it, the Act would protect Black women but leave white women without a remedy, an argument that is itself an early and uncomfortable example of the intersection of race and sex in American law. Title VII's sex discrimination provisions developed several key doctrines through litigation. Phillips v. Martin Marietta (1971), discussed in Chapter 3, established that treating women with young children differently from men with young children was discrimination. Price Waterhouse v. Hopkins (1989) held that an employer that relied on sex stereotypes, in that case telling a senior manager denied partnership that she should walk, talk and dress more femininely, had discriminated because of sex. Meritor Savings Bank v. Vinson (1986) recognized hostile environment sexual harassment, and Oncale v. Sundowner Offshore Services (1998) held that same-sex harassment was actionable. Bostock v. Clayton County (2020) held that firing an employee for being gay or transgender is discrimination because of sex. Pay, class actions and the structure of the wage gap The persistence of a pay gap between men and women, and a larger gap for most women of color, is among the most debated issues in the field. According to Census Bureau data, women who work full-time and year-round have in recent years earned roughly 82 to 84 cents for every dollar earned by comparable men, and Black and Hispanic women earn considerably less relative to white men. Feminist scholars and their critics agree on these raw figures and disagree about how much of the gap reflects discrimination and how much reflects differences in occupation, hours and time spent out of the labor force. The economist Claudia Goldin, awarded the Nobel Memorial Prize in Economic Sciences in 2023, has shown in work including Career and Family (2021) that a large portion of the modern gap arises after the birth of a first child and is concentrated in occupations that pay a premium for long, inflexible hours. Her analysis is often cited by critics of feminist legal theory as showing that the gap reflects choices rather than discrimination. Goldin's own framing is more careful: the "greedy work" structure she describes is itself an institutional arrangement, and the choice to have one partner reduce commitments to the job while the other takes on the greedy one falls predominantly on women for reasons that include social norms. For feminist legal theory, that finding supports the "unmarked baseline" argument of Chapter 1: a labor market that rewards the unencumbered worker is not neutral as between those with and without caregiving obligations. The law has struggled to reach this kind of structural disadvantage. In Ledbetter v. Goodyear Tire & Rubber Co. (2007), the Supreme Court held that Lilly Ledbetter, who learned near the end of a long career that she had been paid less than male colleagues for years, could not recover because the discriminatory pay decisions had been made outside the 180-day filing period. Justice Ginsburg read her dissent from the bench and called on Congress to act; the Lilly Ledbetter Fair Pay Act of 2009, the first bill President Obama signed, reset the filing period with each discriminatory paycheck. In Wal-Mart Stores, Inc. v. Dukes (2011), the Court reversed certification of a nationwide class of roughly 1.5 million female employees alleging discrimination in pay and promotion, holding that they had not shown a common question of law or fact because the challenged decisions were made by many managers exercising discretion. The decision made it much harder to litigate systemic discrimination produced by decentralized decisions, which is precisely the kind of discrimination that feminist and intersectional theory predicts will be most common where explicit policies have been abolished. Care, leave and accommodation The final element of the feminist critique of work concerns caregiving. The Family and Medical Leave Act of 1993 provides eligible workers with up to twelve weeks of unpaid, job-protected leave for the birth or adoption of a child, their own serious health condition, or the care of a family member. It was deliberately written in sex-neutral terms, a choice reflecting the formal equality position that sex-specific leave would reinforce stereotypes about women as caregivers. In Nevada Department of Human Resources v. Hibbs (2003), the Supreme Court upheld the family-care provision as applied to state employers, with Chief Justice Rehnquist's opinion recognizing that the stereotype that caring for family members is women's work had long been a source of employment discrimination. But in Coleman v. Court of Appeals of Maryland (2012), the Court held that the self-care provision did not validly abrogate state sovereign immunity, finding its connection to sex discrimination insufficiently established. Because the leave is unpaid, many workers cannot afford to take it, and because eligibility depends on employer size and length of service, many workers are not covered. The United States remains unusual among wealthy countries in lacking a national paid parental leave program, though a growing number of states have enacted paid family leave laws. The Pregnant Workers Fairness Act of 2022, described in Chapter 2, requires employers with fifteen or more employees to provide reasonable accommodations for known limitations related to pregnancy, childbirth or related medical conditions. The EEOC's implementing regulations, issued in 2024, were challenged in court, in part because they included abortion among the related medical conditions, and the Act itself faced a separate constitutional challenge concerning the House's pandemic-era proxy voting procedure that remained before the full Fifth Circuit in 2026. Surrogacy and the market in reproduction A last example shows how contract questions can divide feminists. In In re Baby M (1988), the New Jersey Supreme Court refused to enforce a surrogacy contract under which Mary Beth Whitehead agreed, for a fee, to be inseminated with William Stern's sperm, carry the child and surrender her parental rights. The court held that the contract violated the state's laws against baby-selling and public policy, though it awarded custody to the Sterns on a best-interests basis and restored Whitehead's visitation. In Johnson v. Calvert (1993), the California Supreme Court enforced a gestational surrogacy arrangement, in which the surrogate was not genetically related to the child, holding that the intended mother who provided the egg was the legal mother. Anna Johnson, the gestational carrier, was a Black woman; Crispina Calvert, the genetic and intended mother, was Filipina, and Mark Calvert was white. Commentators including Dorothy Roberts pointed to the racial dimensions of the case and of the surrogacy market more broadly. Feminists split sharply on surrogacy. Some, drawing on the dominance approach, saw it as the commodification of women's reproductive capacity and a form of exploitation of poor women by wealthy couples. Others, drawing on liberal and autonomy-based arguments, contended that refusing to enforce surrogacy contracts treated women as incapable of making binding decisions about their own bodies, the same paternalism that had justified coverture. The law is now divided by state, with some states permitting and regulating compensated gestational surrogacy and others restricting it, and international surrogacy has raised further questions about the treatment of women in poorer countries. Contract law's promise of equal treatment for equal parties has been genuinely emancipatory for women. The feminist critique does not reject that promise. It asks, as Dalton and Frug did in the casebooks and as Ayres did in the showroom, whether the parties the law treats as equal actually are, and what the law does when they are not. Chapter 8: The Constitution, Sex Equality, and Reproductive Control The constitutional law of sex has developed on two tracks that feminist scholars have long argued should be one. The first is the law of sex equality under the Equal Protection Clause, which governs when government may treat men and women differently. The second is the law of reproduction, which governs contraception, abortion, sterilization and related decisions, and which the Supreme Court for half a century grounded in the liberty protected by the Due Process Clause rather than in equality. Feminist legal theory's most sustained constitutional argument has been that reproductive control is fundamentally a question of sex equality, because laws governing reproduction fall on women and have historically been used to confine them to domestic roles. The Court has never accepted that argument, and its decisions since 2022 have moved further from it. The equal protection framework Chapter 2 described how the formal equality litigation of the 1970s built the modern framework. After Craig v. Boren (1976), sex classifications receive intermediate scrutiny: they must serve important governmental objectives and be substantially related to those objectives. After United States v. Virginia (1996), the government must show an "exceedingly persuasive justification," and the justification must be genuine rather than invented for litigation, and must not rely on overbroad generalizations about the different talents, capacities or preferences of males and females. That framework has been applied rigorously to many classifications. In Mississippi University for Women v. Hogan (1982), the Court held that a state nursing school could not exclude men, reasoning that a women-only policy tended to perpetuate the stereotyped view of nursing as an exclusively women's job. In J.E.B. v. Alabama ex rel. T.B. (1994), it held that peremptory challenges to jurors on the basis of sex violated equal protection, extending the rule of Batson v. Kentucky (1986) on race. The case is worth noting historically: into the 1960s, many states exempted women from jury service unless they volunteered, and in Hoyt v. Florida (1961) the Court had upheld such a scheme on the ground that woman is still regarded as the center of home and family life. Taylor v. Louisiana (1975) effectively ended that practice, and Linda Kerber's history No Constitutional Right to Be Ladies (1998) traces how women's exemption from civic obligations such as jury service and military service was bound up with their exclusion from full citizenship. The framework has also, however, allowed a number of classifications to stand where the Court found a "real" difference between the sexes. In Michael M. v. Superior Court (1981), it upheld a California statutory rape law that punished only males, reasoning that because only women can become pregnant, the state could reasonably seek to deter teenage pregnancy by punishing the male participant. In Rostker v. Goldberg (1981), it upheld male-only draft registration, deferring to Congress on military matters and noting that women were then excluded from combat. In Nguyen v. INS (2001), it upheld a citizenship statute that imposed different requirements on children born abroad to unmarried citizen fathers than on those born to unmarried citizen mothers, reasoning that a mother's relationship to the child is established by birth. In Sessions v. Morales-Santana (2017), by contrast, Justice Ginsburg wrote for the Court striking down a different gender-based provision of the same citizenship law, which imposed longer residency requirements on unmarried fathers than mothers, because it rested on the stereotype that unwed mothers were the natural caretakers of their children. The line between a real difference and a stereotype is where most of the argument now occurs, and feminist scholars have argued that the "real differences" cases often mistook social arrangements for biological facts. The principal decisions are summarized in Table 5. Table 5. Selected Supreme Court decisions on sex classifications under the Equal Protection Clause. Case Year Classification challenged Result Reed v. Reed 1971 Preference for men as estate administrators Struck down Craig v. Boren 1976 Different drinking ages for men and women Struck down; intermediate scrutiny Michael M. v. Superior Court 1981 Male-only statutory rape liability Upheld Rostker v. Goldberg 1981 Male-only draft registration Upheld United States v. Virginia 1996 Male-only state military college Struck down Nguyen v. INS 2001 Different citizenship proof for unmarried fathers Upheld Sessions v. Morales-Santana 2017 Longer residency rule for unmarried fathers Struck down Pregnancy and the problem of classification The equal protection framework applies only to laws that classify by sex. That limitation matters most for pregnancy. As Chapter 2 explained, Geduldig v. Aiello (1974) held that excluding pregnancy from a disability insurance program did not classify on the basis of sex, because it distinguished pregnant women from nonpregnant persons. Congress overrode the statutory version of that reasoning for employment in 1978, but Geduldig remains constitutional law. Feminist scholars have criticized Geduldig more consistently than perhaps any other decision. Reva Siegel's article "Reasoning from the Body," published in the Stanford Law Review in 1992, argued that the case exemplified a pattern in which the law treated regulation of pregnancy as a response to physiological facts, when in reality such regulation reflected and enforced judgments about women's social roles. Laws restricting abortion, she argued, did not merely protect potential life; they also expressed and enforced the expectation that women should bear and raise children, and that expectation was the same one courts had invoked to exclude women from the bar, from juries and from work. On this view, pregnancy regulation was a central instance of sex discrimination, not an exception to it. The Geduldig reasoning reappeared in Bray v. Alexandria Women's Health Clinic (1993), where the Court held that opposition to abortion did not constitute class-based animus against women for purposes of a federal civil rights statute. It appeared again, briefly but decisively, in the Court's most important reproductive rights decision of the century. From Roe to Dobbs Roe v. Wade (1973) held that the constitutional right of privacy, grounded in the liberty protected by the Due Process Clause, was broad enough to encompass a woman's decision whether to terminate a pregnancy, and established a trimester framework for permissible regulation. Planned Parenthood of Southeastern Pennsylvania v. Casey (1992) reaffirmed Roe's central holding while replacing the trimester framework with an "undue burden" standard. Importantly for feminist theory, the joint opinion in Casey explicitly invoked equality. The ability of women to participate equally in the economic and social life of the nation, it said, had been facilitated by their ability to control their reproductive lives. And it struck down Pennsylvania's spousal notification requirement on the ground that it reflected a view of marriage and of women's place in it that was inconsistent with modern understandings, citing coverture and Bradwell in the process. Many feminist scholars nonetheless regarded Roe's privacy foundation as a weakness. Ruth Bader Ginsburg, in a 1985 article in the North Carolina Law Review, "Some Thoughts on Autonomy and Equality in Relation to Roe v. Wade," argued that the decision had been vulnerable because it rested on a medically framed privacy right rather than on the equality of women, and that an equality rationale would have been more secure. Catharine MacKinnon argued that privacy protected the domain in which women were most vulnerable to men's power. Dobbs v. Jackson Women's Health Organization (2022) overruled both Roe and Casey. Justice Alito's opinion for the Court held that the Constitution makes no reference to abortion and that no such right is implicitly protected by the Due Process Clause, because the right is not deeply rooted in the nation's history and tradition. The opinion surveyed common law authorities on abortion, including Sir Matthew Hale, whose treatise also contained the marital rape exemption and who presided over the trial of women accused of witchcraft; the dissent and many scholars noted the irony of relying on a jurist whose views of women's legal status are now universally rejected. On equal protection, the majority addressed the argument in a single paragraph. Relying on Geduldig, it held that the regulation of a medical procedure that only one sex can undergo does not trigger heightened scrutiny unless the regulation is a mere pretext designed to effect invidious discrimination, and that abortion regulations are governed by the same standards as other health and safety measures. It thus rejected precisely the equality argument feminist scholars had advanced for decades. The joint dissent of Justices Breyer, Sotomayor and Kagan emphasized equality. It argued that the Court's reliance on history meant relying on a time when women had few legal rights, and that the people who ratified the Fourteenth Amendment did not perceive women as equals and did not recognize women's rights. The majority's reply was that the history test is the proper method of identifying unenumerated rights and that the Court was returning the question to the people and their elected representatives. After Dobbs, the legal landscape fragmented. A number of states banned abortion at all or nearly all stages of pregnancy, while others adopted constitutional amendments or statutes protecting access; ballot measures in several states in 2022 and 2024 were decided on both sides. Litigation turned to questions that remain unsettled, including the scope of exceptions for medical emergencies, the relationship between state bans and the federal Emergency Medical Treatment and Labor Act, travel across state lines, and access to medication abortion. In FDA v. Alliance for Hippocratic Medicine (2024), the Supreme Court unanimously held that the physicians and associations challenging the FDA's regulation of mifepristone lacked standing, leaving the regulatory status of the drug unchanged, though further litigation and regulatory review followed. Reproductive control and race The intersectional critique adds an essential dimension to the reproductive rights debate. For much of American history the state's efforts to control reproduction were aimed not only at preventing some women from ending pregnancies but at preventing other women from having children. The most notorious legal landmark is Buck v. Bell (1927), in which the Supreme Court upheld Virginia's compulsory sterilization of Carrie Buck, a young white woman labeled "feebleminded," with Justice Oliver Wendell Holmes's infamous statement that three generations of imbeciles are enough. More than thirty states adopted eugenic sterilization laws, and tens of thousands of people were sterilized under them. Buck has never been formally overruled, although Skinner v. Oklahoma (1942), which struck down a law authorizing the sterilization of certain repeat criminals, recognized procreation as a fundamental right. Coercive sterilization continued long after the eugenics era, and it fell disproportionately on poor women of color. In Relf v. Weinberger (1974), a federal district court in Washington, D.C., addressed the case of Minnie Lee and Mary Alice Relf, two Black girls in Montgomery, Alabama, aged fourteen and twelve, who had been surgically sterilized in a federally funded clinic after their illiterate mother signed a consent form with an "X" without understanding it. The court found evidence that an estimated 100,000 to 150,000 poor people had been sterilized annually under federally funded programs, some under threat of losing welfare benefits, and ordered stronger consent protections. In Madrigal v. Quilligan, decided by a federal district court in Los Angeles in 1978, Mexican-American women who had been sterilized at the Los Angeles County-USC Medical Center shortly after or during childbirth, often after being presented with English-language consent forms while in labor, lost their suit, though the litigation contributed to reforms in consent procedures. Investigations in the 1970s also documented widespread sterilizations of Native American women by the Indian Health Service. More recently, California enacted legislation in 2021 to compensate survivors of state-sponsored sterilizations, including women sterilized in state prisons as late as 2010. Dorothy Roberts's Killing the Black Body (1997) drew these histories together. Roberts argued that the regulation of Black women's reproduction, from slavery's commodification of their fertility through coercive sterilization, prosecutions of pregnant drug users and welfare policies designed to discourage childbearing, was central rather than incidental to American racial politics. The mainstream reproductive rights movement, she argued, had focused on the right not to have children, which reflected the concerns of middle-class white women, while neglecting the right to have and raise children, which was what many poor women of color had been denied. That argument helped produce the reproductive justice framework, developed by Black women activists in the 1990s. In 1994 a group of Black women coined the term, and in 1997 the SisterSong Women of Color Reproductive Justice Collective was founded. Loretta Ross, one of its leaders, and the historian Rickie Solinger set out its principles in Reproductive Justice: An Introduction (2017): the right not to have a child, the right to have a child, and the right to parent children in safe and sustainable communities. The framework shifts attention from the legal right to choose toward the conditions, including economic security, health care and freedom from state violence, that make choices meaningful. Sex, gender identity and the limits of the framework The most recent constitutional developments concern whether laws regulating transgender people are sex classifications at all, a question that divides feminists as well as courts. In Bostock v. Clayton County (2020), the Supreme Court held under Title VII that discrimination against an employee for being transgender is discrimination because of sex, because an employer cannot discriminate on that basis without taking the employee's sex into account. The Court declined to extend that reasoning to the Equal Protection Clause in United States v. Skrmetti (2025). Tennessee prohibited certain medical treatments, including puberty blockers and hormones, for minors when used to treat gender dysphoria, while permitting them for other purposes. Chief Justice Roberts's opinion for a six to three majority held that the law classified on the basis of age and medical use rather than sex, that it therefore warranted only rational basis review, and that it satisfied that standard. The majority reasoned that Bostock's Title VII analysis did not control the constitutional question, and in any event did not show that the law turned on sex. Justice Sotomayor's dissent argued that the law classified explicitly by sex, since whether a minor could receive a given drug depended on the minor's sex at birth, and that the majority's approach would invite legislatures to conceal sex classifications in plain sight. The majority's reasoning echoed Geduldig's distinction between a classification by sex and a classification by a condition that only some members of a sex experience. A year later, in West Virginia v. B.P.J. (2026), decided together with Little v. Hecox, the Court held by a six to three vote that state laws limiting participation on girls' and women's school sports teams to students of the female sex did not violate the Equal Protection Clause. The dissenting Justices agreed on the Title IX question but would have found an equal protection violation. These cases expose a genuine division within feminist legal thought, which Chapter 9 considers further. Some feminists, including many in the intersectional and postmodern traditions, argue that laws targeting transgender people enforce the same rigid sex stereotypes that sex discrimination law was designed to dismantle, and that Skrmetti repeats Geduldig's error. Others, often called gender-critical feminists, argue that sex-based protections such as single-sex sports and spaces exist precisely because of physical differences between the sexes and are threatened when sex is redefined as gender identity. The Supreme Court's recent decisions align, in their results, with the second position, though they rest on constitutional reasoning about classification and scrutiny rather than on either feminist theory. The Equal Rights Amendment A final constitutional development illustrates the gap between feminist aspiration and legal reality. The Equal Rights Amendment, providing that equality of rights under the law shall not be denied or abridged on account of sex, passed Congress in 1972 with a seven-year ratification deadline, later extended to 1982. By the deadline, thirty-five of the necessary thirty-eight states had ratified, and five had voted to rescind their ratifications. Phyllis Schlafly's campaign against it, which argued that the amendment would subject women to the draft, eliminate protections for wives and homemakers and require unisex facilities, is widely credited with stopping it. Nevada ratified in 2017, Illinois in 2018 and Virginia in 2020, bringing the count to thirty-eight if the deadline and rescissions are disregarded. The Archivist of the United States declined to certify the amendment, relying on an opinion of the Justice Department's Office of Legal Counsel that the deadline was valid. Litigation to compel certification failed: in Illinois v. Ferriero, the D.C. Circuit in 2023 affirmed the denial of relief, holding that the plaintiffs had not shown a clear right to have the amendment certified. On January 17, 2025, three days before leaving office, President Biden stated that he considered the ERA to be the law of the land. The statement had no formal legal effect; the National Archives maintained its position, and the amendment has not been published as part of the Constitution. Whether the amendment would change much doctrine if it were in force is itself debated. Some scholars argue that it would require strict scrutiny for sex classifications and might provide a firmer basis for reproductive rights; others argue that intermediate scrutiny has already achieved most of what the ERA's supporters sought in the 1970s. The constitutional story thus ends where feminist legal theory began: with the question whether a formally neutral framework, here the text of the Equal Protection Clause and the tiers of scrutiny developed to apply it, can reach the forms of inequality that do not announce themselves as classifications. On pregnancy, on abortion and now on transgender status, the Court has answered that it largely cannot, or should not. The next chapter asks whether the Court's critics, feminist and otherwise, have the better of that argument, and what the strongest objections to feminist legal theory's own answers are. Chapter 9: The Case Against: Critiques from Within and Without A field that criticizes the law's claim to neutrality must expect its own claims to be tested, and feminist legal theory has been tested hard. Some of its most influential critics have been feminists; others have been economists, civil libertarians, conservatives and legal scholars skeptical of theory-driven jurisprudence altogether. This chapter sets out the principal critiques in their strongest form and assesses how well the field has answered them. It groups them into five families: empirical and economic critiques, liberal and civil libertarian critiques, conservative and natural-difference critiques, critiques of intersectionality specifically, and the internal divisions over sex and gender that have become the field's most bitter recent dispute. The empirical and economic critique The first family of criticism asks whether the disparities feminist legal theory attributes to discrimination or patriarchy are actually caused by them. Its most rigorous form comes from economics and from the law-and-economics movement. Richard Posner, in Sex and Reason (1992) and other writings, argued that many legal rules concerning sex and the family could be understood as reasonably efficient responses to real differences in the costs and benefits facing men and women, particularly those arising from pregnancy and childrearing, rather than as instruments of male domination. Richard Epstein, in Forbidden Grounds (1992), went further, arguing that employment discrimination laws were unnecessary and counterproductive in competitive markets, since employers who discriminated on grounds unrelated to productivity would lose to competitors who did not. On this view, persistent gaps in pay or representation are more likely to reflect differences in preferences, investment in skills and time spent in the workforce than irrational discrimination, and legal intervention imposes costs, including costs on the women it seeks to help, because it makes them more expensive or risky to hire. The critique has real force against claims that attribute every disparity to discrimination. As Chapter 7 noted, Claudia Goldin's work shows that much of the modern pay gap arises from the interaction between parenthood and occupations that pay a premium for long, inflexible hours, rather than from employers paying women less for identical work. Economists have also documented unintended effects of well-meant protections: some studies of generous mandated parental leave, for example, find that long leaves can reduce women's advancement, a concern that informed the gender-neutral design of the Family and Medical Leave Act. Feminist scholars have answered in three ways. First, they point to evidence that discrimination persists in competitive markets. Audit studies such as Ayres's car-buying experiment, and the large literature of resume audit studies that followed, find differential treatment by race and sex that is difficult to explain by productivity. Second, they argue that the "choices" invoked by the economic critique are made under constraints, including social expectations of motherhood, the structure of work and the unavailability of affordable childcare, that the law helped create; calling the outcome a preference does not make it pre-legal. Third, they note that the competitive-markets argument predicted that segregation and exclusion would erode on their own, and that the historical record, in which explicit exclusion persisted for generations until law intervened, is not kind to that prediction. A narrower empirical critique concerns the quality of feminist scholarship's own evidence. The Weitzman episode discussed in Chapter 4, in which a widely cited figure on the economic effects of divorce turned out to rest on a calculation error, is one example. Critics have also questioned statistics in the campus sexual assault debate and claims about the effectiveness of mandatory arrest. The fair response is that these episodes show the importance of careful evidence, not that the underlying phenomena are imaginary; the corrected divorce figures still showed a large gender gap. But the critique is a reminder that a field committed to exposing the unexamined assumptions of others must hold its own empirical claims to the same standard. The liberal and civil libertarian critique The second family of criticism accepts the goal of sex equality but argues that some feminist legal proposals sacrifice other important values, especially free expression, due process and individual autonomy. The pornography debate is the paradigm. Nadine Strossen, president of the American Civil Liberties Union from 1991 to 2008, argued in Defending Pornography (1995) that the MacKinnon-Dworkin ordinances would have empowered the state to suppress sexual expression, that such power historically had been used against feminists, sexual minorities and dissidents, and that the Canadian experience after R. v. Butler, in which customs officials seized gay and lesbian literature, confirmed the danger. Judge Easterbrook's opinion in Hudnut, discussed in Chapter 2, made the constitutional version of this argument. Many feminists joined the critique, forming organizations such as the Feminist Anti-Censorship Taskforce in the 1980s, and the "sex wars" left a lasting division between those who see sexual expression as a site of women's subordination and those who see it as a site of women's freedom. The due process critique concerns sexual misconduct proceedings, particularly on campus. As Chapter 5 described, a group of Harvard Law School professors, including feminists such as Janet Halley and Jeannie Suk Gersen, criticized university procedures adopted after 2011 for lacking basic protections for the accused. Their argument was not that sexual assault should be ignored but that procedures which presume guilt, deny the accused access to evidence or combine investigative and adjudicative roles produce unreliable results and, in the long run, undermine the legitimacy of efforts to address sexual violence. The succession of federal regulations since then shows that this critique has had considerable influence. A related liberal critique targets the dominance approach's treatment of women's agency. If apparent consent can always be recharacterized as the product of male power, critics argue, the theory deprives women of the capacity to make choices that the law will respect, which is the very capacity feminism fought to secure. The surrogacy debate and the debate over sex work illustrate the point. Some feminists argue that commercial surrogacy and prostitution are inherently exploitative and should be prohibited or restricted, often through the "Nordic model," adopted by Sweden in 1999, which criminalizes the purchase of sex but not its sale. Others argue that such rules substitute the state's judgment for women's own, and that the harms associated with sex work are largely caused by criminalization itself. Both positions are held by people who call themselves feminists, and the disagreement is ultimately about whose account of women's interests the law should adopt. The conservative and natural-difference critique The third family of criticism challenges the premise that most legal distinctions between the sexes reflect social construction rather than nature. Its proponents argue that men and women differ in physical strength, in reproductive biology and, on average, in some psychological traits and interests, and that a legal order which takes those differences into account is not oppressive but reasonable. Kingsley Browne's Biology at Work (2002) argued that average sex differences in temperament and interests, grounded in evolutionary biology, explain much of the occupational segregation and pay gap that discrimination law seeks to remedy, and that law premised on the assumption that the sexes are interchangeable will produce frustration and unfair results. Social conservative critics go further, arguing that the traditional family, with its differentiation of roles, serves children and society well, and that feminist legal reforms have weakened it, for example through no-fault divorce. Some scholars in the natural law tradition argue that law should recognize and support sex complementarity. The Supreme Court has partly accepted a version of the difference argument, in the "real differences" cases discussed in Chapter 8, from Michael M. to Nguyen, and most recently in the sports decisions. United States v. Virginia itself acknowledged that inherent differences between men and women remain cause for celebration, while insisting that they may not be used to create or perpetuate the legal, social and economic inferiority of women. Feminist legal theorists respond that the natural-difference critique typically proves less than it claims. Average differences, even where real, describe distributions that overlap substantially, and the law's traditional sex classifications applied to every member of each sex. The history of women's exclusion from law, medicine and the professions, which was defended at the time as a reflection of natural differences, suggests caution about any claim that current distributions are natural. And the question whether a difference is biological does not settle the legal question: pregnancy is biological, but whether a pregnant worker keeps her job is a matter of law. The difference feminists of Chapter 2 would add that the conservative critique and their own position agree that sex differences matter; they disagree about whether the law should penalize or accommodate them. Critiques of intersectionality The fourth family targets intersectionality specifically, and it comes from several directions. From the right, critics argue that intersectionality has become a hierarchy of victimhood, sometimes called "oppression Olympics," in which moral authority is assigned according to the number of disadvantaged identities one holds, and that this is incompatible with the liberal principle of treating individuals as individuals. The Supreme Court's emphasis in Ames v. Ohio Department of Youth Services (2025) that Title VII protects every individual against discrimination, without special rules for majority or minority groups, and its decision in Students for Fair Admissions v. Harvard (2023) rejecting race-conscious university admissions, reflect a jurisprudence skeptical of group-based analysis generally. Crenshaw and her defenders reply that this critique confuses the legal concept with its later popular uses. As Chapter 3 explained, legal intersectionality is a claim about how discrimination operates and how the single-axis framework fails to see it. It is, if anything, an individualizing argument: it insists that a Black woman's claim be assessed on the basis of what actually happened to her, rather than being forced into a category designed for someone else. Some scholars have argued that Bostock's but-for test and Ames's insistence on individual treatment can be read to support intersectional claims for exactly this reason. From the left and from within Black feminist thought, a different critique has emerged. Jennifer Nash, in "Re-thinking Intersectionality" (2008) and Black Feminism Reimagined (2019), argued that intersectionality has been institutionalized in ways that make Black women a symbol of diversity in academic and corporate settings without addressing their material conditions, and that the concept has sometimes been used in ways that obscure differences among Black women. Other scholars have raised the problem of infinite regress: if every combination of characteristics is a distinct category, and every category conceals further internal differences, the analysis may never reach a stable group for which the law can provide a remedy. Devon Carbado and others have responded that intersectionality was never meant to produce an exhaustive list of categories, but to direct attention to how particular categories are produced and deployed in particular legal settings. A third, practical critique concerns administrability. Antidiscrimination law requires courts to compare claimants with others. The more finely claimants are defined, the harder it becomes to identify comparators and to generate meaningful statistical evidence. Proponents respond that courts manage similar complexity in many areas of law, and that the difficulty of proof is an argument for developing better methods rather than for denying the claim. The United Kingdom's decision not to bring its combined discrimination provision into force, discussed in Chapter 3, shows that governments have found the administrability concern persuasive, though the provision was dropped on grounds of regulatory burden rather than on any showing that it would be unworkable. Sex, gender and the divided house The fifth and most recent dispute divides feminist legal theory from within, and it has become one of the most consequential and acrimonious debates in the field. It concerns the relationship between sex, understood as a biological category, and gender identity. One position, broadly associated with the postmodern and intersectional traditions and with queer theory, holds that the legal category of sex has always been partly a social construction, that laws restricting transgender people enforce the same stereotypes about how men and women should look and live that sex discrimination law was designed to dismantle, and that the recognition of transgender people's rights extends rather than threatens feminist goals. On this view, Price Waterhouse's prohibition of sex stereotyping and Bostock's but-for test lead naturally to protection for transgender people, and Skrmetti repeats the error of Geduldig. The other position, often called gender-critical feminism and associated with a range of scholars and activists, particularly in the United Kingdom, holds that sex is an immutable biological reality, that women's oppression has been based on sex rather than on identity, and that legal protections such as single-sex spaces, sports and data collection depend on a stable definition of sex. In For Women Scotland Ltd v The Scottish Ministers, decided in April 2025, the United Kingdom Supreme Court held that the terms "woman" and "sex" in the Equality Act 2010 refer to biological sex, a decision welcomed by gender-critical feminists and criticized by transgender advocates. The American Supreme Court's decisions in Skrmetti and B.P.J. reached results consistent with the gender-critical position, though on different reasoning. Each side accuses the other of abandoning feminism. Each claims to be defending women, and each relies on arguments drawn from the field's own history. The dispute shows, perhaps more clearly than any other, that feminist legal theory is not a single program but a set of methods whose application depends on contested premises about what sex is and whose experience counts as women's experience. It is the anti-essentialist question of Chapter 2, "which women?", asked in a new form, and neither side has found an answer that the other accepts. Assessing the critiques What survives this criticism? Three conclusions seem fair. First, the core descriptive claim of feminist legal theory, that ostensibly neutral legal rules have historically reproduced sex-based hierarchy, is well established and is not seriously contested by most critics as a matter of history. Coverture, the marital rape exemption, the chastisement doctrine, protective labor laws and the exclusion of women from juries and professions are matters of record. The disputes are about the present, not the past. Second, the stronger claims, that contemporary disparities are primarily the product of discrimination or patriarchal structures rather than of choice or biology, are genuinely contested, and the answer varies by domain. In some areas, such as audit studies of hiring and bargaining, the evidence of differential treatment is strong. In others, such as the pay gap between workers with identical jobs and hours, it is weaker, and the more plausible story concerns the interaction of work structures with family responsibilities. Feminist legal theory is strongest when it makes specific, testable claims about how specific rules operate, and weakest when it offers totalizing accounts that no evidence could refute. Third, the internal critiques, from Black feminists, anti-carceral scholars, civil libertarian feminists and others, have been the source of the field's most important corrections. Intersectionality began as a criticism of feminism; the anti-carceral critique began as a criticism of feminist reforms; the sex-positive critique began as a criticism of feminist campaigns against pornography. A field that has absorbed such criticism is not a closed ideology. Whether it can absorb the current dispute over sex and gender, which cuts to its definition of its own subject, is the open question. Conclusion: Neutrality as a Question The five women who sued General Motors in 1976 lost. The court told them that their claim, as Black women, did not fit the categories the law had built. Half a century later, most federal courts would allow their claim to proceed, the Equal Employment Opportunity Commission would recognize it, and the word coined to describe their problem is known around the world. That trajectory is a fair emblem of what feminist legal theory has accomplished and of what remains unsettled. The theory changed what the law could see. It did not settle what the law should do once it saw it. The chapters of this book have traced a single argument through several fields. In property, contract, criminal law and constitutional law, the claim of neutrality has repeatedly concealed a default subject, an unmarked baseline or an unexamined distribution. The married woman's earnings were hers under the new statutes, unless she earned them at home. The law of rape protected women who resisted, which in practice meant it protected few. The police would protect a woman from assault, unless the assailant was her husband, in which case the matter was private. The labor contract was a bargain between equals, unless one of them was pregnant, or caring for a parent, or bargaining for a car on the South Side of Chicago. And under slavery, the law that governed white women's property governed Black women as property, a fact that any account of "women's" legal history has to confront. What follows from all of this? Three things, of increasing difficulty. Asking the question is not optional The first lesson is methodological, and it is the one on which the field's friends and many of its critics can agree. Any legal rule that allocates rights, burdens or protections should be examined for how it operates across the people it governs, including those at the margins of every relevant category. Bartlett's "woman question" and Matsuda's "looking to the bottom" are not ideological commitments; they are forms of due diligence. Courts and legislatures that fail to ask them do not thereby achieve neutrality. They simply fail to notice the effects of their choices. This is a modest claim, and its modesty is its strength. It does not presume that every disparity is discriminatory. The Weitzman episode, the replication difficulties of the Minneapolis arrest experiment and Goldin's work on the pay gap all show that the answers can surprise, and sometimes cut against feminist expectations. But the history of coverture, chastisement and the marital exemption shows what happens when no one asks. Those doctrines endured for centuries not because they survived scrutiny but because they were rarely subjected to it. Remedies have their own distributions The second lesson is harder, and it comes largely from feminism's internal critics. Identifying that a rule is not neutral does not tell you what to replace it with, and the replacement will have its own distributive effects. Mandatory arrest was meant to correct police indifference; it also produced dual arrests and took decisions out of victims' hands. Weakened procedural protections in campus proceedings were meant to correct institutional neglect of sexual assault; they also produced findings of responsibility that courts and regulators later judged unreliable. Anti-pornography ordinances were meant to address the subordination of women; in Canada, the obscenity doctrine they influenced was turned against gay and lesbian bookstores. The intersectional method, applied consistently, requires that the same scrutiny directed at old rules be directed at new ones. That is uncomfortable, because it means that some of feminist legal reform's most celebrated achievements must be re-examined in light of who bore their costs. But it is what the method demands, and the scholars who have undertaken that re-examination, from Crenshaw in "Mapping the Margins" to Richie, Goodmark and Gruber, have strengthened the field rather than weakened it. A theory that exempts its own remedies from its own critique has become the kind of neutral-seeming structure it set out to challenge. The category itself is contested The third lesson is the hardest, and it is where the field now stands. Feminist legal theory has always depended on the category "women," whether as a group to be treated equally, a group whose differences deserve recognition, a group subordinated by men, or a group whose internal diversity must be respected. The anti-essentialist critique asked which women the category described. The current dispute over sex and gender identity asks what the category is. Both sides of that dispute invoke feminist principles and feminist history, and the courts, in the United States and the United Kingdom, have for now adopted definitions rooted in biological sex. It would be premature to predict how this will be resolved, or whether it will be. What can be said is that the dispute is not a departure from feminist legal theory but a continuation of its central problem. Every significant development in the field, from Pauli Murray's "Jane Crow" to Crenshaw's intersection, has arisen from someone pointing out that the law's categories, including feminism's own, did not describe their experience. The field has survived those challenges by taking them seriously, sometimes after resisting them for years. The unsettled constitution Meanwhile, American constitutional law has moved in a direction that most feminist legal theorists regard as regression. Dobbs rejected the equality argument for reproductive rights in a single paragraph that relied on Geduldig. Skrmetti applied similar reasoning to laws affecting transgender minors, and the sports cases of 2026 extended the Court's acceptance of sex-based distinctions grounded in biology. The Equal Rights Amendment remains outside the Constitution, its status declared by a departing President and denied by the Archivist. The Court has also, in Rahimi, upheld firearm restrictions for those subject to domestic violence orders, and in Muldrow and Ames has made it somewhat easier to bring discrimination claims under Title VII, so the picture is not uniform. For readers who accept the feminist critique, the lesson of the constitutional story is that much of the future of sex equality law will be written in statutes, state constitutions, administrative rules and private institutions rather than in the Supreme Court's interpretation of the Equal Protection Clause. The Pregnant Workers Fairness Act, state paid-leave laws, CROWN Acts and state constitutional amendments on reproduction are examples. For readers skeptical of the critique, the same developments are a return of contested moral questions to democratic decision, where they arguably belong. Both readings are compatible with the historical record. What neither can do is return to the assumption, common for most of American history, that the law's general words were addressed equally to everyone. What the reader should take away Feminist legal theory is sometimes presented, by admirers and detractors alike, as a set of conclusions: that the law is patriarchal, that women are oppressed, that the remedy is more regulation. This book has tried to show that its most durable contribution is something different. It is a discipline of attention. It asks who a rule was written for, who bears its costs, whose experience its categories describe and whose they omit. It asks those questions of old rules and new ones, of opponents' proposals and of its own. Its practitioners have been wrong about particular facts and particular remedies, and they have disagreed with one another as sharply as with anyone. But the question they insisted on asking, whether the law's neutrality is real, has proved to be the right question often enough that no serious student of law can now avoid it. The women at General Motors were not asking for special treatment. They were asking the court to look at what had actually happened to them. That is what the best of this tradition has always asked, and it remains a reasonable thing to ask of any legal system that claims to treat all persons equally. Notes Introduction 1. DeGraffenreid v. General Motors Assembly Division, 413 F. Supp. 142 (E.D. Mo. 1976), aff'd in part and rev'd in part, 558 F.2d 480 (8th Cir. 1977). 1. Kimberlé Crenshaw, "Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics," University of Chicago Legal Forum 1989. 2. Meritor Savings Bank, FSB v. Vinson, 477 U.S. 57 (1986). Chapter 1 1. William Blackstone, Commentaries on the Laws of England, Book 1, ch. 15 (1765). 1. Bradwell v. Illinois, 83 U.S. (16 Wall.) 130 (1873) (Bradley, J., concurring). 2. Minor v. Happersett, 88 U.S. (21 Wall.) 162 (1875); Muller v. Oregon, 208 U.S. 412 (1908). 3. Ellison v. Brady, 924 F.2d 872 (9th Cir. 1991); Harris v. Forklift Systems, Inc., 510 U.S. 17 (1993). 4. Personnel Administrator of Massachusetts v. Feeney, 442 U.S. 256 (1979); Griggs v. Duke Power Co., 401 U.S. 424 (1971); Dothard v. Rawlinson, 433 U.S. 321 (1977). 5. Joan Williams, Unbending Gender: Why Family and Work Conflict and What to Do About It (Oxford University Press, 2000). 6. Reva B. Siegel, "'The Rule of Love': Wife Beating as Prerogative and Privacy," Yale Law Journal 105 (1996); State v. Rhodes, 61 N.C. 453 (1868). 7. Griswold v. Connecticut, 381 U.S. 479 (1965); Eisenstadt v. Baird, 405 U.S. 438 (1972); Roe v. Wade, 410 U.S. 113 (1973); Dobbs v. Jackson Women's Health Organization, 597 U.S. 215 (2022). 8. Katharine T. Bartlett, "Feminist Legal Methods," Harvard Law Review 103 (1990). 9. Mari J. Matsuda, "Looking to the Bottom: Critical Legal Studies and Reparations," Harvard Civil Rights-Civil Liberties Law Review 22 (1987). Chapter 2 1. Reed v. Reed, 404 U.S. 71 (1971); Frontiero v. Richardson, 411 U.S. 677 (1973); Craig v. Boren, 429 U.S. 190 (1976); Weinberger v. Wiesenfeld, 420 U.S. 636 (1975); United States v. Virginia, 518 U.S. 515 (1996). 1. Geduldig v. Aiello, 417 U.S. 484 (1974); General Electric Co. v. Gilbert, 429 U.S. 125 (1976); Pregnancy Discrimination Act of 1978, Pub. L. No. 95-555. 2. Wendy W. Williams, "The Equality Crisis: Some Reflections on Culture, Courts, and Feminism," Women's Rights Law Reporter 7 (1982). 3. Linda J. Krieger and Patricia N. Cooney, "The Miller-Wohl Controversy: Equal Treatment, Positive Action and the Meaning of Women's Equality," Golden Gate University Law Review 13 (1983). 4. California Federal Savings & Loan Ass'n v. Guerra, 479 U.S. 272 (1987); Young v. United Parcel Service, Inc., 575 U.S. 206 (2015); Pregnant Workers Fairness Act, 42 U.S.C. § 2000gg et seq. 5. Carol Gilligan, In a Different Voice: Psychological Theory and Women's Development (Harvard University Press, 1982). 6. Robin West, "Jurisprudence and Gender," University of Chicago Law Review 55 (1988). 7. Catharine A. MacKinnon, "Feminism, Marxism, Method, and the State: An Agenda for Theory," Signs 7 (1982); Sexual Harassment of Working Women (Yale University Press, 1979); Feminism Unmodified (Harvard University Press, 1987). 8. American Booksellers Ass'n v. Hudnut, 771 F.2d 323 (7th Cir. 1985), summarily aff'd, 475 U.S. 1001 (1986); R. v. Butler, [1992] 1 S.C.R. 452. 9. Angela P. Harris, "Race and Essentialism in Feminist Legal Theory," Stanford Law Review 42 (1990). 10. Patricia J. Williams, The Alchemy of Race and Rights (Harvard University Press, 1991). 11. Mary Joe Frug, "A Postmodern Feminist Legal Manifesto (An Unfinished Draft)," Harvard Law Review 105 (1992). 12. Janet Halley, Split Decisions: How and Why to Take a Break from Feminism (Princeton University Press, 2006). Chapter 3 1. Crenshaw, "Demarginalizing the Intersection of Race and Sex," cited above. 1. Moore v. Hughes Helicopters, Inc., 708 F.2d 475 (9th Cir. 1983); Payne v. Travenol Laboratories, Inc., 673 F.2d 798 (5th Cir. 1982). 2. Jefferies v. Harris County Community Action Ass'n, 615 F.2d 1025 (5th Cir. 1980); Lam v. University of Hawaii, 40 F.3d 1551 (9th Cir. 1994). 3. Pauli Murray and Mary O. Eastwood, "Jane Crow and the Law: Sex Discrimination and Title VII," George Washington Law Review 34 (1965); White v. Crook, 251 F. Supp. 401 (M.D. Ala. 1966). 4. Serena Mayeri, Reasoning from Race: Feminism, Law, and the Civil Rights Revolution (Harvard University Press, 2011). 5. Combahee River Collective, "A Black Feminist Statement" (1977). 6. Kimberlé Crenshaw, "Mapping the Margins: Intersectionality, Identity Politics, and Violence Against Women of Color," Stanford Law Review 43 (1991). 7. Phillips v. Martin Marietta Corp., 400 U.S. 542 (1971); Rogers v. American Airlines, Inc., 527 F. Supp. 229 (S.D.N.Y. 1981); EEOC v. Catastrophe Management Solutions, 852 F.3d 1018 (11th Cir. 2016). 8. Bostock v. Clayton County, 590 U.S. 644 (2020); Muldrow v. City of St. Louis, 601 U.S. 346 (2024); Ames v. Ohio Department of Youth Services, 605 U.S. ___ (2025). 9. Equality Act 2010 (UK), s. 14; Andrews v. Law Society of British Columbia, [1989] 1 S.C.R. 143; Constitution of the Republic of South Africa, 1996, s. 9(3); Committee on the Elimination of Discrimination against Women, General Recommendation No. 28 (2010). Chapter 4 1. Reva B. Siegel, "The Modernization of Marital Status Law: Adjudicating Wives' Rights to Earnings, 1860–1930," Georgetown Law Journal 82 (1994). 1. Kirchberg v. Feenstra, 450 U.S. 455 (1981). 2. O'Brien v. O'Brien, 66 N.Y.2d 576 (1985). 3. Lenore J. Weitzman, The Divorce Revolution (Free Press, 1985); Richard R. Peterson, "A Re-Evaluation of the Economic Consequences of Divorce," American Sociological Review 61 (1996). 4. Martha Albertson Fineman, The Neutered Mother, the Sexual Family and Other Twentieth Century Tragedies (Routledge, 1995); The Autonomy Myth: A Theory of Dependency (New Press, 2004). 5. Adrienne D. Davis, "The Private Law of Race and Sex: An Antebellum Perspective," Stanford Law Review 51 (1999). 6. Melton A. McLaurin, Celia, a Slave (University of Georgia Press, 1991). 7. Cheryl I. Harris, "Whiteness as Property," Harvard Law Review 106 (1993). 8. Matthew Desmond, Evicted: Poverty and Profit in the American City (Crown, 2016). 9. Equal Credit Opportunity Act, 15 U.S.C. § 1691 et seq.; Uniform Law Commission, Uniform Partition of Heirs Property Act (2010). Chapter 5 1. Matthew Hale, Historia Placitorum Coronae: The History of the Pleas of the Crown (1736). 1. Susan Estrich, "Rape," Yale Law Journal 95 (1986); Real Rape (Harvard University Press, 1987). 2. Rusk v. State, 43 Md. App. 476 (1979), rev'd, 289 Md. 230 (1981). 3. Federal Rule of Evidence 412; People v. Rincon-Pineda, 14 Cal. 3d 864 (1975). 4. People v. Liberta, 64 N.Y.2d 152 (1984). 5. State in the Interest of M.T.S., 129 N.J. 422 (1992); Commonwealth v. Berkowitz, 537 Pa. 143 (1994). 6. Sexual Offences Act 2003 (England and Wales), ss. 1, 74; R. v. Ewanchuk, [1999] 1 S.C.R. 330. 7. Powell v. Alabama, 287 U.S. 45 (1932); Coker v. Georgia, 433 U.S. 584 (1977). 8. Danielle L. McGuire, At the Dark End of the Street: Black Women, Rape, and Resistance (Knopf, 2010), on Recy Taylor. 9. Janet Halley, "Trading the Megaphone for the Gavel in Title IX Enforcement," Harvard Law Review Forum 128 (2015). 10. Tennessee v. Cardona (E.D. Ky., decided Jan. 9, 2025). 11. Cassia Spohn and Julie Horney, Rape Law Reform: A Grassroots Revolution and Its Impact (Plenum, 1992). Chapter 6 1. Bradley v. State, 1 Miss. (1 Walker) 156 (1824); Fulgham v. State, 46 Ala. 143 (1871). 1. Siegel, "'The Rule of Love,'" cited above. 2. Thurman v. City of Torrington, 595 F. Supp. 1521 (D. Conn. 1984). 3. Lawrence W. Sherman and Richard A. Berk, "The Specific Deterrent Effects of Arrest for Domestic Assault," American Sociological Review 49 (1984). 4. United States v. Morrison, 529 U.S. 598 (2000); Town of Castle Rock v. Gonzales, 545 U.S. 748 (2005); DeShaney v. Winnebago County Department of Social Services, 489 U.S. 189 (1989). 5. Inter-American Commission on Human Rights, Jessica Lenahan (Gonzales) et al. v. United States, Report No. 80/11 (2011). 6. United States v. Rahimi, 602 U.S. 680 (2024); New York State Rifle & Pistol Ass'n v. Bruen, 597 U.S. 1 (2022). 7. State v. Wanrow, 88 Wash. 2d 221 (1977); State v. Norman, 324 N.C. 253 (1989). 8. Lenore E. Walker, The Battered Woman (Harper & Row, 1979); Elizabeth M. Schneider, Battered Women and Feminist Lawmaking (Yale University Press, 2000). 9. Evan Stark, Coercive Control: How Men Entrap Women in Personal Life (Oxford University Press, 2007); Serious Crime Act 2015, s. 76; Domestic Abuse (Scotland) Act 2018. 10. Beth E. Richie, Arrested Justice: Black Women, Violence, and America's Prison Nation (NYU Press, 2012); Leigh Goodmark, A Troubled Marriage: Domestic Violence and the Legal System (NYU Press, 2012) and Decriminalizing Domestic Violence (University of California Press, 2018); Aya Gruber, The Feminist War on Crime (University of California Press, 2020). 11. Ferguson v. City of Charleston, 532 U.S. 67 (2001). Chapter 7 1. Carole Pateman, The Sexual Contract (Stanford University Press, 1988). 1. Borelli v. Brusseau, 12 Cal. App. 4th 647 (1993); Simeone v. Simeone, 525 Pa. 392 (1990). 2. Marvin v. Marvin, 18 Cal. 3d 660 (1976); Clare Dalton, "An Essay in the Deconstruction of Contract Doctrine," Yale Law Journal 94 (1985). 3. Williams v. Walker-Thomas Furniture Co., 350 F.2d 445 (D.C. Cir. 1965). 4. Mary Joe Frug, "Re-reading Contracts: A Feminist Analysis of a Contracts Casebook," American University Law Review 34 (1985). 5. Ian Ayres, "Fair Driving: Gender and Race Discrimination in Retail Car Negotiations," Harvard Law Review 104 (1991). 6. Adkins v. Children's Hospital, 261 U.S. 525 (1923); West Coast Hotel Co. v. Parrish, 300 U.S. 379 (1937). 7. Price Waterhouse v. Hopkins, 490 U.S. 228 (1989); Oncale v. Sundowner Offshore Services, Inc., 523 U.S. 75 (1998). 8. Claudia Goldin, Career and Family: Women's Century-Long Journey toward Equity (Princeton University Press, 2021). 9. Ledbetter v. Goodyear Tire & Rubber Co., 550 U.S. 618 (2007); Lilly Ledbetter Fair Pay Act of 2009, Pub. L. No. 111-2; Wal-Mart Stores, Inc. v. Dukes, 564 U.S. 338 (2011). 10. Nevada Department of Human Resources v. Hibbs, 538 U.S. 721 (2003); Coleman v. Court of Appeals of Maryland, 566 U.S. 30 (2012). 11. In re Baby M, 109 N.J. 396 (1988); Johnson v. Calvert, 5 Cal. 4th 84 (1993). Chapter 8 1. Mississippi University for Women v. Hogan, 458 U.S. 718 (1982); J.E.B. v. Alabama ex rel. T.B., 511 U.S. 127 (1994); Hoyt v. Florida, 368 U.S. 57 (1961); Taylor v. Louisiana, 419 U.S. 522 (1975). 1. Linda K. Kerber, No Constitutional Right to Be Ladies: Women and the Obligations of Citizenship (Hill and Wang, 1998). 2. Michael M. v. Superior Court, 450 U.S. 464 (1981); Rostker v. Goldberg, 453 U.S. 57 (1981); Nguyen v. INS, 533 U.S. 53 (2001); Sessions v. Morales-Santana, 582 U.S. 47 (2017). 3. Reva Siegel, "Reasoning from the Body: A Historical Perspective on Abortion Regulation and Questions of Equal Protection," Stanford Law Review 44 (1992). 4. Bray v. Alexandria Women's Health Clinic, 506 U.S. 263 (1993); Planned Parenthood of Southeastern Pennsylvania v. Casey, 505 U.S. 833 (1992). 5. Ruth Bader Ginsburg, "Some Thoughts on Autonomy and Equality in Relation to Roe v. Wade," North Carolina Law Review 63 (1985). 6. FDA v. Alliance for Hippocratic Medicine, 602 U.S. 367 (2024). 7. Buck v. Bell, 274 U.S. 200 (1927); Skinner v. Oklahoma ex rel. Williamson, 316 U.S. 535 (1942); Relf v. Weinberger, 372 F. Supp. 1196 (D.D.C. 1974); Madrigal v. Quilligan (C.D. Cal. 1978). 8. Dorothy Roberts, Killing the Black Body: Race, Reproduction, and the Meaning of Liberty (Pantheon, 1997). 9. Loretta J. Ross and Rickie Solinger, Reproductive Justice: An Introduction (University of California Press, 2017). 10. United States v. Skrmetti, 605 U.S. ___ (2025); West Virginia v. B.P.J. and Little v. Hecox (U.S. decided June 30, 2026). 11. Illinois v. Ferriero, 60 F.4th 704 (D.C. Cir. 2023). Chapter 9 1. Richard A. Posner, Sex and Reason (Harvard University Press, 1992); Richard A. Epstein, Forbidden Grounds: The Case Against Employment Discrimination Laws (Harvard University Press, 1992). 1. Nadine Strossen, Defending Pornography: Free Speech, Sex, and the Fight for Women's Rights (Scribner, 1995). 2. Kingsley R. Browne, Biology at Work: Rethinking Sexual Equality (Rutgers University Press, 2002). 3. Students for Fair Admissions, Inc. v. President and Fellows of Harvard College, 600 U.S. 181 (2023). 4. Jennifer C. Nash, "Re-thinking Intersectionality," Feminist Review 89 (2008); Black Feminism Reimagined: After Intersectionality (Duke University Press, 2019). 5. Devon W. Carbado, "Colorblind Intersectionality," Signs 38 (2013); Sumi Cho, Kimberlé Williams Crenshaw and Leslie McCall, "Toward a Field of Intersectionality Studies: Theory, Applications, and Praxis," Signs 38 (2013). 6. For Women Scotland Ltd v The Scottish Ministers [2025] UKSC 16. Further Reading Katharine T. Bartlett and Rosanne Kennedy, eds., Feminist Legal Theory: Readings in Law and Gender (Westview Press, 1991).** A collection of foundational articles from the field's formative period, useful for reading the early debates in their original form. Kimberlé Crenshaw, "Demarginalizing the Intersection of Race and Sex" (1989) and "Mapping the Margins" (1991). The two articles that introduced and developed intersectionality in legal scholarship. Short, readable and far more precise than most secondary accounts. Catharine A. MacKinnon, Feminism Unmodified: Discourses on Life and Law (Harvard University Press, 1987).** Lectures and essays setting out the dominance approach in its most accessible form, including the essay "Difference and Dominance." Robin West, "Jurisprudence and Gender," University of Chicago Law Review (1988).** The central statement of relational feminist jurisprudence and a challenge to both liberal and critical legal theory. Angela P. Harris, "Race and Essentialism in Feminist Legal Theory," Stanford Law Review (1990).** The most influential anti-essentialist critique of the major feminist theorists, and still the best entry point to that debate. Reva B. Siegel, "'The Rule of Love': Wife Beating as Prerogative and Privacy," Yale Law Journal (1996).** A model of feminist legal history, introducing the concept of "preservation through transformation." Serena Mayeri, Reasoning from Race: Feminism, Law, and the Civil Rights Revolution (Harvard University Press, 2011).** A history of how feminist lawyers, including Pauli Murray and Ruth Bader Ginsburg, drew on the race analogy, and what that strategy gained and lost. Dorothy Roberts, Killing the Black Body: Race, Reproduction, and the Meaning of Liberty (Pantheon, 1997).** The foundational intersectional account of reproductive regulation in American law. Martha Albertson Fineman, The Autonomy Myth: A Theory of Dependency (New Press, 2004).** A critique of the liberal ideal of the autonomous individual, with implications for family, property and welfare law. Aya Gruber, The Feminist War on Crime: The Unexpected Role of Women's Liberation in Mass Incarceration (University of California Press, 2020).** The fullest statement of the internal critique of feminist reliance on criminal law, whether or not one accepts its conclusions. Janet Halley, Split Decisions: How and Why to Take a Break from Feminism (Princeton University Press, 2006).** A demanding critique from within, arguing that feminism's focus on male dominance can obscure other forms of power. Claudia Goldin, Career and Family: Women's Century-Long Journey toward Equity (Princeton University Press, 2021).** An economist's account of the modern gender pay gap, essential for anyone assessing claims about discrimination in the labor market. Nadine Strossen, Defending Pornography (Scribner, 1995).** The leading civil libertarian and feminist response to the anti-pornography campaigns of the 1980s. Hashtags: #FeministLegalTheory #FeministJurisprudence #Intersectionality #IntersectionalFeminism #Patriarchy #PatriarchalStructures #GenderAndLaw #WomenAndTheLaw #GenderEquality #SexEquality #SubstantiveEquality #FormalEquality #GenderDiscrimination #SexDiscrimination #AntiDiscriminationLaw #FeministLegalStudies #CriticalLegalTheory #CriticalRaceTheory #BlackFeminism #KimberleCrenshaw #RuthBaderGinsburg #CatharineMacKinnon #FeministLegalMethods #AskingTheWomanQuestion #AntiEssentialism #DominanceTheory #RelationalFeminism #EthicsOfCare #LegalNeutrality #StructuralInequality #GenderJustice #WomensRights #CivilRightsLaw #EqualProtectionClause #TitleVII #EmploymentDiscrimination #SexualHarassmentLaw #WorkplaceEquality #PregnancyDiscrimination #ReproductiveRights #ReproductiveJustice #DomesticViolenceLaw #SexualViolenceLaw #MaritalRapeLaw #Coverture #MaritalPropertyRights #UnpaidCareWork #PublicPrivateDivide #IntersectionalDiscrimination #FeministLegalReform
- Empirical Legal Studies (Using Data to Understand Judicial Behavior)
Download the Book (PDF): Introduction In September 2005, at the opening of his confirmation hearing, John Roberts offered the Senate Judiciary Committee a metaphor that has followed him ever since. Judges, he said, are like umpires. Umpires do not make the rules; they apply them. His job, if confirmed, would be to call balls and strikes, and not to pitch or bat. Thirteen years later, as Chief Justice, Roberts returned to the theme in an unusual public rebuke of a sitting president, insisting that the country does not have Obama judges or Trump judges, Bush judges or Clinton judges, but an independent judiciary of dedicated judges doing their level best to do equal right to those appearing before them. Both statements express an ideal that almost every lawyer shares and that the legitimacy of courts depends upon. Both are also, in a precise sense, empirical claims. Whether judges appointed by different presidents decide cases differently is not a matter of faith or of constitutional theory. It is a question about patterns in observable behavior, and it can be investigated in the same way a public health researcher investigates whether a drug lowers blood pressure: by collecting cases, measuring outcomes, accounting for the other things that vary, and asking whether a difference remains once those other things are held constant. That kind of investigation is the business of empirical legal studies, a field that has grown over the past half-century from a handful of political scientists counting Supreme Court votes into a large, methodologically sophisticated enterprise spanning law schools, economics departments, and sentencing commissions. Its researchers have coded every Supreme Court decision since 1946 along hundreds of dimensions. They have built statistical estimates of where each Justice sits on an ideological spectrum, term by term, since the New Deal. They have linked millions of federal criminal cases from arrest to sentence, compared the grant rates of immigration judges sitting in the same building, and measured how often plaintiffs who win at trial keep their victories on appeal. The results are not comfortable for anyone who holds a simple view of judging. They are uncomfortable for the umpire theory, because on the cases that reach the Supreme Court, and on a meaningful slice of cases in the federal courts of appeals, the party of the appointing president predicts votes with a regularity that doctrine alone cannot explain. But they are equally uncomfortable for the cynic who believes law is merely politics in robes. Most appellate decisions are unanimous. Ideological differences shrink or vanish in whole categories of cases. Judges moderate one another on mixed panels. Precedent and the structure of legal rules measurably constrain outcomes. The picture that emerges is neither pure law nor pure politics, and the most interesting findings concern exactly how the two interact. The argument of this book This booklet makes a single argument: that data can reveal real and important patterns in judicial behavior, including ideological voting and racial disparity, but that what the data reveal depends at every step on choices about measurement and inference, and that the value of empirical legal studies lies as much in its discipline about what numbers cannot prove as in the striking numbers themselves. That argument cuts in two directions. Against the defenders of the judiciary who dismiss empirical findings as crude, it insists that careful quantitative work has established things about courts that no amount of reading opinions would reveal. The United States Sentencing Commission's own analysis of federal sentences between fiscal years 2017 and 2021 found that Black men received sentences about 13 percent longer than similarly situated white men. That is a finding a lawyer reading individual sentencing transcripts would never see, because each sentence, taken alone, has its own reasons. Against the enthusiasts who treat a regression coefficient as a verdict, it insists on the questions that separate evidence from rhetoric. What was counted, and who decided how to code it? Which cases never reached the dataset because they settled, were never charged, or were never appealed? Is the difference between two groups of judges caused by the judges, or by the cases they happen to hear? Does a disparity in sentences reflect bias at sentencing, or decisions made earlier by police and prosecutors that the sentencing data inherit? These questions do not dissolve the findings. They sharpen them, and in several well-known debates they have changed what the findings turned out to mean. How the book proceeds The first chapter traces how the study of courts came to be quantitative, from Oliver Wendell Holmes's prediction in 1897 that the future of legal study belonged to "the man of statistics," through the legal realists and the political scientists who first counted judicial votes, to the modern field. It also sets out the basic logic of inference that the rest of the book relies upon. The second chapter examines the datasets themselves, the Supreme Court Database, the courts of appeals databases, the Federal Judicial Center's case records, and the Sentencing Commission's offender files, and what it means that someone had to decide how to code each case. The third chapter asks how anyone measures something as elusive as a judge's ideology, and why the leading measures, from the party of the appointing president to the Martin-Quinn scores, each carry a distinctive blind spot. Chapters four and five turn to ideological voting: first on the Supreme Court, where the attitudinal model was born and where it faces its strongest and weakest tests, and then on the federal courts of appeals, where the random composition of three-judge panels has produced some of the most elegant natural experiments in the field. Chapters six and seven take up race: the long, contested record of racial disparity in federal sentencing, and then the decisions before and around sentencing, bail, jury selection, and capital charging, where a different body of evidence tells an overlapping story. Chapter eight examines the individual judge as a source of variation, the so-called noise in adjudication, including the famous and famously disputed claim that judges become harsher as lunch approaches. Chapter nine asks the practitioner's question: what does the evidence say about which litigation strategies actually succeed? The conclusion considers where the field is heading, as commercial analytics and machine prediction bring these methods into everyday practice and into political fights over judge shopping and the assignment of cases. Throughout, the book concentrates on American courts, where the data are richest and the debates most developed, with comparative notes where other systems illuminate a point. It names real studies and real datasets, and where a figure is reported it comes from a published source identified in the Notes. It assumes no statistical training. Where a concept such as selection bias, an instrumental variable, or an ideal point matters to the argument, the book explains it in plain terms at the moment it becomes necessary. This is a book about how courts behave, written for readers who want to understand the evidence. It is not legal advice. A final word on attitude. It is tempting to read empirical findings about judges as an exposé, and some of them are. But the enterprise at its best is an act of respect for law rather than a debunking of it. To measure whether like cases are treated alike is to take seriously the promise that they should be. The Sentencing Commission, a body created by Congress within the judicial branch, publishes its findings on racial disparity because the rule of law is a claim about outcomes, not merely about intentions, and claims about outcomes can be checked. The chapters that follow try to show what that checking has found, how much confidence it deserves, and what it asks of the institutions it examines. Chapter 1: The Man of Statistics In January 1897, Oliver Wendell Holmes Jr., then a justice of the Supreme Judicial Court of Massachusetts, delivered an address at the dedication of a new hall at Boston University School of Law. Published that spring as "The Path of the Law," it became one of the most quoted essays in American legal writing. Holmes told the students that law was best understood from the standpoint of a "bad man" who cares only about what the courts will actually do to him, and he defined law itself as "the prophecies of what the courts will do in fact, and nothing more pretentious." Then he added a sentence that reads, more than a century later, like a charter for a field that did not yet exist: "For the rational study of the law the black-letter man may be the man of the present, but the man of the future is the man of statistics and the master of economics." Holmes did not do statistics, and he would probably have been impatient with most of what is done in his name. But his reframing mattered. If law is a prediction of what courts will do, then the behavior of courts is an object of study in its own right, separate from the doctrines courts announce. The opinions judges write are one kind of evidence about that behavior. Their patterns of decision, taken in aggregate, are another. Empirical legal studies is the enterprise of taking the second kind of evidence seriously. From realism to counting The first sustained attack on the idea that legal rules mechanically determine outcomes came from the legal realists of the 1920s and 1930s. Karl Llewellyn, Jerome Frank, Felix Cohen, and their colleagues argued that the formal rules stated in appellate opinions left judges far more room than the rules admitted, and that the real determinants of decisions lay in the facts of cases, the social background of judges, and the institutional settings in which they worked. Frank's Law and the Modern Mind (1930) went furthest, treating the trial judge's reaction to witnesses and parties as the heart of adjudication. The realists were strong on critique and weak on measurement. A few tried to be scientific in a literal sense. Underhill Moore at Yale spent years in the 1930s observing parking behavior on New Haven streets to test how legal rules actually shaped conduct, work that was ambitious in conception and modest in result. But the realists mostly produced essays, not datasets. Their lasting contribution was a question: if rules do not fully determine outcomes, what does? The people who first answered that question with numbers were political scientists. In 1948, C. Herman Pritchett published The Roosevelt Court, a study of the Supreme Court's divided decisions between 1937 and 1947. Pritchett did something that sounds trivial now and was novel then: he counted how often each pair of Justices agreed with one another in non-unanimous cases and showed that the Court divided into stable blocs. Justices were not scattering randomly across cases, as one might expect if each case turned on its own legal merits. They were lining up in consistent groups, and those groups corresponded to recognizable political commitments. Pritchett's work set off two decades of increasingly technical research. Glendon Schubert, in The Judicial Mind (1965), used scaling techniques borrowed from psychology to argue that the Justices' votes could be arranged along a small number of underlying attitudinal dimensions, chiefly one concerning civil liberties and one concerning economic regulation. Stuart Nagel, in a 1961 article in the American Political Science Review, compared state supreme court justices who were Democrats with those who were Republicans and found that Democrats were more likely to side with the defense in criminal cases, with administrative agencies in business regulation cases, and with injured parties in personal injury suits. The party label, in other words, predicted something about judicial behavior. By the 1990s this line of work had crystallized into what became known as the attitudinal model. Its canonical statement is Jeffrey Segal and Harold Spaeth's The Supreme Court and the Attitudinal Model (1993), revised as The Supreme Court and the Attitudinal Model Revisited (2002). Segal and Spaeth argued that Supreme Court Justices, uniquely freed from electoral accountability, from ambition for higher office, and from review by any higher court, decide cases largely on the basis of their sincere policy preferences applied to the facts of cases. Legal doctrine, on this account, supplies the vocabulary of justification rather than the reason for decision. It was a deliberately provocative thesis, and much of the empirical work of the following decades was organized around testing, refining, or refuting it. Law schools came to the enterprise later. Law professors had long written about courts, but their methods were interpretive: reading cases closely, reconstructing doctrine, criticizing reasoning. Through the 1990s, a growing number of legal scholars trained in economics and statistics began to publish quantitative work on litigation, sentencing, and judicial decisions. In 2004 Cornell Law School launched the Journal of Empirical Legal Studies, under the leadership of Theodore Eisenberg, and within a few years a Society for Empirical Legal Studies was holding an annual Conference on Empirical Legal Studies. By the 2010s most elite American law schools employed faculty with doctorates in economics, political science, sociology, or psychology, and quantitative work had become an ordinary part of legal scholarship rather than an exotic import. The field that resulted is interdisciplinary in a strong sense. Political scientists tend to ask about ideology and institutions; economists about incentives, causal identification, and disparities; psychologists about cognition and bias; legal scholars about doctrine and its effects. They do not always agree, and some of the most productive moments in the field have come when researchers from one tradition have challenged the assumptions of another. The logic of inference Before looking at any particular finding, it helps to understand the handful of ideas that determine whether a finding deserves belief. In a 2002 article in the University of Chicago Law Review titled "The Rules of Inference," Lee Epstein and Gary King surveyed empirical work published in law reviews and concluded that much of it violated basic standards familiar in the social sciences: authors drew conclusions from unrepresentative samples, failed to state how they would know if they were wrong, and confused description with causal explanation. The article provoked sharp replies from legal scholars who thought the critique too rigid, but it helped raise the bar, and its core concerns remain the right questions to ask of any study in this book. Four ideas do most of the work. The first is the distinction between description and causation. It is one thing to say that judges appointed by Republican presidents vote to uphold criminal convictions more often than judges appointed by Democrats. It is another thing to say that a judge's appointment by a Republican president causes that judge to vote to uphold. The descriptive claim may be true because Republican appointees happen to sit in circuits with different caseloads, or because both the appointment and the vote reflect some third thing, such as the judge's earlier career as a prosecutor. Much of the sophistication of the field lies in designs that allow causal claims to be made with some confidence. The second is selection. The cases that courts decide are not a random sample of the disputes that arise in the world. Most disputes never become lawsuits. Most lawsuits settle. Most criminal charges are resolved by guilty plea. Most losing parties do not appeal, and the Supreme Court hears only a tiny fraction of the cases in which review is sought. At every stage, parties decide whether to go forward based on their expectations about how courts will rule. This means that the cases that reach a decision are systematically unusual, and that patterns observed among decided cases can be artifacts of which cases were chosen for decision. As later chapters show, selection explains why plaintiffs seem to win about half of tried cases in many areas, why appellate reversal rates look the way they do, and why some apparent racial disparities in sentencing may be partly produced, or partly hidden, by decisions made before sentencing. The third is omitted variables. When a study compares two groups, say Black and white defendants, and finds a difference in outcomes after adjusting for measured factors such as offense and criminal history, the remaining gap may reflect discrimination, or it may reflect factors the researcher could not measure that differ between the groups. The honest response to this problem is not to dismiss every disparity study, but to ask which unmeasured factors could plausibly account for a gap of the observed size, and whether better data or better designs can rule them out. The fourth is measurement. Empirical claims about courts depend on turning complex legal events into variables: a decision becomes "liberal" or "conservative," a judge becomes a point on a scale, a sentence becomes a number of months. Each translation involves judgment. Whether a decision favoring a criminal defendant on Fourth Amendment grounds should be coded "liberal" seems straightforward. Whether a decision striking down a campaign finance restriction on First Amendment grounds is liberal or conservative is much less so. The coding rules adopted by the builders of datasets shape what every downstream study can find. Natural experiments and the judge as instrument The most persuasive empirical work on courts exploits features of the legal system that mimic random assignment. The reason is simple. If cases are assigned to judges by lot, then any systematic difference in outcomes between judges cannot be explained by differences in the cases they receive, because on average they receive the same kinds of cases. Differences in outcomes can then be attributed to the judges themselves. Many American courts assign cases randomly or nearly so. Federal district courts typically use a random draw among the judges of a division. Most federal courts of appeals assign judges to three-judge panels by procedures intended to be random. Many state trial courts, bail courts, and immigration courts rotate assignments in ways that are, for practical purposes, unrelated to the characteristics of individual cases. Researchers have used these features in two ways. The first is to compare judges directly. If random assignment holds, then the fact that one immigration judge grants asylum at several times the rate of a colleague in the same court is evidence that the identity of the judge matters to the outcome. This is the design behind the influential "Refugee Roulette" study discussed in Chapter 8. The second, more technically ambitious, uses the judge as what economists call an instrumental variable. Suppose a researcher wants to know whether incarceration increases or decreases a defendant's later criminal behavior. Comparing incarcerated and non-incarcerated defendants directly is useless, because judges incarcerate the defendants they think most dangerous. But if defendants are randomly assigned to judges who differ in harshness, then the harshness of the assigned judge shifts the probability of incarceration for reasons unrelated to the defendant's own characteristics. The researcher can use that variation to estimate the causal effect of incarceration itself. Jeffrey Kling applied this approach to federal sentence lengths in a 2006 article in the American Economic Review, and it has since become a standard tool, used to study the effects of pretrial detention, disability benefit decisions, bankruptcy protection, and foster care placement. The method depends on the fact, established empirically, that judges differ from one another in consistent and measurable ways. In that sense, the finding that judges are not interchangeable umpires has become the foundation of an entire research industry. Natural experiments have limits. Random assignment of cases to judges does not mean random assignment of judges to cases in any deeper sense; judges are chosen by presidents and governors through processes that are anything but random. A study showing that judges appointed by different presidents decide differently can establish that the judge matters, but it cannot by itself say which feature of the judge, whether ideology, professional background, age, race, sex, or temperament, is doing the work. And, as Chapter 5 explains, researchers have discovered that assignment in some courts is less random than the courts' own rules suggest. What the field is and is not Two misunderstandings about empirical legal studies are common enough to address directly. The first is that the field aims to show that law does not matter. Some of its early practitioners, and the attitudinal model in its strongest form, came close to that claim for the Supreme Court. But the weight of subsequent work points the other way. The most consistent finding across many studies is that ideological effects are real but bounded: largest in the small set of cases that are genuinely contested, smaller or absent in cases where the law is clear, moderated by collegial and institutional pressures, and variable across areas of law. Empirical researchers have also developed methods to detect the influence of legal rules directly, for example by showing that a change in doctrine alters the factors that predict decisions. The quantitative study of courts is as capable of demonstrating legal constraint as of demonstrating its absence. The second misunderstanding is that numbers speak for themselves. They never do. Every finding in this book rests on a chain of decisions about what to count, how to classify it, which cases to include, which factors to control for, and how to interpret the residual. Critics of any particular study can and should interrogate each link. The appropriate response to a surprising finding is neither acceptance nor dismissal but a set of questions: whether it has been replicated, whether it survives alternative specifications, whether the design could distinguish the proposed explanation from its rivals. Holmes's man of statistics, in other words, turned out to need the black-letter man after all. The best empirical work on courts is done by people who understand the law well enough to know what a variable can and cannot capture. A researcher who does not know the difference between a reversal on the merits and a remand for further proceedings, or who does not appreciate how the federal Sentencing Guidelines channel judicial discretion, will build models that measure the wrong things with great precision. The chapters that follow try to keep both perspectives in view: the statistical question of what the pattern is, and the legal question of what the pattern means. Chapter 2: Building the Evidence Every empirical claim about courts begins with an act of transcription. Somebody reads a case and records, in a form a computer can process, who the parties were, what legal issue was decided, who won, and how each judge voted. The act looks clerical. It is not. The choices made when a dataset is built determine which questions can be asked of it, which comparisons are possible, and in some cases which answers come out. Understanding the major datasets, and the judgments embedded in them, is the first step toward reading empirical work critically. The Supreme Court Database The single most influential dataset in the field is the Supreme Court Database, created by the political scientist Harold Spaeth, who began coding the Court's decisions in the 1960s and refined his work with support from the National Science Foundation over several decades. For much of its recent life the database was hosted at Washington University in St. Louis, under the direction of Spaeth together with Lee Epstein, Andrew Martin, Jeffrey Segal, Theodore Ruger, and Sara Benesh. It is now maintained at Pennsylvania State University. Its modern component covers every term from 1946 forward; a September 2026 release extends it through the October 2025 term. A separate "legacy" component, completed in 2021, reaches back to the Court's first decisions in 1791. For each case the database records a large set of variables, well over two hundred in the current documentation. They include the identity of the petitioner and respondent, the court whose decision was reviewed, the source of jurisdiction, the legal provisions at issue, the disposition, the vote, whether the Court declared a statute unconstitutional, whether it formally altered precedent, and which Justice wrote the majority opinion. The data are available in two forms: a case-centered version in which each row is a decision, and a Justice-centered version in which each row is one Justice's vote in one case, which allows researchers to study individual voting patterns. Two variables have carried an enormous share of the scholarly weight. The first, issue area, assigns each case to one of fourteen broad categories, such as criminal procedure, civil rights, First Amendment, economic activity, judicial power, and federalism. The second, decision direction, classifies each outcome as liberal or conservative according to rules set out in the codebook. In criminal procedure, a decision favoring the person accused or convicted is coded liberal. In civil rights and First Amendment cases, a decision favoring the claimant of the right is liberal. In economic activity cases, decisions favoring labor unions, consumers, injured persons, and the government against business are generally coded liberal, as are decisions favoring the underdog. In federalism cases, decisions favoring federal power over the states are coded liberal. These rules capture the mid-twentieth century alignment of American politics well, and for most cases they produce classifications that nearly everyone would accept. But the edges are instructive. A decision striking down a campaign finance restriction favors a First Amendment claimant and so might seem liberal under the general rule, yet most observers regard Citizens United v. FEC (2010) as a conservative decision. The database's documentation contains specific rules for such situations, and they reflect the coders' judgment about the political meaning of the outcome. Religious liberty claims, commercial speech, property rights, and gun rights raise similar puzzles. As political coalitions shift, as they have on free speech, where some of the most prominent claimants are now religious conservatives, a directional code fixed by older alignments can misdescribe what is at stake. Scholars have raised sharper concerns. In a 2009 article in the Hastings Law Journal, Carolyn Shapiro argued that the issue coding, which assigns most cases to a single primary issue, flattens the legal complexity of cases that turn on several questions at once, and that researchers relying on the codes can miss what cases are actually about. In a 2013 article in the Journal of Law, Economics, and Organization, Anna Harvey and Michael Woodruff raised a more troubling possibility: that coders, knowing how the Justices voted, might classify ambiguous outcomes in ways that make votes look more ideologically consistent than they are. If a coder knows that the Court's most liberal members joined an opinion, there is a natural temptation to code its outcome as liberal. Harvey and Woodruff reported evidence consistent with this kind of confirmation bias and cautioned that it could inflate estimates of attitudinal voting. The database's maintainers and other scholars have debated the magnitude of the problem, but the point stands as a caution: when the measure of the outcome and the measure of the judge's ideology both come from the same votes, circularity lurks. None of this diminishes the database's importance. Before it existed, a researcher who wanted to know how often the Court overruled its own precedents in a given decade would have had to read every case. The database made systematic comparison routine. Its limits are the limits of any coding scheme, and the database's own documentation is candid about them. The courts of appeals databases The Supreme Court decides fewer than seventy cases a year after full briefing and argument. The federal courts of appeals decide tens of thousands, and for most litigants they are the courts of last resort. Studying them presented a different challenge: the volume of cases made complete coding impractical. The solution adopted by Donald Songer and his collaborators was sampling. The United States Courts of Appeals Database, built with National Science Foundation support, drew random samples of published decisions from each circuit for each year, beginning with 1925 and eventually running through 1996; Ashlyn Kuersten and Susan Haire later extended it through 2002. The sample was smaller for the earlier decades and larger for later ones, and each sampled case was coded with the same kinds of variables Spaeth used for the Supreme Court: parties, issues, outcomes, directions, and individual judges' votes. A companion database compiled by Gary Zuk, Deborah Barrow, and Gerard Gryski recorded biographical attributes of the appeals judges themselves, including appointing president, prior career, race, sex, and education. Because the samples were drawn from published opinions, the database inherits a selection problem. In the modern era a large majority of federal appellate decisions are issued as unpublished, nonprecedential dispositions, and they differ systematically from published ones: they are more often affirmances, more often unanimous, and more often in cases such as prisoner appeals where the outcome was considered clear. A study based only on published decisions describes the contested frontier of appellate law, not the ordinary run of cases. That is appropriate for some questions and misleading for others. Much of the most prominent work on appellate panels, discussed in Chapter 5, relied instead on datasets that researchers assembled themselves for particular areas of law, such as environmental regulation, sex discrimination, affirmative action, or review of agency interpretations. These focused datasets can be coded more carefully and can include variables specific to the doctrine at issue, at the cost of generality. The Federal Judicial Center's Integrated Database For questions about litigation as a whole, including how cases end, how often plaintiffs win, and how long cases last, the principal federal source is the Integrated Database maintained by the Federal Judicial Center, the research and education agency of the federal courts. It is built from reports that court clerks file with the Administrative Office of the United States Courts when cases open and close, and it covers civil and criminal cases in the district courts from the early 1970s forward, as well as appeals and, more recently, bankruptcy matters. The Integrated Database is comprehensive in a way no hand-coded dataset can be: it includes essentially every federal case. For each civil case it records the nature of the suit (from a list of categories such as contract, personal injury, civil rights, and securities), the basis of jurisdiction, the procedural stage at which the case ended, the method of disposition, and, where there was a judgment, which side prevailed and in what amount. It has been the foundation for decades of research on settlement, trial rates, and win rates. Its weaknesses stem from the same source as its strengths. The codes are entered by clerks for administrative purposes, not by researchers for analysis, and errors creep in. In a 2003 article in the Notre Dame Law Review, Theodore Eisenberg and Margo Schlanger compared the database's coding of judgments with the underlying docket records for a sample of cases and found errors concentrated in particular fields, including the recording of which party won in some categories of cases. They concluded that the database was reliable for many purposes but that researchers needed to understand where its codes were weakest. The database also does not identify the judge assigned to each case, which makes it unusable for studying individual judges without laborious matching to docket records. The Sentencing Commission's offender files Research on federal criminal sentencing rests primarily on data collected by the United States Sentencing Commission. For every federal offender sentenced for a felony or serious misdemeanor, courts send the Commission a set of documents, including the judgment and the statement of reasons for the sentence, and the Commission codes them into a record for each case. The resulting files, released publicly each fiscal year, contain the offender's demographic characteristics, the offense of conviction, the guideline calculations (base offense level, adjustments, criminal history category, and resulting range), whether the sentence fell within, above, or below the range and why, and the sentence imposed. The Commission also offers an online tool, the Interactive Data Analyzer, that allows anyone to produce basic tabulations. These files are unusually rich. Because the guidelines themselves quantify the seriousness of the offense and the offender's criminal history, researchers can compare offenders who are similar on the dimensions the law says should matter, which makes disparity analysis far more precise than in systems without such structured inputs. The Commission uses the same data for its own periodic reports on demographic differences in sentencing, discussed in Chapter 6. The files have two notable limitations. First, they begin at sentencing. They contain nothing about the charges originally brought, the plea negotiations, or the cases that were declined or dismissed. A disparity that arises in charging will be baked into the offense of conviction and the guideline range, and a sentencing study that controls for those factors will treat it as a legitimate difference rather than a disparity. Second, the Commission does not release the identity of the sentencing judge. Researchers who have studied individual judges, such as Crystal Yang in her work on the effects of the Supreme Court's decision in United States v. Booker (2005), have had to match Commission records to court docket information themselves. To address the first limitation, the Bureau of Justice Statistics maintains the Federal Justice Statistics Program, which links records across agencies, from arrest by federal law enforcement through prosecution, adjudication, sentencing, and imprisonment. Linked data of this kind made possible the study by M. Marit Rehavi and Sonja Starr, discussed in Chapter 6, that traced racial disparity in federal sentences back to prosecutors' charging decisions. Beyond the federal core The federal system dominates empirical legal studies because its data are centralized, but most American litigation and nearly all criminal prosecution happens in state courts. State data are fragmented. Some states, such as Pennsylvania, Minnesota, and Virginia, maintain sentencing commissions that publish detailed data. Many local court systems publish little or nothing in usable form. The National Center for State Courts compiles aggregate caseload statistics through its Court Statistics Project, and the Bureau of Justice Statistics has periodically surveyed felony sentencing and civil trials in large counties. Nonprofit organizations such as Measures for Justice have assembled county-level criminal justice indicators from records requests. Researchers who have produced some of the most important findings on bail and jury selection did so by obtaining administrative data directly from particular counties, as Chapter 7 describes. A final category consists of the raw text of legal decisions and dockets. The federal courts' electronic filing system, PACER, contains nearly every document filed in federal litigation, but charges fees for access and was not designed for bulk research. The nonprofit Free Law Project's CourtListener and its RECAP archive have made a large and growing portion of that material freely searchable. Harvard Law School's Caselaw Access Project digitized the published case law of American courts from its library's print collection, and in 2024 made its bulk data freely available. These text collections are the raw material for the newer computational work on legal language and prediction discussed in the Conclusion. The sources differ in coverage, in the kinds of questions they answer, and in their characteristic weaknesses, as Table 1 summarizes. Table 1. Major data sources for the empirical study of American courts. Source Coverage Unit of observation Typical uses Key limitation Supreme Court Database Supreme Court, 1791 to present (modern file from 1946) Case, or Justice's vote in a case Ideological voting, precedent, issue trends Directional coding requires judgment on ambiguous cases U.S. Courts of Appeals Database Random samples of published circuit opinions, 1925–2002 Case and judge vote Panel voting, judge attributes Published opinions only; ends in 2002 FJC Integrated Database Nearly all federal district and appellate cases since the 1970s Case Win rates, settlement, trial rates, duration Clerical coding errors; no judge identity Sentencing Commission offender files All sentenced federal offenders, annual releases Offender Sentencing disparity, guideline compliance Starts at sentencing; judge identity withheld Federal Justice Statistics Program Linked federal records from arrest to prison Defendant across stages Charging and case-processing effects Complex linkage; less detail per stage The coder's fingerprints The lesson of this survey is not that the data are unreliable. It is that data about courts are built, not found, and that the builder's choices travel with the data into every study that uses it. A disparity study built on sentencing files will be structurally silent about charging. A study of ideological voting built on directional codes will inherit the coders' view of what counts as liberal. A study of win rates built on administrative codes will inherit clerical error. Good empirical work states these limitations openly and tests whether its conclusions survive alternative codings. The circularity problem, for instance, can be attacked by having ambiguous outcomes recoded by people who are not told how the Justices voted, or by testing findings against ideology measures that are not derived from votes at all. The published-opinion problem can be attacked by checking whether a pattern found in published appellate decisions also appears in unpublished ones. When a finding survives such tests, confidence in it rises. When it does not, the finding was about the dataset, not the courts. The chapters that follow report findings that, in most cases, have survived a good deal of such scrutiny, and note where they have not. Chapter 3: Measuring the Unmeasurable To test whether ideology shapes judicial decisions, a researcher needs two measurements: one of the decisions and one of the ideology. The previous chapter showed how decisions get coded. This chapter concerns the harder problem. Ideology is not observable. No judge wears a number. What researchers call a judge's ideology is always an inference from something else: who appointed the judge, what newspapers said about the nominee, how the judge has voted, whom the judge gave money to before joining the bench. Each source of inference has a characteristic strength and a characteristic blind spot, and many disagreements about whether judges are "political" turn out, on inspection, to be disagreements about measurement. The party of the appointing president The simplest measure, and still the most widely used in studies of the lower federal courts, is the party of the president who appointed the judge. A judge appointed by a Republican president is coded one way, a judge appointed by a Democrat the other. The measure has real virtues. It is known for every federal judge. It is fixed before the judge decides a single case, so it cannot be contaminated by the votes it is used to explain. And because presidents of both parties have, since at least the Reagan administration, devoted considerable effort to selecting judges whose views align with their own, the party of the appointing president has become a better proxy for ideology over time rather than a worse one. Its vices are equally clear. It is crude: it treats every Republican appointee as identical, although a judge appointed by President Eisenhower and one appointed by President Trump plainly differ. It ignores the way senators shape appointments to the lower courts. By long-standing custom, the senators from a nominee's home state have had substantial influence over district court nominations and some influence over appellate nominations, so a Democratic president filling a seat in a state with two Republican senators may end up appointing a judge more moderate than the president would prefer. And it is historically unstable. Before the realignment of Southern politics in the second half of the twentieth century, Democratic presidents appointed many conservative Southern judges, and a party-based measure misclassifies them. For all that, the appointing president's party remains a remarkably good predictor in contested areas of law, and its simplicity has an underappreciated advantage: studies using it are easy to understand and hard to manipulate. When a finding depends on a sophisticated ideology score but disappears when the simple party measure is used, a reader has reason to ask why. Scores from the confirmation press For Supreme Court Justices, a measure independent of their votes was developed by Jeffrey Segal and Albert Cover in a 1989 article in the American Political Science Review. They collected the editorials published about each nominee between nomination and confirmation in four major newspapers, two generally regarded as liberal and two as conservative, and coded each paragraph that characterized the nominee's views on civil liberties and civil rights as liberal, moderate, or conservative. From these they computed a score for each nominee. The scores correlated strongly with the Justices' subsequent votes in civil liberties cases; the correlation Segal and Cover reported was about 0.80, which in social science is very high. The Segal-Cover scores solved the circularity problem elegantly. Because they measure perceived ideology before a Justice casts any vote, they can be used to predict those votes without assuming the conclusion. Their limits lie elsewhere. They capture how elite editorial writers saw a nominee at one moment, which may reflect the nominee's public record, the political climate, or the strategies of the confirmation process as much as the nominee's real views. They predict civil liberties votes much better than votes in economic cases. They cannot register change over a career, and several Justices have changed a great deal. And they exist only for Supreme Court nominees; nobody writes editorials about most circuit judges. Ideal points from votes: the Martin-Quinn scores The most influential modern measure of Supreme Court ideology takes a different approach entirely. In a 2002 article in Political Analysis, Andrew Martin and Kevin Quinn introduced a method for estimating each Justice's ideological position, term by term, from the pattern of votes in non-unanimous cases. The resulting Martin-Quinn scores now cover every term from October 1937 onward and are updated as new terms conclude. The underlying idea can be explained without mathematics. Imagine that every Justice occupies a point on a single left-right line, which the method calls an ideal point, and that every case presents a choice between two outcomes that can also be located on that line. A Justice votes for whichever outcome lies closer to the Justice's own ideal point, subject to some randomness. Given only the record of who voted with whom across many cases, one can work backward to estimate where each Justice and each case must lie to make the observed votes most probable. Justices who frequently vote together end up close to one another; Justices who rarely do end up far apart. This family of techniques, known as item response models, is the same one used in educational testing to estimate students' abilities from their patterns of right and wrong answers, and in political science to estimate legislators' positions from roll-call votes. Martin and Quinn's contribution was to allow the ideal points to change over time in a disciplined way. Each Justice's position in one term is assumed to be close to the position in the previous term, but not identical, so the model can detect gradual drift. In a 2007 article in the Northwestern University Law Review, Lee Epstein, Andrew Martin, Kevin Quinn, and Jeffrey Segal used the scores to show that ideological drift is common rather than exceptional. Justices such as Harry Blackmun, John Paul Stevens, and David Souter moved leftward over their tenures in ways that are visible in the data; others moved in the opposite direction or stayed largely stable. The finding matters for presidents, who cannot count on a nominee's views staying fixed, and for theories of judging, which must account for change. The scores also make it possible to identify the median Justice, the member of the Court who sits at the ideological center and whose vote therefore decides cases that divide the Court along ideological lines. By the Martin-Quinn measure, Justice Anthony Kennedy occupied that position for most of the decade before his retirement in 2018. After his departure, the median shifted to Chief Justice Roberts for the next two terms, and after Justice Amy Coney Barrett replaced Justice Ruth Bader Ginsburg in 2020, it moved to Justice Brett Kavanaugh. These shifts correspond closely to changes in the outcomes of closely divided cases, which is itself a form of validation. The limitations of vote-based scores follow from their construction. First, because they are estimated from votes, using them to "explain" votes is circular. Martin-Quinn scores are a summary of how Justices vote, not an independent cause of it. They are appropriate for describing the Court's ideological structure and its changes; they are inappropriate as evidence that ideology, rather than something else, drives votes. Second, the model assumes a single dimension. When Justices divide along lines that do not map onto left and right, for example in some criminal procedure cases in which Justices Scalia and Gorsuch have sided with defendants on originalist or textualist grounds, the model treats those votes as noise. Third, comparisons across long spans of time depend on assumptions about how the Court's agenda has changed. A Justice who looks moderate in 1950 and one who looks moderate in 2020 were voting on very different sets of cases. Placing all judges on one scale Questions about the lower federal courts and about interactions among institutions require a measure that places circuit judges, Justices, presidents, and senators on a common scale. Two related efforts accomplished this. In 2001, Micheal Giles, Virginia Hettinger, and Todd Peppers proposed a measure for lower federal court judges built on the observation about senatorial courtesy made earlier. If a home-state senator belonged to the appointing president's party, the judge was assigned the ideology score of that senator (or the average of the two, if both qualified); otherwise, the judge was assigned the president's score. The scores themselves came from the NOMINATE family of measures that political scientists Keith Poole and Howard Rosenthal developed from congressional roll-call votes. The approach assumes that judges reflect the views of the politicians who controlled their selection, which is plausible on average and doubtful for any individual judge. In 2007, Lee Epstein, Andrew Martin, Jeffrey Segal, and Chad Westerland extended the idea to the Supreme Court in what they called the Judicial Common Space, transforming Martin-Quinn scores so that Justices could be placed on the same scale as the lower court judges, members of Congress, and presidents. The Judicial Common Space made it possible to ask, for example, whether a circuit panel's decisions respond to the position of the Supreme Court's median Justice, or whether the Court's decisions shift when the ideological distance between it and Congress changes. Following the money A newer approach draws on the political behavior of judges before they became judges. Adam Bonica, working with Michael Woodruff and later with Maya Sen, used the Database on Ideology, Money in Politics, and Elections, which compiles records of political contributions, to estimate the ideology of lawyers and judges based on the candidates and committees to which they donated before joining the bench. Because contributions are recorded for millions of people, these "CFscores" can place state and federal judges, and the lawyers who appear before them, on the same scale as politicians. Bonica and Sen's work, including their book The Judicial Tug of War (2020), documented that the American legal profession as a whole leans to the left of the general public, that elite lawyers lean further, and that the federal judiciary is substantially more conservative than the bar from which it is drawn, a gap they attribute to deliberate efforts by Republican presidents and allied organizations to select conservative judges. Contribution-based measures have the advantage of being fixed before judicial service and of covering many judges for whom no other measure exists. Their weakness is that many judges never made political contributions, and those who did may differ systematically from those who did not. A measure that exists only for politically active lawyers risks describing a skewed subset of the bench. Choosing and comparing measures Table 2 sets the leading measures side by side. The most important column is the last, because the choice of measure should depend on the question being asked. A researcher who wants to describe how the Supreme Court's center has moved over time should use Martin-Quinn scores. A researcher who wants to test whether ideology predicts votes needs a measure, such as Segal-Cover scores or the appointing president's party, that was fixed before the votes were cast. Table 2. Leading measures of judicial ideology. Measure Basis Coverage Main strength Main weakness Party of appointing president Who nominated the judge All federal judges Simple, pre-dates votes Crude; ignores senators and era Segal-Cover scores Newspaper editorials during confirmation Supreme Court nominees since 1937 Independent of votes Snapshot; weak on economic issues Martin-Quinn scores Justices' votes in divided cases Supreme Court, 1937 onward, term by term Tracks change over time Circular if used to explain votes Giles-Hettinger-Peppers / Judicial Common Space Scores of appointing politicians, rescaled Federal judges and Justices together Common scale across courts and branches Assumes judge mirrors selectors Campaign-finance (CFscores) Donations before appointment Judges and lawyers who donated Pre-dates service; very broad reach Missing for non-donors In a 2009 article in the Washington University Journal of Law and Policy, Joshua Fischman and David Law surveyed these measures and pressed a more fundamental question: what exactly is being measured? "Ideology" can mean a set of policy preferences, a jurisprudential philosophy such as originalism or pragmatism, a disposition toward particular kinds of litigants, or a partisan identity. The measures capture different blends of these. A judge who reliably votes to restrict federal regulatory power may be expressing a policy preference for smaller government, a sincere theory of constitutional structure, or both at once, and no voting pattern alone can distinguish them. Daniel Ho and Kevin Quinn made a related point in a 2010 California Law Review article whose title, "How Not to Lie with Judicial Votes," signals its purpose. They examined common ways of summarizing judicial votes, including raw agreement rates and simple percentages of liberal votes, and showed how they can mislead when the mix of cases changes. A Justice's percentage of liberal votes can rise in a term simply because the Court took more cases in which the conservative position was extreme, without any change in the Justice's views. Scaling models such as Martin-Quinn's are designed to account for such agenda effects, but they introduce their own assumptions. The practical lesson for a reader of empirical work is to ask three questions of any claim about ideology. Where did the measure come from, and could it have been influenced by the outcome being explained? Does the finding hold across different measures, or only one? And does the measure capture the kind of ideology the argument is about? A finding that survives all three questions is strong evidence. A finding that depends on a single measure, especially a vote-based one used to explain votes, is much weaker than its statistical precision might suggest. With these tools in hand, the next two chapters turn to what the measures reveal: first on the Supreme Court, and then on the federal courts of appeals. Chapter 4: Ideology at the Top The Supreme Court is where the empirical study of judging began and where its central claim, that judges' votes reflect their policy preferences, finds its strongest support. It is also the court least representative of judging in general. Understanding both facts at once is the key to reading the evidence well. Why the Court is a special case Three features make the Supreme Court an unusually favorable environment for ideological voting. The first is freedom from external constraint. Justices serve for life, no court can reverse them, and few harbor ambitions for any higher office. Segal and Spaeth built the attitudinal model on precisely this foundation: if any judges are free to vote their preferences, these are. The second is control of the agenda. The Court decides which cases to hear, granting review in only a tiny fraction of the several thousand petitions it receives each year. Its own rules direct attention to cases in which lower courts are divided or an important federal question is unsettled. The result is a docket composed disproportionately of cases in which the legal materials, the text, the history, and the precedents, do not point clearly in one direction. If the law were clear, the case would usually not be there. The third is the selection effect that follows from agenda control. This point, borrowed from the economics of litigation discussed in Chapter 9, is easy to overlook. Because the Court chooses hard cases, the fraction of its decisions in which ideology appears to matter overstates the fraction of legal questions in which ideology matters. A study of the Court's docket is a study of the legal system's most contested edge. Evidence of ideological voting there is evidence about that edge, not about law in general. With those caveats, the evidence is formidable. The attitudinal evidence The most basic evidence is structural. Since Pritchett's work in the 1940s, researchers have found that the Justices' votes in divided cases arrange themselves along a largely one-dimensional spectrum. Scaling methods, from Schubert's to Martin and Quinn's, consistently find that a single left-right dimension accounts for most of the variation in how Justices vote. Justices at the ends of the spectrum rarely join each other in divided cases; Justices near the middle join both sides. The second kind of evidence is predictive. Segal and Cover's editorial-based scores, measured before any Justice cast a vote, correlate at about 0.80 with later votes in civil liberties cases. A nominee's perceived ideology at confirmation is, in other words, a strong predictor of how that Justice will decide the Court's most contested rights questions. The third kind of evidence concerns the relationship between the Justices' votes and the party of the president who appointed them. For most of the twentieth century, that relationship was imperfect. Republican presidents appointed Justices such as Earl Warren, William Brennan, Harry Blackmun, John Paul Stevens, and David Souter, all of whom came to be identified with the Court's liberal wing. Democratic presidents appointed Byron White, who voted with conservatives on many criminal justice and abortion questions. In their 2019 book The Company They Keep, Neal Devins and Lawrence Baum documented how this changed. Since the retirement of Justice Stevens in 2010, for the first time in the Court's modern history, every Justice appointed by a Republican president has been more conservative than every Justice appointed by a Democrat. Devins and Baum attribute the shift to changes in the elite social and professional networks in which Justices move, which have themselves become polarized along partisan lines, as well as to more ideologically focused selection by presidents. The Court's ideological blocs and its partisan blocs now coincide. The consequences are visible in the decisions that define the current era. In Dobbs v. Jackson Women's Health Organization (2022), overruling Roe v. Wade; in Students for Fair Admissions v. Harvard (2023), holding race-conscious university admissions unlawful; in Loper Bright Enterprises v. Raimondo (2024), overruling the Chevron doctrine of deference to agency interpretations of statutes; and in Trump v. United States (2024), recognizing presidential immunity from criminal prosecution for official acts, the Court divided six to three along the lines of the appointing presidents' parties. Each decision was defended in the language of text, history, and precedent, and each was criticized in the same language. The empirical point is not that one side's legal reasoning was wrong. It is that in the cases with the highest political stakes, the Justices' legal conclusions have aligned with their partisan origins with near-perfect regularity. A subtler piece of evidence comes from studies that look for ideological consistency within an area of law where the doctrinal principle is supposed to be neutral. In a 2018 article in the Journal of Law and Courts, Lee Epstein, Christopher Parker, and Jeffrey Segal examined the Justices' votes in free speech cases from 1953 onward and asked whether a Justice's support for the free speech claim depended on the ideological character of the speaker. They found that it did: Justices were more likely to support free speech claims brought by speakers whose views aligned with their own, a pattern the authors described as in-group bias. The First Amendment's commitment to viewpoint neutrality, on this evidence, is applied less neutrally than its doctrine requires. Against the pure attitudinal model For all its force, the evidence has never supported the strongest version of the attitudinal claim, that Justices simply vote their preferences and law is window dressing. Several bodies of work show why. Unanimity. A large share of the Court's argued cases are decided without dissent. In recent terms the proportion has typically fallen somewhere in the range of roughly two-fifths to one-half, and in some terms it has been higher. These unanimous decisions often involve questions of statutory interpretation, procedure, and jurisdiction on which the legal materials, once the Court has studied them, point clearly enough to persuade Justices of every stripe. A pure attitudinal model has no easy account of why Justices with sharply different policy views agree so often, other than that many cases carry no ideological charge, which is itself a concession that something other than preference is doing the work. Strategic behavior. In The Choices Justices Make (1998), Lee Epstein and Jack Knight drew on the private papers of Justices William Brennan, Thurgood Marshall, and Lewis Powell to show that Justices do not simply announce their preferred outcomes. They bargain over the content of opinions, trade suggestions and threats to write separately, and moderate their positions to hold majorities together. Forrest Maltzman, James Spriggs, and Paul Wahlbeck, in Crafting Law on the Supreme Court: The Collegial Game (2000), documented similar dynamics in the circulation of draft opinions. This strategic account does not deny that Justices have policy goals. It denies that the goals translate directly into votes without passing through an institutional process that shapes the result. The practical consequence is that the content of majority opinions, which is what binds lower courts, is often more moderate than the preferences of the majority's median member would suggest. Jurisprudential regimes. A more direct test of legal influence was offered by Mark Richards and Herbert Kritzer in a 2002 article in the American Political Science Review. They argued that if law matters, then major precedents should change the factors that predict the Justices' votes in later cases. Examining free expression cases, they identified a set of decisions in 1972, notably Grayned v. City of Rockford and Police Department of Chicago v. Mosley, that established the modern framework distinguishing content-based from content-neutral regulations of speech. After those decisions, they found, the content-neutrality of a regulation became a significant predictor of the Justices' votes in a way it had not been before, while the Justices' ideology remained important as well. The finding suggests that precedent structures the Court's decisions by defining what counts as relevant, even when it does not dictate outcomes. Later studies applied the same approach to other areas of law, with varying results. The weight of precedent. Harold Spaeth and Jeffrey Segal, the architects of the attitudinal model, tested the influence of precedent directly in Majority Rule or Minority Will (1999). They asked what happened when a Justice who had dissented from a landmark decision later confronted a case that applied it. If precedent constrains, the dissenter should now accept the landmark and apply it. They found that such Justices overwhelmingly continued to vote in line with their original dissent rather than yielding to the precedent, a result they read as strong confirmation that preferences dominate. Critics replied that the test was too narrow, because it examined only Justices who had expressed a strong disagreement and only the cases in which that disagreement was likely to resurface. Thomas Hansford and James Spriggs, in The Politics of Precedent on the U.S. Supreme Court (2006), offered a more nuanced view: Justices use precedent strategically, interpreting favorable precedents expansively and unfavorable ones narrowly, but the existing body of precedent shapes the arguments available and therefore the outcomes that can be justified. Constrained preferences. Michael Bailey and Forrest Maltzman, in The Constrained Court (2011), used a sophisticated scaling approach to estimate how much weight individual Justices gave to legal values, such as adherence to precedent, deference to Congress, and protection of speech, apart from their policy preferences. They found that the weight varied considerably among Justices and that legal values explained a meaningful share of votes that pure policy preferences could not. Their conclusion was that both law and ideology matter, and that the mix differs from one Justice to another. What the Justices see A different line of work asks not whether Justices vote their preferences but whether their preferences shape how they perceive the facts. In Scott v. Harris (2007), the Court held that a police officer did not use excessive force when he ended a high-speed chase by ramming the fleeing motorist's car, rendering the driver a quadriplegic. The majority opinion took the unusual step of relying on a video of the chase, which it said showed that the driver posed a grave danger to the public and that no reasonable jury could conclude otherwise. Justice Stevens, dissenting, saw the same video differently. In a 2009 article in the Harvard Law Review, Dan Kahan, David Hoffman, and Donald Braman showed the video to a diverse national sample of about 1,350 Americans. Most agreed with the Court that the driver posed a serious danger, but substantial minorities disagreed, and the disagreements were not random. They tracked cultural and political identity, with African Americans, low-income respondents, liberals, and Democrats more likely to see the chase as less dangerous and the police use of force as less justified. The authors' point was not that the majority was wrong about the video but that its confidence that no reasonable person could see it differently was itself a product of a particular perspective. The study suggests that ideological influence on judging may operate as much through perception of facts as through choice of rules, and that such influence may be invisible to the judges experiencing it. The emergency docket and the limits of the data The newest challenge to empirical study of the Court arises from the growth of what William Baude, in a 2015 article, called the "shadow docket": orders on emergency applications, stays, and injunctions issued without full briefing or argument and often without explanation. In recent years such orders have decided matters of great consequence, from the enforcement of state election laws to the implementation of federal immigration and personnel policies. Stephen Vladeck's book The Shadow Docket (2023) documented how the volume and significance of these rulings grew from the late 2010s onward, and applications by the federal government rose sharply again in 2025. For empirical researchers, the emergency docket poses a measurement problem as much as a legal one. Many orders do not record how each Justice voted; a Justice's disagreement is visible only if that Justice chooses to note it. The reasons for the order are often unstated, so there is nothing to code for legal rationale. Datasets built on argued cases systematically omit these decisions, which means that measures of the Court's behavior based on the merits docket may increasingly miss where much of its power is exercised. Researchers have begun building separate datasets of emergency orders, but the gaps in the underlying record limit what can be learned. Reading the evidence What, then, does the empirical record establish about the Supreme Court? It establishes that in the Court's divided cases, which are its most consequential, ideology is the strongest single predictor of how Justices vote, and that since 2010 ideology and the party of the appointing president have coincided. It establishes that Justices change over their careers, sometimes substantially. It establishes that perceptions of facts, not only choices of rules, can vary along ideological lines. It does not establish that law is irrelevant. Unanimous decisions, strategic moderation, shifts in the factors that predict votes after major precedents, and variation in the weight Justices give to legal values all point to a Court whose members are constrained, in different degrees, by the materials and institutions of law. The most accurate description is that the Court's docket is a filter that removes most easy cases, leaving a residue in which law runs out often enough that the Justices' deeper commitments become decisive. Whether one calls those commitments "ideology" or "judicial philosophy" is partly a matter of taste. A Justice who reads the Constitution as its original public meaning and one who reads it in light of evolving principles will reach systematically different results, and each will regard the other's method as the ideological one. The data cannot adjudicate between methods. What they can do is make visible the regularity with which method and outcome align, and invite the question whether that alignment is coincidence. The following chapter turns to courts where that question can be answered with more precision, because the composition of the bench, unlike the Supreme Court's, changes from case to case. Chapter 5: Panels, Colleagues, and Circuits The federal courts of appeals offer something the Supreme Court cannot: variation in who decides. The Supreme Court hears every case with the same nine members. A court of appeals hears most cases in panels of three, drawn from a pool that may include more than a dozen active judges and several senior ones. Because panels are, in principle, assigned without regard to the cases they will decide, the same kinds of cases are heard by panels of every possible composition: three Republican appointees, three Democratic appointees, or mixtures of two and one. That variation turns the courts of appeals into a vast natural experiment in how judges' backgrounds and colleagues shape their decisions. The results have been among the most striking and most carefully replicated in the field. Ideology on the circuits The first question is whether individual appellate judges vote differently depending on the party of the president who appointed them. The answer, across many studies and many areas of law, is yes, but by smaller margins than on the Supreme Court and with substantial variation across areas. The most comprehensive early evidence came from a project led by Cass Sunstein, David Schkade, and Lisa Ellman, published first as a 2004 article in the Virginia Law Review and then expanded, with Andres Sawicki, into the book Are Judges Political? An Empirical Analysis of the Federal Judiciary (2006). The researchers assembled thousands of published appellate decisions across more than a dozen areas of law chosen because they were politically salient, including affirmative action, sex discrimination, disability discrimination, environmental regulation, campaign finance, abortion, and capital punishment. In most of those areas, Democratic appointees cast liberal votes noticeably more often than Republican appointees. In a few areas, however, including criminal appeals and some questions of constitutional structure, the differences between the two groups were small or absent. Ideological voting was a feature of particular kinds of disputes, not a universal law of appellate behavior. This pattern fits a broader finding reported by Lee Epstein, William Landes, and Richard Posner in The Behavior of Federal Judges (2013): the influence of ideology increases as one moves up the judicial hierarchy. It is weakest in the district courts, where judges handle an enormous volume of routine matters and are subject to reversal; intermediate in the courts of appeals; and strongest in the Supreme Court. The explanation is consistent with the selection logic of the previous chapter. Each level up the hierarchy filters out more of the easy cases, leaving a higher proportion of genuinely open questions. Panel effects The more surprising discovery concerned not the judges themselves but their colleagues. If judges simply voted their preferences, a Republican appointee's vote should be the same whether that judge sits with two other Republican appointees or with two Democratic appointees. It is not. The pioneering study was Richard Revesz's 1997 article in the Virginia Law Review, "Environmental Regulation, Ideology, and the D.C. Circuit." Revesz examined the D.C. Circuit's review of environmental agency decisions over roughly a quarter century and found not only that judges' party affiliations predicted their votes, but that a judge's vote depended heavily on the party affiliations of the other judges on the panel. In some specifications, the composition of the panel mattered more than the judge's own affiliation. Sunstein and his colleagues found the same thing across their broader dataset and gave the phenomena names. Ideological amplification describes the tendency of judges on panels composed entirely of appointees of one party to vote more ideologically than they would on mixed panels. A Republican appointee sitting with two other Republican appointees votes in a more conservative direction, on average, than a Republican appointee sitting with one Democrat. Ideological dampening describes the converse: a judge in the minority on a panel moves toward the majority's position. The combined effect is that the outcome of a case depends substantially on the composition of the panel. In several areas the difference between the rate of liberal outcomes on all-Democratic panels and on all-Republican panels was much larger than the difference between individual Democratic and Republican appointees, because amplification pushed unified panels to the extremes. There were revealing exceptions. In abortion and capital punishment cases, Sunstein's team found strong ideological voting but little or no panel effect. Judges' views on those questions appeared to be so firmly held that colleagues could not move them. The exception strengthens the interpretation of panel effects elsewhere: where views are less fixed, collegial interaction shapes outcomes. Why colleagues matter Several mechanisms could produce panel effects, and they carry different implications for how we should evaluate them. The whistleblower. In a 1998 article in the Yale Law Journal, Frank Cross and Emerson Tiller offered an explanation rooted in legal doctrine. They studied D.C. Circuit review of agency statutory interpretations under the Chevron doctrine, which then required courts to defer to reasonable agency readings of ambiguous statutes. They found that panels whose members all came from one party were considerably less likely to follow the doctrine when it pointed toward a politically unwelcome result than panels that included a member of the other party. A Republican-majority panel with a Democratic member deferred to liberal agency interpretations much more often than a panel of three Republicans did. Cross and Tiller argued that the minority judge functions as a potential "whistleblower": by threatening to write a dissent exposing the majority's departure from doctrine, which could attract en banc or Supreme Court review, the minority judge induces the majority to follow the law. On this account, panel effects are evidence that doctrine constrains judges, but only when someone is present to enforce it. Deliberation and persuasion. A more benign reading is that judges actually persuade one another. Panels confer, exchange memoranda, and circulate drafts. A judge who hears a colleague's argument may genuinely change position. If so, mixed panels produce better-reasoned decisions because they are exposed to a wider range of perspectives. Dissent aversion. A third explanation emphasizes cost. Writing a dissent takes time, may irritate colleagues with whom a judge will sit for decades, and rarely changes the outcome. In a 2011 article in the Journal of Legal Analysis, Epstein, Landes, and Posner documented that dissent rates in the courts of appeals are low and that judges appear to dissent less as their workloads rise. A judge who disagrees with two colleagues may go along rather than pay the cost of dissenting. On this view, dampening reflects not persuasion but a reluctance to break consensus. Joshua Fischman, in a 2015 article in the Journal of Law, Economics, and Organization, developed a formal model of panel voting and concluded that the observed patterns are consistent with judges placing substantial weight on reaching unanimous decisions, a norm of consensus that shapes votes on the margin. The mechanisms are not mutually exclusive, and the data cannot always distinguish them. But the distinction matters normatively. If panel effects reflect whistleblowing or persuasion, mixed panels are a safeguard of the rule of law, and the practice of drawing panels randomly from ideologically diverse courts is an institutional virtue. If they reflect only dissent aversion, then the outcome of a case turns on the arbitrary composition of the panel, and the judge who goes along is voting against his or her own best judgment of the law. Sex, race, and the panel The panel structure has also allowed researchers to ask whether judges' personal characteristics, beyond ideology, affect decisions, and whether those characteristics influence colleagues. In a 2010 article in the American Journal of Political Science, "Untangling the Causal Effects of Sex on Judging," Christina Boyd, Lee Epstein, and Andrew Martin examined appellate decisions across thirteen areas of law. Using matching methods designed to compare male and female judges who were otherwise similar in ideology, background, and circuit, they found a consistent sex effect in only one area: sex discrimination cases. There, female judges were roughly ten percentage points more likely than comparable male judges to decide in favor of the party alleging discrimination. Moreover, male judges were significantly more likely to vote for the plaintiff when a woman sat on the panel. In the other twelve areas, the researchers found no reliable difference between male and female judges. The finding is precise in a way that matters: it does not show that women judge differently in general, but that in cases where their distinctive experience is directly relevant, they do, and their presence affects their colleagues. Sean Farhang and Gregory Wawro had reported related panel effects in employment discrimination cases in a 2004 article. Race has produced similar findings. In a 2008 article in the Columbia Law Review, Adam Cox and Thomas Miles studied appellate decisions under Section 2 of the Voting Rights Act and found that Black judges were substantially more likely than white judges to find liability, and that a white judge sitting with a Black colleague was significantly more likely to do so than a white judge on an all-white panel. Jonathan Kastellec, in a 2013 article in the American Journal of Political Science, examined affirmative action cases and found that Black judges were considerably more likely to vote to uphold affirmative action programs than non-Black judges, and that adding a Black judge to a panel substantially increased the probability that the non-Black judges on it would do the same. These studies bear directly on debates about judicial diversity. They suggest that diversity on the bench can matter to outcomes in a narrow but important set of cases, not because judges of different backgrounds apply different law generally, but because they bring perspectives relevant to specific questions and because those perspectives are shared in deliberation. They also raise hard questions. If the presence of a Black judge changes the outcome in a voting rights case, was the outcome on the all-white panel wrong, or the outcome on the mixed panel? The data cannot say. They can only establish that the outcome depends on who is in the room. Is assignment really random? All these designs depend on the premise that panel composition is unrelated to case characteristics. If a chief judge or clerk steered particular cases to particular panels, apparent panel effects could reflect case selection rather than judicial behavior. The premise has been tested. In a 2015 article in the Cornell Law Review, Adam Chilton and Marin Levy analyzed panel assignments across the federal courts of appeals over several years and compared the actual ideological composition of panels with what random assignment would have produced. In most circuits the actual assignments were consistent with randomness. In several, however, the distribution departed from what random assignment would predict in statistically significant ways. The authors emphasized that nonrandom assignment need not reflect manipulation: circuits use different procedures to accommodate scheduling, visiting judges, senior judges' preferences, and the practice of sending related appeals to the same panel. But their findings caution researchers to verify assignment procedures rather than assume them, and they raised institutional questions about transparency that circuits have been slow to answer. The concern about assignment has grown more pointed at the district court level. In several federal judicial divisions, only one judge sits, so a plaintiff who files there knows in advance who will hear the case. Plaintiffs seeking nationwide relief against federal policies have exploited this, filing in single-judge divisions whose judges they expected to be sympathetic. In March 2024 the Judicial Conference of the United States issued guidance encouraging district courts to assign cases seeking statewide or nationwide injunctive or declaratory relief randomly across the district, not only within the division of filing, though adoption was left to individual courts and some declined. The Supreme Court's June 2025 decision in Trump v. CASA, Inc., which sharply limited the authority of federal courts to issue "universal" injunctions protecting parties not before the court, reduced one of the incentives for such forum selection, though other routes to broad relief, such as class actions and suits under the Administrative Procedure Act, remain the subject of litigation. The episode illustrates how an empirical fact, that the identity of the judge predicts outcomes, becomes a strategic resource and then a problem of institutional design. Deference as a test case The Chevron doctrine provides a final and instructive window into the relationship between legal rules and ideology. From 1984 until 2024, Chevron U.S.A. v. Natural Resources Defense Council instructed courts to defer to reasonable agency interpretations of ambiguous statutes. Its defenders argued that it constrained judges by removing policy choices from them; its critics argued that it abdicated the judicial duty to say what the law is. Empirical work spoke to both claims. In a 2006 article in the University of Chicago Law Review, Thomas Miles and Cass Sunstein examined Supreme Court and circuit decisions applying Chevron and found that judges' willingness to uphold agency interpretations varied with their own ideology and with the political direction of the agency's decision: conservative judges were more likely to uphold conservative agency interpretations, and liberal judges liberal ones. Chevron, on this evidence, did not eliminate ideological voting. In a 2017 article in the Michigan Law Review, Kent Barnett and Christopher Walker assembled a large dataset of circuit court decisions reviewing agency statutory interpretations over roughly a decade and found that agencies won substantially more often when courts applied the Chevron framework than when they did not, a gap of more than thirty percentage points in their data. They also found evidence that applying Chevron reduced, though it did not eliminate, the influence of judges' ideology on outcomes. When the Supreme Court overruled Chevron in Loper Bright Enterprises v. Raimondo (2024), the empirical record was part of the argument on both sides: evidence that Chevron moderated ideological voting supported those who defended it as a constraint, while evidence that ideology persisted under it supported those who saw it as ineffective as well as misguided. Whether the end of Chevron will increase ideological divergence in the courts of appeals, as the Barnett and Walker findings would suggest, is a question that the data will be able to answer only after several years of post-Loper Bright decisions accumulate. What the circuits teach The evidence from the courts of appeals supports a more textured account of judicial behavior than either the umpire metaphor or the attitudinal model. Ideology matters, but in specific areas and to a limited degree. Colleagues matter, sometimes as much as a judge's own leanings. Personal experience matters in cases where it is directly relevant. Legal doctrine matters, especially when someone on the panel is positioned to insist on it. And the institutional rules that determine who sits on which case, rules that most lawyers and citizens never think about, turn out to shape outcomes in measurable ways. Chapter 6: Race and the Federal Sentence No subject in empirical legal studies carries higher stakes than racial disparity in criminal punishment, and none better illustrates how the answer to an empirical question depends on precisely what is being asked. That Black Americans are incarcerated at far higher rates than white Americans is not in dispute. The contested questions are narrower and harder: whether, and at which stages of the criminal process, people who are similarly situated in legally relevant ways are treated differently because of race, and how much of the overall disparity such differential treatment explains. The federal system, with its detailed sentencing data and its long history of reform aimed at reducing unwarranted disparity, is where these questions have been studied most intensively. A system built to measure disparity Before 1984, federal judges had nearly unlimited discretion to impose any sentence within the broad statutory range for an offense, and parole boards later decided how much of that sentence would actually be served. Judge Marvin Frankel's 1973 book Criminal Sentences: Law Without Order argued that this system produced lawless variation, with sentences depending on the identity of the judge rather than the nature of the crime. Frankel's critique, shared by liberals concerned with discrimination and conservatives concerned with leniency, led to the Sentencing Reform Act of 1984. The Act abolished federal parole, created the United States Sentencing Commission, and directed it to write guidelines that would reduce "unwarranted sentencing disparities among defendants with similar records who have been found guilty of similar conduct." The first guidelines took effect in November 1987, and the Supreme Court upheld the Commission's constitutionality in Mistretta v. United States (1989). The guidelines work by converting the offense and the offender into numbers. The offense receives a base level, adjusted up or down for specific characteristics such as drug quantity, loss amount, weapon use, role in the offense, and acceptance of responsibility. The offender's prior record places him or her in one of six criminal history categories. The intersection of the final offense level and the criminal history category yields a range of months. For nearly two decades, judges were required to sentence within the range unless a recognized ground for departure applied. In United States v. Booker (2005), the Supreme Court held that mandatory guidelines violated the Sixth Amendment jury trial right, because they required judges to increase sentences based on facts not found by a jury. The Court's remedy was to make the guidelines advisory. Judges must still calculate the range and consider it, but they may vary from it based on the broader sentencing purposes set out in the statute. In Gall v. United States and Kimbrough v. United States, both decided in 2007, the Court confirmed that judges have substantial latitude to impose sentences outside the range, including, in Kimbrough, on the basis of disagreement with the policy judgments embodied in the guidelines themselves. Mandatory minimum sentences set by Congress operate alongside the guidelines and are not affected by Booker. For drug offenses, the most important were those enacted in the Anti-Drug Abuse Act of 1986, which keyed five- and ten-year minimums to drug quantities and treated crack cocaine one hundred times more harshly than powder cocaine by weight. This history matters for empirical work because the three regimes, before the guidelines, under mandatory guidelines, and under advisory guidelines, offer natural comparisons. If judicial discretion produces racial disparity, disparity should have fallen when the guidelines constrained discretion and risen when Booker restored it. The actual evidence is more complicated. Did the guidelines reduce disparity between judges? The first question researchers asked was whether the guidelines achieved their basic goal of reducing variation among judges. In a 1999 article in the Journal of Law and Economics, James Anderson, Jeffrey Kling, and Kate Stith used the random assignment of cases to judges within districts to estimate how much sentences depended on the identity of the judge before and after the guidelines took effect. They found that the expected difference in sentence length between two typical judges fell after the guidelines were introduced. The reduction was real but moderate. The guidelines narrowed differences among judges without eliminating them, partly because other actors, especially prosecutors, gained discretion as judges lost it. That last point would become central to the debate. Kate Stith and José Cabranes, in Fear of Judging (1998), argued that the guidelines had not eliminated discretion so much as transferred it from judges, who exercise it in public and must give reasons, to prosecutors, who exercise it privately through charging and plea bargaining. A prosecutor who chooses to charge a drug quantity triggering a mandatory minimum, or to file a notice of a prior conviction doubling it, determines the sentence as effectively as any judge. Measuring racial disparity at sentencing The earliest large-scale studies of race under the guidelines found persistent disparities. In a 2001 article in the Journal of Law and Economics, David Mustard examined federal sentences from the early 1990s and found that, after controlling for the offense level, criminal history, and other factors, Black and Hispanic defendants received longer sentences than white defendants. He found that a large portion of the gap arose through departures from the guideline range, the one point in the mandatory system where judges retained substantial discretion. Black defendants were less likely than white defendants to receive downward departures and, when they did, received smaller ones. Sociologists and criminologists studying state and federal courts reached compatible conclusions through a different theoretical lens. Darrell Steffensmeier, Jeffery Ulmer, and John Kramer, in a 1998 article in Criminology based on Pennsylvania sentencing data, found that race interacted with age and sex: young Black men received the harshest sentences, more than would be predicted by race, age, or sex separately. They interpreted the findings through a "focal concerns" framework, in which judges making quick decisions under uncertainty rely on perceptions of blameworthiness, dangerousness, and practical constraints, perceptions that may be shaped by racial stereotypes. The Sentencing Commission itself began publishing systematic analyses of demographic differences after Booker, using multivariate regression to compare offenders similar in their guideline range, offense type, criminal history, and other legally relevant factors. Its reports in 2010, 2012, and 2017 found that Black male offenders received sentences substantially longer than similarly situated white male offenders, and that the gap had grown in the years after Booker. The 2017 report, covering fiscal years 2012 through 2016, estimated the gap at 19.1 percent. In November 2023 the Commission released a new analysis covering fiscal years 2017 through 2021, using refined methods and additional data. It found that Black men received sentences 13.4 percent longer, and Hispanic men 11.2 percent longer, than similarly situated white men. It also identified the decision whether to impose any prison term at all as a principal driver of the difference: Black men were 23.4 percent less likely, and Hispanic men 26.6 percent less likely, than comparable white men to receive a probationary sentence. The Commission has been careful to describe its findings as differences that remain after accounting for the factors its data measure, not as proof that the differences are caused by race. Tracing disparity upstream The Commission's analyses, like most sentencing studies, compare offenders who are similar at the point of sentencing. That design has a fundamental limitation: it takes the offense of conviction and the guideline range as given. But those inputs are themselves the product of earlier decisions by police and prosecutors. If prosecutors charge Black defendants more harshly than white defendants for the same underlying conduct, the harsher charges will produce higher guideline ranges, and a sentencing study that controls for the guideline range will treat the resulting difference in sentences as legally warranted. The most important study to address this problem was published by M. Marit Rehavi and Sonja Starr in the Journal of Political Economy in 2014. Using linked federal data that followed cases from arrest through sentencing, they compared Black and white men arrested for similar offenses with similar criminal histories, rather than those convicted of similar offenses. Measured from the point of arrest, they found that Black men received sentences almost ten percent longer than comparable white men. More strikingly, they found that a large share of that gap, more than half in their main analysis, could be explained by prosecutors' initial charging decisions, and in particular by the decision to bring charges carrying mandatory minimum sentences. Black defendants arrested for comparable conduct were substantially more likely to face such charges. In a companion 2013 article in the Yale Law Journal, Starr and Rehavi examined whether Booker's restoration of judicial discretion had increased racial disparity, as the Commission's reports suggested. Using their arrest-based design, they found no evidence that it had; instead, they argued that the apparent post-Booker increase in the Commission's data was better explained by other changes occurring over the same period than by judges' new discretion, and they again located much of the disparity in prosecutors' use of mandatory minimum charges. Their work shifted the focus of the disparity debate from judges to prosecutors. Not all researchers agreed about Booker. Crystal Yang, in a 2015 article in the Journal of Legal Studies titled "Free at Last? Judicial Discretion and Racial Disparities in Federal Sentencing," matched Commission data to judge identities and used the random assignment of cases to study how individual judges' sentencing changed after Booker. She found that racial disparities increased after Booker and that a meaningful portion of the increase was attributable to judges' use of their newly restored discretion. In a separate 2014 article in the New York University Law Review, Yang found that differences among judges in the same district, which the guidelines had narrowed, widened after Booker. Her findings supported the view that judicial discretion does contribute to disparity, even if prosecutors contribute more. Alma Cohen and Crystal Yang pursued the question of which judges produce disparities in a 2019 article in the American Economic Journal: Economic Policy. Using random assignment within districts, they found that judges appointed by Republican presidents sentenced Black defendants to substantially longer terms than similar non-Black defendants, compared with the difference among judges appointed by Democratic presidents, a gap of roughly three months. They also found that the racial gap was smaller among female judges. The finding connects the ideological voting documented in Chapters 4 and 5 to racial disparity: the identity of the sentencing judge, including the party of the appointing president, affects the size of the racial gap a defendant faces. The main studies, their designs, and their findings are summarized in Table 3. The table should be read with the design column in view, because studies that start their comparison at different stages of the process are answering different questions. Table 3. Selected studies of racial disparity in federal sentencing. Study Data and period Comparison point Headline finding Mustard (2001) Commission data, early 1990s Sentencing, same offense level and history Longer sentences for Black and Hispanic defendants; departures a major channel Rehavi and Starr (2014) Linked federal records, arrest to sentence Arrest, similar offense and history Black men's sentences almost 10% longer; over half explained by charging, especially mandatory minimums Yang (2015) Commission data matched to judges, around Booker Sentencing, same judge before and after Racial gap grew after Booker, partly through judicial discretion Cohen and Yang (2019) Commission data with judge identity Randomly assigned judges Republican appointees' racial gap about three months larger than Democratic appointees' Sentencing Commission (2017) Fiscal years 2012–2016 Sentencing, similar offenders Black men's sentences 19.1% longer than white men's Sentencing Commission (2023) Fiscal years 2017–2021 Sentencing, similar offenders Black men 13.4% and Hispanic men 11.2% longer; less likely to receive probation Crack, powder, and disparity by design Some disparities arise not from how actors exercise discretion but from the design of the law itself. The hundred-to-one ratio between crack and powder cocaine is the leading example. Because crack offenders were overwhelmingly Black, historically well over four-fifths of those sentenced federally for crack offenses, while powder cocaine offenders were more racially mixed, the ratio produced large racial differences in average sentences without any individual decision-maker treating similar defendants differently. The Sentencing Commission itself repeatedly urged Congress, in reports beginning in 1995, to reduce the ratio, citing both the lack of pharmacological justification and its disparate racial impact. Congress acted in the Fair Sentencing Act of 2010, reducing the ratio to eighteen to one, and in the First Step Act of 2018, making that reduction retroactive for many prisoners and expanding the "safety valve" that allows some low-level drug offenders to be sentenced below mandatory minimums. The Supreme Court's decision in Pulsifer v. United States (2024), which read the expanded safety valve's eligibility criteria narrowly, limited the number of defendants who can benefit from that reform. The Commission has continued to revise the guidelines in ways relevant to disparity, including 2023 amendments reducing sentences for offenders with no criminal history points and limiting the effect of certain criminal history enhancements, and a 2024 amendment restricting the use of acquitted conduct in calculating the guideline range. The crack ratio illustrates why "disparity" is not a single concept. A disparity study that controls for the offense of conviction will find no racial disparity between a Black and a white crack defendant who receive the same sentence, and will not register the fact that the law treats crack so much more harshly than a chemically similar drug more often associated with white defendants. Disparity built into the law is invisible to designs that take the law as given. What the residual means Every study discussed here reports a residual: the difference in outcomes that remains after controlling for measured factors. Two opposite objections can be raised against any such residual. The first is that it overstates discrimination, because unmeasured legitimate factors may differ by race. Criminal history categories are coarse; two offenders in the same category may have very different records. The seriousness of conduct within an offense may vary in ways the guidelines do not capture. Defendants may differ in the quality of their counsel, which is legitimate from the court's perspective even if troubling from society's. The second objection runs the other way: the residual may understate discrimination, because some of the "legitimate" factors controlled for are themselves products of bias. Criminal history reflects past policing, and if Black neighborhoods are policed more intensively, Black defendants will accumulate longer records for the same underlying conduct. Controlling for criminal history then removes from the measured disparity a component that is itself racially influenced. Controlling for the offense of conviction, as Rehavi and Starr showed, removes any disparity that arises in charging. These objections cannot both be resolved by adding more controls, because they pull in opposite directions. The most useful studies accept this and design their comparisons to answer a specific question: is there disparity at sentencing, holding charging fixed? Is there disparity in charging, holding arrest fixed? Is there disparity in the way a particular judge treats defendants of different races? Each design answers one question well and the others not at all. The accumulated evidence, read together, supports a clear conclusion: racial disparity in federal sentences is real, it arises at multiple stages, the largest single contributor identified so far is prosecutorial charging, and judicial discretion contributes a smaller but measurable share that varies with the identity of the judge. Chapter 7: Before and Beyond the Sentence The previous chapter ended with the observation that racial disparity in sentences arises at several stages of the criminal process, and that the stage before sentencing, the prosecutor's charging decision, appears to contribute more than the sentence itself. This chapter follows that insight to three other points in the process where the evidence is rich and the stakes high: the decision to detain or release a defendant before trial, the selection of the jury, and the decision to seek and impose death. Each has generated a distinctive empirical method. Each has also produced a distinctive legal response, and the gap between what the statistics show and what the law will accept as proof is itself one of the central lessons of the field. The benchmark problem and the outcome test Measuring discrimination requires a comparison, and the choice of comparison, the benchmark, is often the hardest part. Suppose police search Black drivers at twice the rate they search white drivers. Is that discrimination? It depends on whether Black and white drivers who are stopped differ in the likelihood that they are carrying contraband, and that is exactly the thing the searches are meant to discover. A simple comparison of rates cannot distinguish bias from differences in the underlying conduct. Economists devised an ingenious response known as the outcome test, whose logic traces back to Gary Becker's work on the economics of discrimination. If decision-makers apply the same standard to everyone, then the marginal person they act upon, the one who just barely met the threshold, should have the same probability of the relevant outcome regardless of race. If officers search Black drivers on less evidence than white drivers, then searches of Black drivers will turn up contraband less often, because the officers are searching people who are, on average, less likely to be carrying it. A lower "hit rate" for one group is thus evidence of a lower threshold for acting against that group. John Knowles, Nicola Persico, and Petra Todd applied this logic to highway searches in a 2001 article in the Journal of Political Economy, and it has since been refined and applied widely. The Stanford Open Policing Project, which assembled records of roughly one hundred million traffic stops, reported in a 2020 article in Nature Human Behaviour led by Emma Pierson that Black drivers were searched on the basis of less evidence than white drivers, and that the share of stopped drivers who were Black fell after dark, when officers can less easily see a driver's race. The outcome test's difficulty is that it concerns the marginal case, which is not directly observable; the average hit rate may differ from the marginal one. Researchers have developed statistical methods for estimating marginal outcomes, and the most powerful application of those methods to courts concerns bail. Bail and the marginal defendant Pretrial detention is among the most consequential decisions in American criminal justice. A defendant held before trial may lose a job, housing, or custody of children, and faces strong pressure to plead guilty to secure release. In a 2018 article in the American Economic Review, Will Dobbie, Jacob Goldin, and Crystal Yang used the random assignment of cases to bail judges in Philadelphia and Miami-Dade County to estimate the effects of detention itself. Because some judges were systematically more likely to detain than others, defendants who happened to draw a stricter judge were more likely to be detained for reasons unrelated to their own characteristics. The researchers found that pretrial detention significantly increased the probability of conviction, driven largely by an increase in guilty pleas, and reduced later formal employment. Detention, in other words, does not merely precede the outcome of a case; it helps determine it. In a companion study published in the Quarterly Journal of Economics the same year, David Arnold, Will Dobbie, and Crystal Yang used the same setting to test for racial bias in bail decisions. The randomly varying strictness of judges allowed them to estimate the rearrest rate of the marginal released defendant for each race, precisely the quantity the outcome test requires. They found that marginal Black defendants who were released were substantially less likely to be rearrested for a new crime before trial than marginal white defendants. Judges were, in effect, requiring Black defendants to be less risky than white defendants before releasing them. The bias appeared among both white and Black judges, and it was larger among less experienced judges and those serving part-time. The authors concluded that the pattern was most consistent with racial stereotypes that led judges to overestimate the risk posed by Black defendants, particularly in the rushed conditions of bail hearings where judges have little time and information. Partly in response to concerns about both the consequences of detention and the biases of human decision-makers, many jurisdictions adopted algorithmic risk assessment tools to inform pretrial and sentencing decisions. These tools created a new empirical controversy. In 2016, the news organization ProPublica analyzed a widely used tool, COMPAS, in Broward County, Florida, and reported that among defendants who did not go on to reoffend, Black defendants were nearly twice as likely as white defendants to have been classified as high risk. The tool's developer responded that its scores were equally predictive for both groups: a given score corresponded to about the same probability of reoffending regardless of race. Both claims were accurate, and computer scientists soon showed why they could not both be satisfied. Independent papers in 2017 by Alexandra Chouldechova and by Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan demonstrated mathematically that when two groups have different underlying rates of the predicted outcome, no imperfect predictive tool can simultaneously achieve equal accuracy of scores across groups and equal rates of false positives across groups. The choice among definitions of fairness is not a technical question but a moral and legal one. The Wisconsin Supreme Court, in State v. Loomis (2016), permitted the use of COMPAS at sentencing but required that judges be warned of its limitations and that it not be the determinative factor. The episode is a reminder that quantification does not remove judgment from the system; it relocates it into the design of the instrument. The jury Jury selection has long been a site of racial exclusion, and it offers some of the clearest statistical evidence in the field. The legal framework is set by Batson v. Kentucky (1986), which held that the Equal Protection Clause forbids prosecutors from using peremptory challenges to strike prospective jurors because of race. Under Batson, once a defendant makes a prima facie showing that a strike was racially motivated, the prosecutor must offer a race-neutral reason, and the trial judge decides whether that reason is genuine or a pretext. Because almost any reason can be stated in race-neutral terms, Batson has been widely criticized as ineffective. Statistical evidence has repeatedly shown what individual Batson hearings rarely establish. In a 2012 article in the Iowa Law Review, Catherine Grosso and Barbara O'Brien examined jury selection in capital cases in North Carolina over roughly two decades and found that prosecutors struck eligible Black venire members at about twice the rate of other eligible venire members, approximately 53 percent compared with 26 percent, and that the disparity persisted after controlling for the reasons prosecutors commonly offered for strikes. Their study was prepared in connection with North Carolina's Racial Justice Act of 2009, which uniquely allowed capital defendants to use statistical evidence to prove that race was a significant factor in their cases. After several defendants won relief under the Act, the legislature repealed it in 2013; in 2020 the North Carolina Supreme Court held that the repeal could not constitutionally be applied to defendants whose claims were already pending. Two recent Supreme Court decisions turned on patterns that were, in essence, statistical. In Foster v. Chatman (2016), the Court found a Batson violation after the defendant obtained the prosecution's jury selection notes, in which the names of Black prospective jurors had been highlighted and marked. In Flowers v. Mississippi (2019), the Court reversed a conviction obtained by a prosecutor who, over six trials of the same defendant, had struck forty-one of the forty-two Black prospective jurors he could have struck. Justice Kavanaugh's majority opinion relied heavily on that history. Neither case adopted a general rule that statistical patterns establish a violation; both rested on the particular prosecutor's record. Does the racial composition of juries affect outcomes? In a 2012 article in the Quarterly Journal of Economics, Shamena Anwar, Patrick Bayer, and Randi Hjalmarsson studied felony trials in two Florida counties and took advantage of the fact that the racial composition of the jury pool, the group from which jurors are selected, varies randomly from day to day. They found that when the pool contained no Black members, Black defendants were convicted significantly more often than white defendants, a gap of roughly sixteen percentage points. When the pool included at least one Black member, the gap essentially disappeared. The finding implies that the composition of juries has real consequences for verdicts, and that even the possibility of including a Black juror changes outcomes, perhaps by changing how both prosecutors and defense lawyers exercise their challenges. Some states have responded to this evidence by moving beyond Batson. Washington adopted General Rule 37 in 2018, which asks whether an objective observer aware that implicit bias exists could view race as a factor in a peremptory strike, and which declares several commonly offered reasons, such as distrust of police or living in a high-crime neighborhood, presumptively invalid. California enacted a similar statute in 2020. These reforms represent a rare instance of courts and legislatures translating empirical findings about bias directly into procedural rules. Death and the limits of statistical proof The most famous confrontation between statistics and constitutional law occurred in McCleskey v. Kemp (1987). Warren McCleskey, a Black man sentenced to death in Georgia for killing a white police officer, relied on a study led by David Baldus, George Woodworth, and Charles Pulaski that examined more than two thousand Georgia murder cases from the 1970s. After controlling for many non-racial variables, the study found that defendants charged with killing white victims were about 4.3 times as likely to receive a death sentence as defendants charged with killing Black victims. The race of the victim, more than the race of the defendant, predicted who would be sentenced to die. The Supreme Court, by a vote of five to four, rejected McCleskey's claims. Justice Lewis Powell's majority opinion assumed the study was statistically valid but held that it did not establish that any decision-maker in McCleskey's own case had acted with discriminatory purpose, which the Equal Protection Clause requires. Powell also expressed concern that accepting the claim would call into question racial disparities throughout the criminal justice system, an argument Justice Brennan's dissent described as a fear of too much justice. After his retirement, Powell told his biographer, John Jeffries, that he would change his vote in McCleskey if he could. McCleskey established a doctrinal barrier that remains in place: statistical evidence of systemic disparity is not, by itself, enough to prove an equal protection violation in an individual case. A related barrier governs claims of selective prosecution. In United States v. Armstrong (1996), the Court held that a defendant seeking discovery to support such a claim must first produce some evidence that similarly situated defendants of other races could have been prosecuted but were not, a showing that is extremely difficult without the discovery the defendant is seeking. Empirical work on the death penalty has nevertheless continued and has occasionally prevailed in state courts. A 1990 review by the federal General Accounting Office of studies on capital sentencing found that in the large majority of studies, 82 percent, the race of the victim was associated with the likelihood of being charged with capital murder or receiving a death sentence. In State v. Gregory (2018), the Washington Supreme Court struck down that state's death penalty under the state constitution, relying in significant part on a statistical study by Katherine Beckett and Heather Evans finding that Black defendants in Washington were more likely to be sentenced to death than similarly situated white defendants. The court accepted what the United States Supreme Court in McCleskey would not: that systemic evidence of racial influence can render a punishment scheme arbitrary. Law's discomfort with numbers The contrast between McCleskey and Gregory captures a tension that runs through this chapter. Empirical methods identify disparities in the aggregate: across thousands of bail hearings, jury selections, or capital cases. American constitutional doctrine, for the most part, adjudicates individual claims and requires proof of intentional discrimination by identifiable actors. A statistical pattern can establish, with high confidence, that race influences outcomes somewhere in a system, while being unable to establish that it influenced any particular outcome. This mismatch is not simply a failure of courts to understand statistics. It reflects a genuine question about what individual justice requires. If a disparity study shows that one in ten defendants who received a harsh outcome would not have received it but for race, it cannot identify which one in ten. Courts have treated that uncertainty as a reason to deny relief to all. Critics argue that it should be a reason to reform the system that produces the disparity, and some legislatures and state courts have done so through measures such as the Racial Justice Act, jury selection rules like General Rule 37, and the abolition of capital punishment. What empirical work contributes, in either case, is knowledge that would otherwise be unavailable. No individual bail hearing reveals that marginal Black defendants are being held to a higher standard. No individual jury selection reveals that Black venire members are struck at twice the rate of others. No individual capital trial reveals that the victim's race predicts the sentence. These patterns become visible only in aggregate, and only because researchers built the datasets and devised the methods to see them. Chapter 8: Noise, Mood, and the Individual Judge The preceding chapters have been concerned mostly with systematic influences on judging: ideology, race, colleagues, and the structure of legal rules. This chapter turns to something less orderly. Even when two judges share an ideology, sit in the same courthouse, and apply the same law, they may decide similar cases very differently. And the same judge may decide similar cases differently on different days. Daniel Kahneman, Olivier Sibony, and Cass Sunstein, in their 2021 book Noise, gave this phenomenon a name and argued that it is at least as important a source of injustice as bias. Bias is a consistent tilt in one direction. Noise is unwanted variability in judgments that should be identical. A legal system can be free of bias against any group and still be deeply unfair if the outcome of a case depends on which judge happens to hear it, or on what time of day it is heard. Refugee roulette The most striking documentation of judge-to-judge variation comes from immigration adjudication, where individual judges decide life-altering claims with limited review and where caseloads are large enough to reveal patterns clearly. In a 2007 article in the Stanford Law Review, titled "Refugee Roulette: Disparities in Asylum Adjudication," Jaya Ramji-Nogales, Andrew Schoenholtz, and Philip Schrag examined asylum decisions at every level of the system: asylum officers in the Department of Homeland Security, immigration judges in the Department of Justice, the Board of Immigration Appeals, and the federal courts of appeals. Their central finding was that the chance of winning asylum depended enormously on the identity of the decision-maker. Within the same immigration court, where cases were assigned without regard to their merits, some judges granted asylum to applicants from a given country at many times the rate of their colleagues. Applicants from the same country, presenting claims through the same court, faced odds that varied from long to favorable depending on the judge they drew. The authors expanded their study into a 2009 book with the same title, and the Transactional Records Access Clearinghouse at Syracuse University has since published judge-by-judge asylum grant rates that continue to show wide variation. The study also produced one of the most practically important numbers in empirical legal studies. Asylum seekers who were represented by counsel before immigration judges won asylum at a rate of 45.6 percent, nearly three times the 16.3 percent rate for those who appeared without a lawyer. Part of that difference surely reflects selection, since lawyers may decline weak cases. But the magnitude, and later research pointing in the same direction, suggests that representation itself substantially affects outcomes in a proceeding where the government is always represented and the applicant often is not. Similar variation has been documented in other high-volume adjudicative systems. Administrative law judges deciding Social Security disability appeals have long shown wide differences in allowance rates. Economists have turned that variation into a research tool: because disability cases are assigned to judges in a manner unrelated to the applicant's characteristics, the leniency of the assigned judge serves as an instrument for estimating the effect of receiving benefits on employment, as Nicole Maestas, Kathleen Mullen, and Alexander Strand did in a 2013 article in the American Economic Review. The same features that make the variation useful for research make it troubling for justice: an applicant's benefits may depend less on the severity of the disability than on the docket assignment. The gambler's fallacy Variation within a single judge's decisions can be as revealing as variation between judges. In a 2016 article in the Quarterly Journal of Economics, Daniel Chen, Tobias Moskowitz, and Kelly Shue examined decisions by asylum judges, along with loan officers reviewing applications and baseball umpires calling pitches, for evidence of the gambler's fallacy: the mistaken belief that random events must balance out in the short run, so that after several heads a tail becomes "due." Because cases are assigned to immigration judges in roughly the order they arrive, the merits of successive cases should be independent: whether the previous applicant deserved asylum says nothing about whether the next one does. Yet the researchers found that judges were significantly less likely to grant asylum immediately after granting it in the previous case, and the effect was stronger after a streak of grants. The judges appeared to be engaging in a form of negative autocorrelation, subconsciously treating a run of grants as a sign that the next case should be denied. The effect was larger among judges with less experience and among those facing heavier caseloads. The same pattern appeared among loan officers and umpires, suggesting a general feature of sequential human judgment rather than a failing peculiar to immigration courts. The finding is modest in magnitude compared with the differences between judges, but it is remarkable in kind. It shows that an applicant's chance of asylum can depend not only on which judge hears the case, but on the outcome of the unrelated case heard just before. The hungry judge, and a lesson in skepticism No empirical study of judging has attracted more public attention than a 2011 article in the Proceedings of the National Academy of Sciences by Shai Danziger, Jonathan Levav, and Liora Avnaim-Pesso. The authors examined more than a thousand rulings by Israeli parole boards, each chaired by a judge, over a ten-month period. The boards' days were divided into three sessions separated by a mid-morning snack and a lunch break. The researchers reported that the proportion of favorable rulings began each session at about 65 percent and fell steadily, nearly to zero, by the end of the session, then jumped back to about 65 percent after the break. They interpreted the pattern as evidence that mental depletion led judges to default to the easier, status quo decision of denying parole as they grew tired and hungry, and that food and rest restored their capacity for more demanding decisions. The study was widely reported, often as proof that justice depends on what the judge ate for breakfast, an old realist quip that seemed at last to have found its data. But it also drew immediate and substantial criticism, and the debate is an object lesson in how to read empirical claims. In a letter published in the same journal later in 2011, Keren Weinshall-Margel and John Shapard, drawing on their knowledge of how Israeli parole boards actually operated, pointed out that the order of cases was not random. Prisoners without legal representation, whose requests were less likely to be granted, tended to be heard at the end of each session, because their lawyers were not present to request earlier scheduling. Boards also tried to complete all cases from a given prison before a break. If weaker cases were systematically clustered at the end of sessions, the decline in favorable rulings could reflect case order rather than judicial fatigue. The original authors responded that their results held after accounting for representation, and the exchange did not fully resolve the question. A later critique came from a different direction. In a 2016 article in Judgment and Decision Making, Andreas Glöckner used simulations to show that a pattern resembling the one Danziger and his colleagues reported could arise from rational time management alone. If judges tend to end sessions before starting a case they expect to take a long time, and if favorable rulings take longer to deliver than unfavorable ones, then the observed decline in favorable rulings at the ends of sessions would emerge even if judges' decisions were entirely unaffected by fatigue. The simulations did not prove that this was what happened, but they showed that the data were consistent with an explanation that involved no depletion at all. The hungry judge study remains contested, and it is often still cited as settled fact. Its lesson for readers of empirical work is not that its authors were careless, but that a striking pattern in observational data can have several explanations, that institutional knowledge of how a court actually operates is essential to interpreting such patterns, and that the explanations which spread most widely are not always those with the strongest support. A similar cautionary tale involves the weather. In a 2019 article in the American Economic Journal: Applied Economics, Anthony Heyes and Soodeh Saberian reported that U.S. immigration judges were less likely to grant asylum on hotter days. Holger Spamann later reanalyzed the data in a 2022 comment published in the same journal. He reported that correcting coding errors, data errors, and sample selection problems cut the estimated effect by about two-thirds and left it statistically indistinguishable from zero, and that extending the sample to a longer span of years reversed its direction altogether. The episode is a reminder that replication, and access to the data needed for it, are not academic niceties but the means by which the field corrects itself. Emotions and extraneous events Other studies of extraneous influences have rested on stronger designs. In a 2018 article in the American Economic Journal: Applied Economics, Ozkan Eren and Naci Mocan examined sentencing decisions by juvenile court judges in Louisiana and asked whether they were affected by the outcomes of Louisiana State University football games. Because the outcome of a game, and particularly whether a team expected to win suffers an upset loss, is unrelated to the facts of the juvenile cases decided in the following week, it provides a natural experiment in the effects of emotional shocks. The researchers found that unexpected losses were followed by longer sentences, that the effect was concentrated among judges who had graduated from LSU, and that the burden fell disproportionately on Black defendants. Expected losses and wins had no comparable effect. The specificity of the finding, which appears only after surprising losses and only among judges with an emotional stake in the team, makes it harder to explain away than many studies of extraneous factors. Judges in the laboratory Observational studies must infer psychological mechanisms from patterns in decisions. A complementary body of work tests judges directly, by presenting them with hypothetical cases in controlled experiments. The most extensive program of such research was conducted by Chris Guthrie, Jeffrey Rachlinski, and Andrew Wistrich. In a 2001 article in the Cornell Law Review titled "Inside the Judicial Mind," they presented 167 federal magistrate judges with problems designed to test five cognitive illusions well documented in ordinary people: anchoring, framing, hindsight bias, the inverse fallacy, and egocentric bias. In one experiment, half the judges evaluated a personal injury case after first ruling on a meritless motion to dismiss for failure to meet the federal courts' $75,000 amount-in-controversy requirement. Judges exposed to that irrelevant number awarded substantially lower damages than judges who were not, a classic anchoring effect. The judges proved susceptible to most of the illusions tested, though somewhat less than lay subjects to some. In later work the same authors found that judges relied heavily on intuition, and in a 2009 article in the Notre Dame Law Review, Rachlinski, Sheri Lynn Johnson, Wistrich, and Guthrie reported that trial judges, like the general population, displayed implicit racial associations on standard tests, though judges who were aware of the need to monitor their own judgments could, in some conditions, compensate. German researchers produced an even starker anchoring result. In a 2006 article in the Personality and Social Psychology Bulletin, Birte Englich, Thomas Mussweiler, and Fritz Strack asked experienced legal professionals, including judges, to sentence a hypothetical shoplifting defendant after being given a prosecutor's sentencing demand determined, in full view of the participants, by the roll of loaded dice. Participants whose dice produced a higher number imposed longer sentences, even though they knew the number was random. Laboratory studies have obvious limits. Hypothetical cases lack the stakes, time, briefing, and argument of real litigation, and judges may treat an experimental survey less carefully than a real decision. But they are the best available way to isolate a specific cognitive mechanism, and their consistency with observational findings, such as the importance of prosecutorial sentencing recommendations in real cases, strengthens confidence in both. What to do about noise The evidence surveyed in this chapter establishes that outcomes in adjudication depend, sometimes heavily, on factors that the law regards as irrelevant: the identity of the decision-maker, the sequence of cases, emotional events, and arbitrary numbers. Some of the most famous findings are contested, and a careful reader will weigh them accordingly. But the core finding, that different judges presented with similar cases reach systematically different results, has been replicated too often and in too many settings to doubt. The legal system has several tools for reducing noise, each with costs. Sentencing guidelines, as Chapter 6 showed, reduced but did not eliminate differences between judges, and they transferred discretion to prosecutors. Appellate review corrects some errors but is costly and deferential, and in high-volume systems such as immigration, few decisions are reviewed. Multi-member panels average out individual idiosyncrasies, which is one reason appellate courts sit in panels, but they are expensive. Structured decision tools and checklists can discipline judgment, and algorithms can deliver perfect consistency, but, as the previous chapter showed, consistency is not the same as fairness, and algorithmic tools can embed and conceal the very disparities they were meant to remove. Perhaps the most important remedy is transparency. Much of the noise documented in this chapter was discovered only because researchers gained access to decision-level data. Where such data are published, as with immigration judges' grant rates, decision-makers and the public can at least see the variation. Where they are withheld, as with the identity of sentencing judges in the Sentencing Commission's public files, noise remains invisible. A system that takes the promise of equal treatment seriously has good reason to measure how well it keeps it. Chapter 9: What Wins Lawyers make strategic choices constantly: whether to sue, where to file, whether to demand a jury, whether to settle or try the case, whether to appeal, how to argue, and whom to recruit as allies. For most of the history of the profession, these choices were guided by experience, anecdote, and professional lore. Empirical legal studies has begun to supply evidence about which choices tend to succeed. That evidence is valuable, but it is also treacherous, because the single most important fact about litigation statistics is that the cases in them chose themselves. Understanding why is the key to using any number in this chapter correctly. The selection of disputes In 1984, George Priest and Benjamin Klein published an article in the Journal of Legal Studies titled "The Selection of Disputes for Litigation" that has shaped every serious discussion of litigation outcomes since. Their argument begins with the observation that trial is expensive and uncertain, so rational parties will settle most disputes. The cases that go to trial are those in which the parties disagree about the likely outcome, and parties disagree most when the case is close. Clear cases, in which both sides can foresee the result, settle. Close cases, in which reasonable predictions differ, are tried. The implication is striking. Whatever the underlying legal standard, and however favorable or unfavorable it is to plaintiffs, the cases that survive to trial will tend to be those near the decision threshold, and plaintiffs will therefore tend to win about half of them. Priest and Klein did not claim that plaintiff win rates would always be exactly fifty percent. Asymmetric stakes, as when a defendant cares about precedent or reputation more than the plaintiff does, differences in information, and other factors shift the rate, as Daniel Kessler, Thomas Meites, and Geoffrey Miller showed in a 1996 article in the same journal. But the core insight has been borne out repeatedly: trial win rates tell us much less about the law than they appear to, because the process that selects cases for trial adjusts to the law. For anyone evaluating litigation statistics, the selection hypothesis supplies a warning that applies far beyond trials. A high success rate for some litigation strategy may mean that the strategy works, or it may mean that the strategy is used only in cases likely to succeed anyway. A low reversal rate for some judge may mean the judge is accurate, or that losing parties before that judge rarely appeal. The raw numbers cannot distinguish these possibilities; only research designs that account for selection can. The haves and the repeat players In 1974, the sociologist Marc Galanter published an article in the Law & Society Review with a title that became a slogan: "Why the 'Haves' Come Out Ahead." Galanter distinguished between "one-shotters," parties who litigate rarely, such as an individual injured in an accident or a tenant facing eviction, and "repeat players," parties who litigate often, such as insurance companies, landlords, prosecutors, and the government. Repeat players, he argued, enjoy structural advantages. They have expertise and access to specialists. They can spread costs across many cases. They can play for rules as well as for immediate results, settling cases likely to produce unfavorable precedent and litigating those likely to produce favorable ones. Over time, the rules themselves tilt in their favor. Empirical tests have largely confirmed Galanter's predictions. In a 1992 article in the American Journal of Political Science, Donald Songer and Reginald Sheehan examined decisions in the federal courts of appeals and found that parties' success rates rose with their resources and experience: the federal government fared best, followed by state and local governments, businesses, and finally individuals. A follow-up study by Songer, Sheehan, and Susan Haire, published in the Law & Society Review in 1999, found that the advantage persisted over most of the twentieth century. The pattern holds even when the weaker party is the appellant, which makes it harder to attribute entirely to selection. At the Supreme Court, the most successful repeat player by far is the federal government, represented by the Solicitor General. The Solicitor General's office wins a substantially higher share of its cases than other litigants, both as a party and as an amicus curiae, and it is highly influential in the Court's decisions about which cases to hear. Its success reflects several things at once: the government's careful selection of which cases to bring, the office's expertise, the Justices' trust in its candor, and the institutional weight of the executive branch. Private Supreme Court advocacy has followed Galanter's logic too. In a 1995 article in the Journal of Politics, Kevin McGuire found that lawyers with more prior experience arguing before the Court were more successful, even after accounting for other factors. Richard Lazarus, in a 2008 article in the Georgetown Law Journal, documented the rise since the 1980s of an elite Supreme Court bar, a small group of specialists, many of them former Solicitor General's office lawyers, who argue a growing share of the Court's cases and who have become especially skilled at persuading the Court to take cases on behalf of business clients. Lazarus argued that this specialization has influenced not only which side wins but which issues the Court decides at all. Advocacy at the Court Does the quality of oral argument matter? For most of the Court's history the question could not be answered empirically, because there was no independent measure of argument quality. Timothy Johnson, Paul Wahlbeck, and James Spriggs found one in the papers of Justice Harry Blackmun, who for many years graded the lawyers arguing before him on a numerical scale. In a 2006 article in the American Political Science Review, they showed that Justices were more likely to vote for the side whose lawyer received a higher grade from Blackmun, even after accounting for the Justices' ideology, the position of the Solicitor General, and other factors. The effect was largest for Justices ideologically closest to the middle, as one would expect if argument matters most to those not already committed. Other researchers asked whether the Justices' behavior at argument predicts the result. In a 2010 article in the Journal of Legal Studies, Lee Epstein, William Landes, and Richard Posner examined oral argument transcripts and found that the side asked more questions tended to lose, both in the Court's overall decision and in the votes of individual Justices. Timothy Johnson, Ryan Black, Jerry Goldman, and Sarah Treul reported similar findings in a 2009 study. The Justices, it seems, use questions largely to probe the weaknesses of the side they are inclined to rule against, and the balance of questioning is a useful, though imperfect, predictor of the outcome. Amicus briefs Amicus curiae briefs, filed by non-parties to support one side or to present a perspective, have multiplied at the Supreme Court. Their influence depends on the stage. In a 1988 article in the American Political Science Review, Gregory Caldeira and John Wright showed that amicus briefs filed at the certiorari stage significantly increased the likelihood that the Court would grant review, and that this was true even of briefs urging the Court to deny review. The briefs apparently served as a signal that a case was important enough to attract organized attention. On the merits, the evidence is more mixed. Joseph Kearney and Thomas Merrill, in a 2000 article in the University of Pennsylvania Law Review, examined amicus filings over several decades and found that briefs filed by the Solicitor General were strongly associated with success for the supported side, while the effects of other amicus briefs were modest and inconsistent. Paul Collins, in Friends of the Supreme Court (2008), found that amicus briefs can influence Justices, particularly by providing information, but that their effects are conditioned by ideology. The practical implication is that recruiting amici is most valuable when seeking review and when the brief comes from a source the Court trusts. Choosing the tribunal A litigant's most consequential strategic decision may be where to file. The empirical findings about judges surveyed in earlier chapters, that outcomes depend on who decides, give that choice obvious value. Patent litigation has offered the clearest examples. For many years, patent plaintiffs filed a large share of all federal patent suits in the Eastern District of Texas, where local rules, jury pools, and judicial practices were perceived as favorable to patent holders. The Supreme Court's decision in TC Heartland LLC v. Kraft Foods Group Brands LLC (2017), which restricted where domestic corporations can be sued for patent infringement, curtailed that pattern. Filings then shifted toward the Waco division of the Western District of Texas, where a single judge heard nearly all cases and had actively encouraged patent litigation. In 2022, the chief judge of that district ordered that patent cases filed in Waco be distributed randomly among the district's judges, and filings there fell. As Chapter 5 described, similar concerns about single-judge divisions later extended to suits challenging federal policies. The choice between judge and jury is another strategic decision on which data exist. In a 1992 article in the Cornell Law Review, Kevin Clermont and Theodore Eisenberg examined federal trials and found that plaintiffs won more often in bench trials than in jury trials in product liability and medical malpractice cases, contrary to the widespread belief that juries favor injured plaintiffs. Clermont and Eisenberg explained the finding largely through selection: because lawyers share the belief that juries are sympathetic, defendants in weak cases settle rather than face a jury, while plaintiffs with weak cases demand one, so the cases that reach jury verdicts are, on average, weaker for plaintiffs. The folk wisdom produces behavior that makes the folk wisdom appear false in the statistics. Trial, settlement, and appeal The biggest strategic fact about modern American litigation is that trials have almost disappeared. In a 2004 article in the Journal of Empirical Legal Studies, "The Vanishing Trial," Marc Galanter documented that the proportion of federal civil cases resolved by trial fell from about 11.5 percent in 1962 to 1.8 percent in 2002, even as the number of cases filed rose sharply. The share has since fallen further, to around one percent. In federal criminal cases, a 2019 analysis by the Pew Research Center found that only about two percent of defendants went to trial; most of the rest pleaded guilty. Litigation strategy is overwhelmingly settlement strategy. Are settlement decisions made well? In a 2008 article in the Journal of Empirical Legal Studies, Randall Kiser, Martin Asher, and Blakeley McShane examined California cases in which the parties rejected a settlement offer and went to trial, and compared the trial result to the rejected offer. They found that plaintiffs made the wrong decision, obtaining a result no better than or worse than the offer they rejected, in about 61 percent of cases, while defendants did so in about 24 percent. But defendants' errors were far more costly on average, because when they misjudged a case, they misjudged it by large amounts. Both sides, in other words, systematically overestimated their prospects at trial, a finding consistent with psychological research on optimism bias. Appeals present a sharper asymmetry. In a 2001 article in the American Law and Economics Review, "Appeal from Jury or Judge Trial: Defendants' Advantage," Clermont and Eisenberg found that defendants who appealed losses at trial in federal civil cases obtained reversals in about a third of appeals, while plaintiffs appealing losses succeeded in only about an eighth. The asymmetry was especially large in appeals from jury trials. The authors suggested that appellate judges, who view trial records from a distance, may perceive trial courts and juries as more favorable to plaintiffs than they actually are, and correct for a bias that the evidence suggests does not exist. In employment discrimination cases, Clermont and Stewart Schwab found an even more pronounced gap: defendants appealing trial losses obtained reversals more than forty percent of the time, while plaintiffs succeeded in about six percent. The arbitration of employment claims, which has grown as employers have required it in employment contracts, offers a further comparison. In a 2011 article in the Journal of Empirical Legal Studies, Alexander Colvin examined employment arbitration cases administered by the American Arbitration Association and found that employees won about one in five cases, with median awards far lower than those reported in federal court trials, though comparisons across forums are complicated by the differences in cases that reach each. The principal findings of this chapter are gathered in Table 4, with the caveat that attaches to each. Table 4. Selected findings on litigation strategy and success. Strategic question Finding Source Principal caveat Do trial win rates show which side the law favors? Tried cases cluster near the decision threshold Priest and Klein (1984) Asymmetric stakes shift the rate Do resources matter on appeal? Government and business outperform individuals Songer and Sheehan (1992) Case selection by stronger parties Does oral argument quality matter? Higher-graded advocates win more votes Johnson, Wahlbeck, and Spriggs (2006) Grades from a single Justice Do amici help get review? Cert-stage amicus briefs raise grant rates Caldeira and Wright (1988) Effects on merits weaker Should a plaintiff choose a jury? Plaintiffs won more bench trials in tort cases studied Clermont and Eisenberg (1992) Driven by which cases reach each Is it worth appealing? Defendants' appeals succeed far more than plaintiffs' Clermont and Eisenberg (2001) Selection of which losses are appealed Should one reject a settlement offer? Most plaintiffs who went to trial did no better Kiser, Asher, and McShane (2008) Only cases with recorded offers Using the numbers The commercial legal analytics industry now offers practitioners statistics on individual judges' rulings, opposing counsel's track records, and motion grant rates by court. Lex Machina, which began as a project at Stanford and was acquired by LexisNexis, pioneered this work in intellectual property, and similar tools are now offered by the major legal research services. These tools make the empirical insights of this chapter directly available to practicing lawyers. They also make the chapter's warnings more urgent. A judge's grant rate on summary judgment motions reflects the motions filed before that judge, which reflect the lawyers' own expectations about the judge. A judge perceived as receptive to such motions will receive more of them, including weaker ones, and may therefore have a lower grant rate than a less receptive judge who receives only strong motions. Small numbers of cases produce unstable percentages. And differences in case mix across courts and judges are rarely fully captured in the categories the tools use. The lawyer who reads litigation statistics well asks the same questions that a careful researcher asks: which cases are in the denominator, how they got there, and what would have happened to the cases that are not. The most robust findings in this chapter, that repeat players prevail, that trials are rare and settlement decisions are frequently mistaken, that appeals favor defendants, and that argument quality and the choice of forum matter, have survived this scrutiny. Many of the numbers circulating in marketing materials for analytics products have not been subjected to it at all. Conclusion: Measuring the Promise of Equal Justice Holmes defined law as a prediction of what courts will do. A century later, prediction has become a technical enterprise, and its results bear directly on the questions this book has examined. In 2002, a group of scholars led by Theodore Ruger, Pauline Kim, Andrew Martin, and Kevin Quinn ran an experiment. Before the Supreme Court's October 2002 term was decided, they generated predictions for every argued case using two methods. One was a simple statistical model that used only six observable features of each case, such as the circuit of origin, the issue area, the type of petitioner, and the ideological direction of the lower court ruling, and ignored the legal arguments entirely. The other was the judgment of legal experts, many of them distinguished academics and former Supreme Court clerks, who were asked to predict cases in their areas of expertise. The results, published in the Columbia Law Review in 2004, were humbling for the experts. The model correctly predicted about 75 percent of case outcomes; the experts, about 59 percent. The experts did better at predicting individual Justices' votes in some cases, but the crude model, knowing nothing about law, outperformed them on the bottom line. In 2017, Daniel Martin Katz, Michael Bommarito, and Josh Blackman reported in PLOS ONE a machine learning model trained on the Supreme Court Database that predicted around 70 percent of case outcomes and 72 percent of individual Justices' votes over nearly two centuries of decisions, using only information available before each decision. In 2016, a team led by Nikolaos Aletras reported that a model analyzing the text of cases before the European Court of Human Rights could predict whether the Court would find a violation in about 79 percent of the cases studied. Since then, large language models have made it possible to analyze legal text at a scale and sophistication that would have seemed fanciful a decade ago, and commercial services now market predictions of motion outcomes and case values to lawyers and litigation funders. These achievements confirm the central empirical finding of this book: judicial behavior is patterned, and the patterns can be learned. They also sharpen the book's central caution. A model that predicts outcomes does not explain them. A model trained on past decisions will reproduce whatever regularities those decisions contain, including the ideological divisions of Chapters 4 and 5, the disparities of Chapters 6 and 7, and the noise of Chapter 8. Prediction is not justification. The fact that a Justice's vote can be forecast from the party of the president who appointed her does not tell us whether her reasoning was right. The politics of knowing Because empirical knowledge about judges is valuable, it is also contested. In 2019, France enacted a law prohibiting the use of data identifying individual judges to evaluate, analyze, compare, or predict their professional practices, backed by criminal penalties. The provision was a direct response to legal analytics companies that had begun publishing judge-by-judge statistics, including on asylum decisions, that revealed wide variation among judges. Its defenders argued that such analysis would encourage forum shopping and undermine public confidence in an independent judiciary. Its critics saw it as an effort to suppress precisely the kind of knowledge that makes judicial accountability possible. The American legal system has taken the opposite approach in principle while sometimes retaining the French instinct in practice. Judicial decisions are public, and the analytics industry operates freely. But the Sentencing Commission does not publish judge identities with its sentencing data. The emergency orders of the Supreme Court often omit individual votes. Assignment procedures in the courts of appeals have been, as Chapter 5 described, less transparent than the courts' claims of randomness would suggest. And federal court records remain behind PACER's paywall, which limits bulk research by anyone without substantial funding. The debate over judge shopping in 2024 and 2025 shows what is at stake. The empirical fact that the identity of the judge predicts the outcome was not news to litigators, who acted on it by filing in single-judge divisions. It became a matter of institutional concern when those filings produced nationwide rulings, and the responses, from the Judicial Conference's guidance on random assignment to the Supreme Court's limitation of universal injunctions in Trump v. CASA, treated the predictability of individual judges as a problem requiring structural correction. The episode demonstrates a proposition that the umpire metaphor obscures: institutions of judicial administration, such as case assignment, panel composition, and the scope of remedies, exist in part to manage the reality that judges differ. What follows Several conclusions follow from the evidence examined in this book, each of which could be reached only by combining its findings. First, the choice between "law" and "politics" as explanations of judicial behavior is a false one. The evidence shows ideology operating most strongly where law is least determinate, which is to say at the top of the judicial hierarchy and in the most contested areas of law. It shows legal rules and institutional structures, such as mixed panels, deference doctrines, and the prospect of review, measurably constraining ideological voting. The useful question is not whether judges are political but under what conditions their commitments decide cases, and how institutions can be designed so that those conditions are rare and visible. Second, racial disparity in criminal justice is real, is present at multiple stages, and is most heavily concentrated where discretion is least visible. The finding that prosecutorial charging contributes more to sentencing disparities than judicial sentencing does, and that bail decisions and jury selection show measurable racial effects, points reform toward stages of the process that have historically attracted less scrutiny than sentencing. It also points toward data. Charging decisions, plea offers, and pretrial detention decisions are recorded far less systematically than sentences, and what is not recorded cannot be evaluated. Third, the legal system's treatment of statistical evidence deserves reconsideration. McCleskey v. Kemp established that aggregate evidence of disparity does not by itself prove discrimination in an individual case, and that principle may be defensible as a matter of individual adjudication. But it cannot justify indifference at the level of institutions. When a court system learns that its bail judges hold Black defendants to a higher standard, or that its asylum outcomes depend on the luck of the docket, the response required is administrative and legislative rather than case-by-case. Some jurisdictions have responded, through jury selection rules like Washington's General Rule 37, sentencing reforms, and published decision data. Most have not. Fourth, the tools of empirical legal studies are now in everyone's hands, and their misuse is at least as likely as their use. Selection effects, omitted variables, circular measures, and small samples have produced misleading findings in peer-reviewed journals; they produce far more in the dashboards of commercial analytics. The contested history of the hungry judge study and the collapse of the temperature effect on reanalysis show how quickly striking claims spread and how slowly corrections follow. The discipline this book has tried to convey, of asking what was counted, how cases got into the data, and what the design can and cannot rule out, is the essential companion to the numbers. What remains unsettled Some of the most important questions remain open. It is not yet known how the end of Chevron deference will affect the influence of ideology in the courts of appeals, although the earlier evidence gives reason to expect some increase. It is not yet known whether the growing polarization of the federal judiciary's selection process, documented in the convergence of party and ideology on the Supreme Court, will produce similar convergence in the lower courts. The emergency docket's decisions remain poorly measured. And the effects of algorithmic tools, whether risk assessments in criminal courts or predictive models in civil litigation, on the behavior of judges and lawyers have barely begun to be studied. What is known is enough to change how a thoughtful citizen should hear the umpire metaphor. Judges do call balls and strikes, and the evidence shows that most of the time they call them the same way. But the strike zone is not identical from one umpire to the next, and in the closest cases, which are the ones that matter most, the call depends in measurable ways on who is behind the plate. The promise of equal justice under law is not a description of how courts behave. It is a standard against which their behavior can be measured. Empirical legal studies is the practice of taking that measurement honestly, and of insisting that the results matter. Notes Introduction 1. Confirmation Hearing on the Nomination of John G. Roberts, Jr. to be Chief Justice of the United States: Hearing Before the Senate Committee on the Judiciary, 109th Cong. (2005) (opening statement of John G. Roberts, Jr.). 1. Statement of Chief Justice John G. Roberts, Jr., November 21, 2018, as widely reported by the Associated Press. 2. United States Sentencing Commission, 2023 Demographic Differences in Federal Sentencing (November 2023). Chapter 1 1. Oliver Wendell Holmes, Jr., "The Path of the Law," Harvard Law Review (1897). 1. Jerome Frank, Law and the Modern Mind (New York: Brentano's, 1930). 2. C. Herman Pritchett, The Roosevelt Court: A Study in Judicial Politics and Values, 1937–1947 (New York: Macmillan, 1948). 3. Glendon Schubert, The Judicial Mind: The Attitudes and Ideologies of Supreme Court Justices, 1946–1963 (Evanston: Northwestern University Press, 1965). 4. Stuart S. Nagel, "Political Party Affiliation and Judges' Decisions," American Political Science Review (1961). 5. Jeffrey A. Segal and Harold J. Spaeth, The Supreme Court and the Attitudinal Model (Cambridge: Cambridge University Press, 1993); The Supreme Court and the Attitudinal Model Revisited (Cambridge University Press, 2002). 6. Lee Epstein and Gary King, "The Rules of Inference," University of Chicago Law Review (2002). 7. Jeffrey R. Kling, "Incarceration Length, Employment, and Earnings," American Economic Review (2006). Chapter 2 1. The Supreme Court Database, Modern Database, 2026 Release 01 (Pennsylvania State University), and Legacy Database (2021), with accompanying codebooks. 1. Carolyn Shapiro, "Coding Complexity: Bringing Law to the Empirical Analysis of the Supreme Court," Hastings Law Journal (2009). 2. Anna Harvey and Michael J. Woodruff, "Confirmation Bias in the United States Supreme Court Judicial Database," Journal of Law, Economics, and Organization (2013). 3. Donald R. Songer, The United States Courts of Appeals Database (1925–1996); Ashlyn K. Kuersten and Susan B. Haire, update (1997–2002); Gary Zuk, Deborah J. Barrow, and Gerard S. Gryski, Multi-User Database on the Attributes of United States Appeals Court Judges. 4. Federal Judicial Center, Integrated Database (civil, criminal, appellate, and bankruptcy files). 5. Theodore Eisenberg and Margo Schlanger, "The Reliability of the Administrative Office of the U.S. Courts Database: An Initial Empirical Analysis," Notre Dame Law Review (2003). 6. United States Sentencing Commission, Individual Offender Datafiles and Interactive Data Analyzer. 7. Bureau of Justice Statistics, Federal Justice Statistics Program. 8. Harvard Law School Library, Caselaw Access Project; Free Law Project, CourtListener and RECAP. Chapter 3 1. Jeffrey A. Segal and Albert D. Cover, "Ideological Values and the Votes of U.S. Supreme Court Justices," American Political Science Review (1989). 1. Andrew D. Martin and Kevin M. Quinn, "Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953–1999," Political Analysis (2002); Martin-Quinn Scores, covering the October 1937 through October 2024 terms. 2. Lee Epstein, Andrew D. Martin, Kevin M. Quinn, and Jeffrey A. Segal, "Ideological Drift among Supreme Court Justices: Who, When, and How Important?," Northwestern University Law Review (2007). 3. Micheal W. Giles, Virginia A. Hettinger, and Todd Peppers, "Picking Federal Judges: A Note on Policy and Partisan Selection Agendas," Political Research Quarterly (2001). 4. Lee Epstein, Andrew D. Martin, Jeffrey A. Segal, and Chad Westerland, "The Judicial Common Space," Journal of Law, Economics, and Organization (2007). 5. Adam Bonica and Michael J. Woodruff, "A Common-Space Measure of State Supreme Court Ideology," Journal of Law, Economics, and Organization (2015); Adam Bonica and Maya Sen, The Judicial Tug of War: How Lawyers, Politicians, and Ideological Incentives Shape the American Judiciary (Cambridge University Press, 2020). 6. Joshua B. Fischman and David S. Law, "What Is Judicial Ideology, and How Should We Measure It?," Washington University Journal of Law and Policy (2009). 7. Daniel E. Ho and Kevin M. Quinn, "How Not to Lie with Judicial Votes: Misconceptions, Measurement, and Models," California Law Review (2010). Chapter 4 1. Neal Devins and Lawrence Baum, The Company They Keep: How Partisan Divisions Came to the Supreme Court (Oxford University Press, 2019). 1. Dobbs v. Jackson Women's Health Organization, 597 U.S. 215 (2022); Students for Fair Admissions, Inc. v. President and Fellows of Harvard College, 600 U.S. 181 (2023); Loper Bright Enterprises v. Raimondo, 603 U.S. 369 (2024); Trump v. United States, 603 U.S. 593 (2024). 2. Lee Epstein, Christopher M. Parker, and Jeffrey A. Segal, "Do Justices Defend the Speech They Hate? In-Group Bias, Opportunism, and the First Amendment," Journal of Law and Courts (2018). 3. Lee Epstein and Jack Knight, The Choices Justices Make (Washington, DC: CQ Press, 1998). 4. Forrest Maltzman, James F. Spriggs II, and Paul J. Wahlbeck, Crafting Law on the Supreme Court: The Collegial Game (Cambridge University Press, 2000). 5. Mark J. Richards and Herbert M. Kritzer, "Jurisprudential Regimes in Supreme Court Decision Making," American Political Science Review (2002); Grayned v. City of Rockford, 408 U.S. 104 (1972); Police Department of Chicago v. Mosley, 408 U.S. 92 (1972). 6. Harold J. Spaeth and Jeffrey A. Segal, Majority Rule or Minority Will: Adherence to Precedent on the U.S. Supreme Court (Cambridge University Press, 1999). 7. Thomas G. Hansford and James F. Spriggs II, The Politics of Precedent on the U.S. Supreme Court (Princeton University Press, 2006). 8. Michael A. Bailey and Forrest Maltzman, The Constrained Court: Law, Politics, and the Decisions Justices Make (Princeton University Press, 2011). 9. Scott v. Harris, 550 U.S. 372 (2007); Dan M. Kahan, David A. Hoffman, and Donald Braman, "Whose Eyes Are You Going to Believe? Scott v. Harris and the Perils of Cognitive Illiberalism," Harvard Law Review (2009). 10. William Baude, "Foreword: The Supreme Court's Shadow Docket," NYU Journal of Law and Liberty (2015); Stephen Vladeck, The Shadow Docket: How the Supreme Court Uses Stealth Rulings to Amass Power and Undermine the Republic (Basic Books, 2023). Chapter 5 1. Cass R. Sunstein, David Schkade, and Lisa Michelle Ellman, "Ideological Voting on Federal Courts of Appeals: A Preliminary Investigation," Virginia Law Review (2004); Cass R. Sunstein, David Schkade, Lisa M. Ellman, and Andres Sawicki, Are Judges Political? An Empirical Analysis of the Federal Judiciary (Brookings Institution Press, 2006). 1. Lee Epstein, William M. Landes, and Richard A. Posner, The Behavior of Federal Judges: A Theoretical and Empirical Study of Rational Choice (Harvard University Press, 2013). 2. Richard L. Revesz, "Environmental Regulation, Ideology, and the D.C. Circuit," Virginia Law Review (1997). 3. Frank B. Cross and Emerson H. Tiller, "Judicial Partisanship and Obedience to Legal Doctrine: Whistleblowing on the Federal Courts of Appeals," Yale Law Journal (1998). 4. Lee Epstein, William M. Landes, and Richard A. Posner, "Why (and When) Judges Dissent: A Theoretical and Empirical Analysis," Journal of Legal Analysis (2011). 5. Joshua B. Fischman, "Interpreting Circuit Court Voting Patterns: A Social Interactions Framework," Journal of Law, Economics, and Organization (2015). 6. Christina L. Boyd, Lee Epstein, and Andrew D. Martin, "Untangling the Causal Effects of Sex on Judging," American Journal of Political Science (2010); Sean Farhang and Gregory Wawro, "Institutional Dynamics on the U.S. Court of Appeals: Minority Representation Under Panel Decision Making," Journal of Law, Economics, and Organization (2004). 7. Adam B. Cox and Thomas J. Miles, "Judging the Voting Rights Act," Columbia Law Review (2008); Jonathan P. Kastellec, "Racial Diversity and Judicial Influence on Appellate Courts," American Journal of Political Science (2013). 8. Adam S. Chilton and Marin K. Levy, "Challenging the Randomness of Panel Assignment in the Federal Courts of Appeals," Cornell Law Review (2015). 9. Judicial Conference of the United States, guidance on random case assignment in cases seeking statewide or nationwide relief (March 2024); Trump v. CASA, Inc., 606 U.S. ___ (2025). 10. Chevron U.S.A. Inc. v. Natural Resources Defense Council, Inc., 467 U.S. 837 (1984); Thomas J. Miles and Cass R. Sunstein, "Do Judges Make Regulatory Policy? An Empirical Investigation of Chevron," University of Chicago Law Review (2006); Kent Barnett and Christopher J. Walker, "Chevron in the Circuit Courts," Michigan Law Review (2017). Chapter 6 1. Marvin E. Frankel, Criminal Sentences: Law Without Order (Hill and Wang, 1973); Sentencing Reform Act of 1984, Pub. L. No. 98-473; 28 U.S.C. § 991(b)(1)(B); Mistretta v. United States, 488 U.S. 361 (1989). 1. United States v. Booker, 543 U.S. 220 (2005); Gall v. United States, 552 U.S. 38 (2007); Kimbrough v. United States, 552 U.S. 85 (2007). 2. James M. Anderson, Jeffrey R. Kling, and Kate Stith, "Measuring Interjudge Sentencing Disparity: Before and After the Federal Sentencing Guidelines," Journal of Law and Economics (1999). 3. Kate Stith and José A. Cabranes, Fear of Judging: Sentencing Guidelines in the Federal Courts (University of Chicago Press, 1998). 4. David B. Mustard, "Racial, Ethnic, and Gender Disparities in Sentencing: Evidence from the U.S. Federal Courts," Journal of Law and Economics (2001). 5. Darrell Steffensmeier, Jeffery Ulmer, and John Kramer, "The Interaction of Race, Gender, and Age in Criminal Sentencing: The Punishment Cost of Being Young, Black, and Male," Criminology (1998). 6. United States Sentencing Commission, Demographic Differences in Sentencing: An Update to the 2012 Booker Report (November 2017); 2023 Demographic Differences in Federal Sentencing (November 2023). 7. M. Marit Rehavi and Sonja B. Starr, "Racial Disparity in Federal Criminal Sentences," Journal of Political Economy (2014); Sonja B. Starr and M. Marit Rehavi, "Mandatory Sentencing and Racial Disparity: Assessing the Role of Prosecutors and the Effects of Booker," Yale Law Journal (2013). 8. Crystal S. Yang, "Free at Last? Judicial Discretion and Racial Disparities in Federal Sentencing," Journal of Legal Studies (2015); Crystal S. Yang, "Have Interjudge Sentencing Disparities Increased in an Advisory Guidelines Regime? Evidence from Booker," New York University Law Review (2014). 9. Alma Cohen and Crystal S. Yang, "Judicial Politics and Sentencing Decisions," American Economic Journal: Economic Policy (2019). 10. Fair Sentencing Act of 2010, Pub. L. No. 111-220; First Step Act of 2018, Pub. L. No. 115-391; Pulsifer v. United States, 601 U.S. 124 (2024). Chapter 7 1. John Knowles, Nicola Persico, and Petra Todd, "Racial Bias in Motor Vehicle Searches: Theory and Evidence," Journal of Political Economy (2001); Emma Pierson et al., "A Large-Scale Analysis of Racial Disparities in Police Stops across the United States," Nature Human Behaviour (2020). 1. Will Dobbie, Jacob Goldin, and Crystal S. Yang, "The Effects of Pretrial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges," American Economic Review (2018). 2. David Arnold, Will Dobbie, and Crystal S. Yang, "Racial Bias in Bail Decisions," Quarterly Journal of Economics (2018). 3. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner, "Machine Bias," ProPublica, May 23, 2016; Alexandra Chouldechova, "Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments," Big Data (2017); Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan, "Inherent Trade-Offs in the Fair Determination of Risk Scores," Proceedings of Innovations in Theoretical Computer Science (2017); State v. Loomis, 881 N.W.2d 749 (Wis. 2016). 4. Batson v. Kentucky, 476 U.S. 79 (1986); Catherine M. Grosso and Barbara O'Brien, "A Stubborn Legacy: The Overwhelming Importance of Race in Jury Selection in 173 Post-Batson North Carolina Capital Trials," Iowa Law Review (2012); State v. Ramseur, 374 N.C. 658 (2020). 5. Foster v. Chatman, 578 U.S. 488 (2016); Flowers v. Mississippi, 588 U.S. 284 (2019). 6. Shamena Anwar, Patrick Bayer, and Randi Hjalmarsson, "The Impact of Jury Race in Criminal Trials," Quarterly Journal of Economics (2012). 7. Washington Court Rules, General Rule 37 (2018); California Code of Civil Procedure § 231.7 (enacted 2020). 8. McCleskey v. Kemp, 481 U.S. 279 (1987); David C. Baldus, George Woodworth, and Charles A. Pulaski, Jr., Equal Justice and the Death Penalty: A Legal and Empirical Analysis (Northeastern University Press, 1990); John C. Jeffries, Jr., Justice Lewis F. Powell, Jr. (Charles Scribner's Sons, 1994). 9. United States v. Armstrong, 517 U.S. 456 (1996). 10. U.S. General Accounting Office, Death Penalty Sentencing: Research Indicates Pattern of Racial Disparities, GAO/GGD-90-57 (February 1990). 11. State v. Gregory, 192 Wash. 2d 1, 427 P.3d 621 (2018). Chapter 8 1. Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein, Noise: A Flaw in Human Judgment (Little, Brown Spark, 2021). 1. Jaya Ramji-Nogales, Andrew I. Schoenholtz, and Philip G. Schrag, "Refugee Roulette: Disparities in Asylum Adjudication," Stanford Law Review (2007); Refugee Roulette: Disparities in Asylum Adjudication and Proposals for Reform (NYU Press, 2009). 2. Nicole Maestas, Kathleen J. Mullen, and Alexander Strand, "Does Disability Insurance Receipt Discourage Work? Using Examiner Assignment to Estimate Causal Effects of SSDI Receipt," American Economic Review (2013). 3. Daniel L. Chen, Tobias J. Moskowitz, and Kelly Shue, "Decision Making Under the Gambler's Fallacy: Evidence from Asylum Judges, Loan Officers, and Baseball Umpires," Quarterly Journal of Economics (2016). 4. Shai Danziger, Jonathan Levav, and Liora Avnaim-Pesso, "Extraneous Factors in Judicial Decisions," Proceedings of the National Academy of Sciences (2011); Keren Weinshall-Margel and John Shapard, "Overlooked Factors in the Analysis of Parole Decisions," Proceedings of the National Academy of Sciences (2011); Andreas Glöckner, "The Irrational Hungry Judge Effect Revisited: Simulations Reveal That the Magnitude of the Effect Is Overestimated," Judgment and Decision Making (2016). 5. Anthony Heyes and Soodeh Saberian, "Temperature and Decisions: Evidence from 207,000 Court Cases," American Economic Journal: Applied Economics (2019); Holger Spamann, "Comment on 'Temperature and Decisions: Evidence from 207,000 Court Cases,'" American Economic Journal: Applied Economics (2022). 6. Ozkan Eren and Naci Mocan, "Emotional Judges and Unlucky Juveniles," American Economic Journal: Applied Economics (2018). 7. Chris Guthrie, Jeffrey J. Rachlinski, and Andrew J. Wistrich, "Inside the Judicial Mind," Cornell Law Review (2001); Jeffrey J. Rachlinski, Sheri Lynn Johnson, Andrew J. Wistrich, and Chris Guthrie, "Does Unconscious Racial Bias Affect Trial Judges?," Notre Dame Law Review (2009). 8. Birte Englich, Thomas Mussweiler, and Fritz Strack, "Playing Dice with Criminal Sentences: The Influence of Irrelevant Anchors on Experts' Judicial Decision Making," Personality and Social Psychology Bulletin (2006). Chapter 9 1. George L. Priest and Benjamin Klein, "The Selection of Disputes for Litigation," Journal of Legal Studies (1984); Daniel Kessler, Thomas Meites, and Geoffrey Miller, "Explaining Deviations from the Fifty-Percent Rule: A Multimodal Approach to the Selection of Cases for Litigation," Journal of Legal Studies (1996). 1. Marc Galanter, "Why the 'Haves' Come Out Ahead: Speculations on the Limits of Legal Change," Law & Society Review (1974). 2. Donald R. Songer and Reginald S. Sheehan, "Who Wins on Appeal? Upperdogs and Underdogs in the United States Courts of Appeals," American Journal of Political Science (1992); Donald R. Songer, Reginald S. Sheehan, and Susan Brodie Haire, "Do the 'Haves' Come Out Ahead Over Time? Applying Galanter's Framework to Decisions of the U.S. Courts of Appeals, 1925–1988," Law & Society Review (1999). 3. Kevin T. McGuire, "Repeat Players in the Supreme Court: The Role of Experienced Lawyers in Litigation Success," Journal of Politics (1995); Richard J. Lazarus, "Advocacy Matters Before and Within the Supreme Court: Transforming the Court by Transforming the Bar," Georgetown Law Journal (2008). 4. Timothy R. Johnson, Paul J. Wahlbeck, and James F. Spriggs II, "The Influence of Oral Arguments on the U.S. Supreme Court," American Political Science Review (2006). 5. Lee Epstein, William M. Landes, and Richard A. Posner, "Inferring the Winning Party in the Supreme Court from the Pattern of Questioning at Oral Argument," Journal of Legal Studies (2010); Timothy R. Johnson, Ryan C. Black, Jerry Goldman, and Sarah A. Treul, "Inquiring Minds Want to Know: Do Justices Tip Their Hands with Questions at Oral Argument in the U.S. Supreme Court?," Washington University Journal of Law and Policy (2009). 6. Gregory A. Caldeira and John R. Wright, "Organized Interests and Agenda Setting in the U.S. Supreme Court," American Political Science Review (1988); Joseph D. Kearney and Thomas W. Merrill, "The Influence of Amicus Curiae Briefs on the Supreme Court," University of Pennsylvania Law Review (2000); Paul M. Collins, Jr., Friends of the Supreme Court: Interest Groups and Judicial Decision Making (Oxford University Press, 2008). 7. TC Heartland LLC v. Kraft Foods Group Brands LLC, 581 U.S. 258 (2017). 8. Kevin M. Clermont and Theodore Eisenberg, "Trial by Jury or Judge: Transcending Empiricism," Cornell Law Review (1992). 9. Marc Galanter, "The Vanishing Trial: An Examination of Trials and Related Matters in Federal and State Courts," Journal of Empirical Legal Studies (2004); John Gramlich, "Only 2% of Federal Criminal Defendants Go to Trial, and Most Who Do Are Found Guilty," Pew Research Center, June 11, 2019. 10. Randall L. Kiser, Martin A. Asher, and Blakeley B. McShane, "Let's Not Make a Deal: An Empirical Study of Decision Making in Unsuccessful Settlement Negotiations," Journal of Empirical Legal Studies (2008). 11. Kevin M. Clermont and Theodore Eisenberg, "Appeal from Jury or Judge Trial: Defendants' Advantage," American Law and Economics Review (2001); Kevin M. Clermont and Stewart J. Schwab, "How Employment Discrimination Plaintiffs Fare in Federal Court," Journal of Empirical Legal Studies (2004). 12. Alexander J. S. Colvin, "An Empirical Study of Employment Arbitration: Case Outcomes and Processes," Journal of Empirical Legal Studies (2011). Conclusion 1. Theodore W. Ruger, Pauline T. Kim, Andrew D. Martin, and Kevin M. Quinn, "The Supreme Court Forecasting Project: Legal and Political Science Approaches to Predicting Supreme Court Decisionmaking," Columbia Law Review (2004). 1. Daniel Martin Katz, Michael J. Bommarito II, and Josh Blackman, "A General Approach for Predicting the Behavior of the Supreme Court of the United States," PLOS ONE (2017). 2. Nikolaos Aletras, Dimitrios Tsarapatsanis, Daniel Preoţiuc-Pietro, and Vasileios Lampos, "Predicting Judicial Decisions of the European Court of Human Rights: A Natural Language Processing Perspective," PeerJ Computer Science (2016). 3. France, Loi n° 2019-222 du 23 mars 2019 de programmation 2018–2022 et de réforme pour la justice, art. 33. Further Reading Lee Epstein and Andrew D. Martin, An Introduction to Empirical Legal Research (Oxford University Press, 2014).** The clearest introduction for lawyers and law students to research design, measurement, and statistical inference, written by two of the field's leading figures and illustrated throughout with legal examples. Lee Epstein, William M. Landes, and Richard A. Posner, The Behavior of Federal Judges (Harvard University Press, 2013).** A comprehensive economic and statistical study of judging at every level of the federal system, from district courts to the Supreme Court, including ideology, dissent, and the effects of career incentives. Jeffrey A. Segal and Harold J. Spaeth, The Supreme Court and the Attitudinal Model Revisited (Cambridge University Press, 2002).** The strongest statement of the claim that Justices' votes reflect their policy preferences, and the book against which most later work defines itself. Cass R. Sunstein, David Schkade, Lisa M. Ellman, and Andres Sawicki, Are Judges Political? An Empirical Analysis of the Federal Judiciary (Brookings Institution Press, 2006).** An accessible account of ideological voting and panel effects in the federal courts of appeals across many areas of law. Michael A. Bailey and Forrest Maltzman, The Constrained Court (Princeton University Press, 2011).** A careful attempt to measure how legal values such as precedent and deference shape Supreme Court votes alongside policy preferences. Neal Devins and Lawrence Baum, The Company They Keep (Oxford University Press, 2019).** Explains how partisan and ideological divisions came to coincide on the modern Supreme Court, emphasizing the elite networks in which Justices live. Adam Bonica and Maya Sen, The Judicial Tug of War (Cambridge University Press, 2020).** Uses campaign contribution data to map the ideology of American lawyers and judges and to explain the gap between the bar and the bench. Kate Stith and José A. Cabranes, Fear of Judging (University of Chicago Press, 1998).** A critical history of the federal Sentencing Guidelines by a scholar and a federal judge, essential background for the disparity debates. Jaya Ramji-Nogales, Andrew I. Schoenholtz, and Philip G. Schrag, Refugee Roulette (NYU Press, 2009).** The book-length version of the landmark study of variation in asylum adjudication, with proposals for reform. Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein, Noise (Little, Brown Spark, 2021).** A general account of unwanted variability in professional judgment, with extended discussion of judges and sentencing. United States Sentencing Commission, 2023 Demographic Differences in Federal Sentencing (2023), and the Commission's Interactive Data Analyzer.** The Commission's most recent analysis of sentencing disparities and a free tool for exploring federal sentencing data directly. The Supreme Court Database and the Martin-Quinn Scores. The two most widely used public resources for studying the Supreme Court, each available online with documentation explaining its coding rules and methods. Hashtags: #EmpiricalLegalStudies #JudicialBehavior #JudicialDecisionMaking #LegalEmpiricism #LegalResearch #QuantitativeLegalResearch #DataDrivenLaw #LegalDataAnalytics #JudicialAnalytics #CourtAnalytics #LawAndStatistics #LawAndEconomics #LegalRealism #JudicialPolitics #JudicialIdeology #JudicialIndependence #JudicialDiscretion #JudicialBias #JudicialImpartiality #SupremeCourtResearch #SupremeCourtDatabase #FederalCourts #AppellateCourts #JudicialVoting #AttitudinalModel #JudicialPanelEffects #JudicialDiversity #IdeologicalVoting #MartinQuinnScores #JudicialIdeologyMeasurement #SentencingDisparities #RacialDisparities #CriminalJusticeResearch #FederalSentencing #SentencingGuidelines #CausalInference #StatisticalAnalysis #RegressionAnalysis #SelectionBias #NaturalExperiments #InstrumentalVariables #RandomAssignment #CourtData #LitigationAnalytics #LegalPrediction #JudicialOutcomePrediction #EmpiricalEvidence #RuleOfLaw #EvidenceBasedJustice #ComputationalLegalStudies
- Diffusion Models (Mathematical Foundations, Score Matching, and Generation)
Download the Book (PDF): Introduction Every image produced by a modern diffusion model begins as a field of pure static. A grid of numbers is drawn independently from a Gaussian distribution, one number per pixel or per latent coordinate, and a neural network is then asked, over and over, a single modest question: given this noisy thing and a rough idea of how noisy it is, which direction would make it slightly more plausible? Twenty, fifty or a thousand small steps later the static has become a photograph of a harbour at dusk, a few seconds of a cello sonata, or a short clip of a dog running through surf. Nothing in the network's architecture knows about harbours, cellos or dogs in the way a graphics engine knows about polygons. What it knows is a vector field: at every point in a very high-dimensional space, and at every level of corruption, it has learned an arrow pointing toward regions where real data is more dense. That vector field has a name. It is the score of the noise-perturbed data distribution, the gradient of its log-density with respect to the data. The argument of this booklet is that the score is the single mathematical object around which the whole subject is organised. Training a diffusion model is estimating the score at every noise level. Generating a sample is integrating a differential equation, stochastic or deterministic, whose drift is built from that score. Noise schedules are choices about which noise levels to visit and how to weight them. Samplers are numerical methods for the same equation. Classifier-free guidance is arithmetic on scores. Latent diffusion is the observation that the score need not be learned in pixel space. The practical systems that now generate high-fidelity images, audio and video differ in thousands of engineering details, but they agree on this skeleton, and a reader who holds the skeleton firmly can read almost any paper in the field and know where its contribution sits. The subject is often taught the other way round. A newcomer meets the 2020 paper by Jonathan Ho, Ajay Jain and Pieter Abbeel on denoising diffusion probabilistic models, learns that a network is trained to predict the noise that was added to an image, and memorises a sampling loop with Greek letters in it. The loop works, and the loss is simple, but the reasons are hidden. Why should predicting noise produce a generative model? Why does the sampler add fresh noise at every step, and why do some later samplers stop doing so without harm? Why does a schedule that was fine at 32 by 32 pixels become poor at 1024 by 1024? Why does turning up the guidance scale make images sharper and then garish? Each of these questions has a clean answer once the score is put at the centre, and each answer is a small piece of probability theory or numerical analysis rather than a trick. What the reader needs, and what the book supplies The mathematics here is that of a strong undergraduate course: multivariable calculus, the Gaussian distribution and its algebra, conditional probability and Bayes' rule, and some comfort with the idea of a differential equation. Stochastic differential equations appear in the middle chapters, but they are introduced from scratch and used for what they say rather than for their measure-theoretic foundations. Where a result requires a heavy proof, such as the existence of a reverse-time diffusion, the book states it precisely, explains why it is plausible, and shows what it buys. Notation is kept consistent throughout. A clean data point is written x₀, and a noisy version at time or noise level t is written xₜ. The Gaussian noise drawn to corrupt it is ε. The distribution of clean data is p_data, and the distribution of noisy data at level t is pₜ. The score of that distribution, the vector ∇ log pₜ(x), is written s(x, t) when it is the true score and s_θ(x, t) when it is a neural network's estimate with parameters θ. The noise level is sometimes described by a time t running from 0 to 1, sometimes by a standard deviation σ, and sometimes by a signal-to-noise ratio. Part of the work of the book is to show that these are interchangeable descriptions of the same path, and that choosing among them is a matter of convenience and numerical stability rather than of substance. The route through the subject The first chapter builds the forward process: the deliberately simple procedure that destroys data by adding Gaussian noise, and the closed-form expressions that let a model jump to any level of corruption in one step. The second chapter introduces the score and explains why learning it is tractable when learning the density itself is not; this is where denoising score matching and Tweedie's formula appear, and where the equivalence between predicting noise, predicting the clean image and predicting the score becomes visible. The third chapter turns the score into samples through Langevin dynamics, and shows why a single noise level fails and an annealed ladder of noise levels succeeds. The fourth chapter returns to the discrete-time model of Ho and colleagues and derives its training objective as a variational bound, which explains where the simple noise-prediction loss came from and what it quietly throws away. The fifth chapter moves to continuous time, where the forward process becomes a stochastic differential equation, the reverse process becomes another one, and a deterministic probability flow ODE emerges that carries the same marginal distributions. With that unified view in hand, the sixth chapter treats noise schedules and network parameterisation, the choices that most affect quality in practice and that are most often handled by folklore. The seventh chapter treats sampling as numerical integration and covers the solvers and distillation methods that cut generation from a thousand network evaluations to a handful. The eighth chapter is about control: how a model trained on images and captions can be made to follow the caption more closely through classifier guidance and classifier-free guidance, what those procedures do to the distribution being sampled, and why they fail in characteristic ways at high strength. The ninth chapter covers the networks and representations that make the mathematics practical at scale, from the U-Net to the diffusion transformer and from pixel space to learned latent spaces. The tenth chapter follows the same machinery into audio and video, where the data has different structure, different notions of fidelity and, for video, a temporal dimension that makes coherence the central difficulty. The eleventh chapter places diffusion beside its closest relatives, flow matching and rectified flow, which have become the training recipe of choice for several recent large systems, and shows that the relationship is closer to a change of coordinates than a change of paradigm. The twelfth chapter confronts a puzzle that the rest of the theory sets aside: a network that minimised its training objective perfectly would only reproduce its training data, so the novelty of what diffusion models generate must come from what the network fails to learn, and the same analysis explains when these systems copy. Why the foundations matter now It would be possible to treat diffusion models purely as engineering artefacts, judged by benchmarks and demos. That approach ages quickly. The specific architectures that dominated in 2022 were largely displaced within two years; the schedules that were standard in 2021 were shown to be flawed in 2023; the field's preferred training objective has shifted from noise prediction to velocity prediction to flow matching. What has not shifted is the underlying picture of a family of distributions connected by noise, a learned score field over that family, and a numerical procedure that walks backward through it. Readers who understand that picture can evaluate new methods on their merits: they can ask whether a proposal changes the target of learning, the weighting across noise levels, the integration scheme, or the distribution being sampled, and they can predict what the change will cost. There is also a matter of honesty about what these systems are. A diffusion model does not retrieve images, and it does not paste fragments. It samples from an approximation to a probability distribution defined by its training data and conditioning, and the quality of that approximation varies across the space in ways that the mathematics makes predictable. Guidance trades diversity for adherence. Few-step samplers trade fidelity for speed. Latent compression trades fine detail for tractability. Knowing the mathematics is knowing where these trades are made and who is making them. The chapters that follow aim to give the reader that knowledge in a form that lasts: the derivations that matter, carried through in words and a small number of displayed equations, with the intuition kept in view at each step. Chapter 1. Destroying Data on Purpose: The Forward Process The strangest feature of a diffusion model, to anyone meeting it for the first time, is that its design begins with a procedure for ruining the data. Before the network learns anything, the modeller specifies, by hand and with no learnable parameters, a way to corrupt every training example gradually until nothing recognisable remains. This forward process is not a preprocessing step or a regulariser. It defines the whole family of distributions that the model will later learn to traverse, and its properties determine what the network is asked to learn, how hard each part of that task is, and how sampling will eventually proceed. Why corruption, and why Gaussian The difficulty a generative model must overcome is that natural data occupies a tiny, intricately shaped region of a vast space. A 256 by 256 colour image is a point in a space of roughly 196,000 dimensions. Almost every point in that space looks like static. The set of points that look like photographs is thin, curved and full of structure at every scale, and nobody can write down its shape. A model that tries to learn the density of photographs directly faces two problems at once. It must represent a function that is essentially zero almost everywhere and sharply peaked on a complicated set, and it must somehow normalise that function so that it integrates to one over the whole space, an integral nobody can compute. Adding noise attacks both problems. Take any distribution, however intricate, and convolve it with a Gaussian of standard deviation σ: replace each data point with a fuzzy cloud of points around it. The result is smooth and positive everywhere, and as σ grows it becomes progressively simpler, until at very large σ it is indistinguishable from a broad Gaussian centred on the data's mean. Between the data distribution at one end and a plain Gaussian at the other lies a continuum of intermediate distributions, each a blurred version of the last. The forward process is a precise description of that continuum. Gaussian noise is chosen for reasons that compound. The sum of independent Gaussians is Gaussian, so many small noising steps collapse into a single closed-form jump. A Gaussian is determined by its mean and covariance, so every conditional distribution in the forward process can be written in one line. The gradient of a Gaussian log-density is linear in its argument, which makes the training targets simple. And the Gaussian is the distribution of maximum entropy for a given variance, so the endpoint of the process carries no information beyond its scale. Other corruptions have been explored, including blurring, masking and discrete token replacement for text, and some work well, but the Gaussian case is where the mathematics is cleanest and where nearly all high-fidelity image, audio and video systems sit. The discrete Markov chain The formulation that brought diffusion models to prominence, from Ho, Jain and Abbeel in 2020 and building on the 2015 paper of Jascha Sohl-Dickstein and colleagues, defines the forward process as a Markov chain of T steps. At each step a small amount of noise is added and the signal is slightly shrunk: xₜ = √(1 − βₜ) · xₜ₋₁ + √βₜ · εₜ, with εₜ drawn from a standard Gaussian. The numbers β₁, β₂, …, β_T form the variance schedule. Each is small and positive. In the original paper T was 1,000 and the βs rose linearly from 0.0001 to 0.02. The shrinking factor √(1 − βₜ) is the detail that matters most. If the chain merely added noise, the variance of xₜ would grow without bound. By scaling the previous state down at the same time as noise is added, the chain keeps the total variance fixed: if xₜ₋₁ has unit variance per coordinate, then xₜ has variance (1 − βₜ) + βₜ = 1. This is why the formulation is called variance preserving. Data is usually normalised to lie roughly between minus one and one, so the process keeps every intermediate state on roughly the same scale, which is convenient for a neural network that must handle all of them. Because each step is a linear operation plus independent Gaussian noise, the composition of many steps is again a linear operation plus Gaussian noise. Define αₜ = 1 − βₜ and let ᾱₜ be the running product α₁α₂…αₜ. Then a short induction shows that the state at step t, given the clean data x₀, is xₜ = √ᾱₜ · x₀ + √(1 − ᾱₜ) · ε, with ε drawn from a standard Gaussian. This closed form is the practical heart of the forward process. To train at step t, one does not simulate t noising steps; one draws a single ε and mixes it with x₀ in the right proportions. The quantity ᾱₜ starts near one, when xₜ is nearly clean, and falls toward zero, when xₜ is nearly pure noise. Under the original linear schedule ᾱ at step 1,000 is about 0.00004, small enough that the endpoint is, for most purposes, a standard Gaussian. It is worth dwelling on the two coefficients. The coefficient on x₀, √ᾱₜ, measures how much signal survives. The coefficient on ε, √(1 − ᾱₜ), measures how much noise has been mixed in. Their squares sum to one. The forward process can therefore be pictured as rotating, step by step, from the data axis to the noise axis on a quarter-circle, and much later work makes this rotation explicit by writing the coefficients as the cosine and sine of an angle. Signal-to-noise ratio as the master variable The index t is an arbitrary label. What the network actually experiences at step t is a mixture of signal and noise in a particular proportion, and the cleanest way to describe that proportion is the signal-to-noise ratio: SNR(t) = ᾱₜ / (1 − ᾱₜ). At small t the SNR is large, and the noisy image is visibly the original with a little grain. At large t the SNR is tiny and the image is static with a faint ghost. Diederik Kingma, Tim Salimans, Ben Poole and Jonathan Ho, in their 2021 work on variational diffusion models, argued that the SNR, or more conveniently its logarithm, is the natural coordinate for the whole process. Two forward processes with different step counts and different β sequences, but which visit the same range of log-SNR values, define the same family of noisy distributions; they differ only in how the steps are spaced along that family. Much of the folklore around schedules dissolves once one plots everything against log-SNR rather than against step index. The log-SNR view also clarifies what is being learned at each level. At very high SNR, the network sees a slightly grainy image and must remove fine grain; the task is local and resembles classical denoising. At very low SNR, the network sees almost pure noise and must guess the coarse layout of an image, its dominant colours and composition, from faint correlations. In the middle range, where log-SNR is near zero and signal and noise are of comparable size, the network must make decisions about object identity, shape and arrangement. These are different tasks, and how a training procedure distributes its effort across them is one of the most consequential choices in the field. Chapter 6 returns to this in detail. The variance-exploding alternative A second family of forward processes, developed by Yang Song and Stefano Ermon in their 2019 work on noise-conditional score networks, does not shrink the signal at all. It simply adds noise of increasing standard deviation: x_σ = x₀ + σ · ε. Here σ ranges from a small value, around 0.01 for normalised images, up to a large value comparable to the largest distances between training examples, which for some image datasets meant values of fifty or more. Because the signal is never scaled down, the variance of x_σ grows as σ² and the process is called variance exploding. At the top of the ladder, the data's contribution is negligible relative to the noise, and the distribution is effectively a very wide Gaussian. The variance-exploding and variance-preserving processes look different but are closely related. Take a variance-preserving state xₜ = √ᾱₜ x₀ + √(1 − ᾱₜ) ε and divide it by √ᾱₜ. The result is x₀ + σ ε with σ = √((1 − ᾱₜ)/ᾱₜ), which is one over the square root of the SNR. So every variance-preserving state is a rescaled variance-exploding state, and the two processes visit the same family of distributions up to a known scale factor at each level. Tero Karras and colleagues at NVIDIA made this observation central to their 2022 paper on the design space of diffusion models, often called EDM, in which they parameterise everything by σ and treat the scaling as a separate, explicit choice. Their view, now widely adopted, is that the scale factor is a matter of network input conditioning rather than a fundamental property of the model. Continuous time as the limit The discrete chain with T steps is one discretisation of an underlying continuous process. If one lets T grow while each βₜ shrinks proportionally, the chain converges to the solution of a stochastic differential equation, in which the state drifts gently toward zero while being buffeted by Brownian motion. Chapter 5 develops this view properly. For now the point is that the step count T has no deep significance. The forward process is a continuous path through distribution space, from data to noise, and any finite T is a choice of how finely to sample that path during training. Modern systems typically train with a continuous noise level drawn from some distribution at each step, and decide on the number of sampling steps separately and much later. Properties that make the forward process useful Four properties of the forward process carry the rest of the subject, and it is worth stating them plainly before moving on. First, it is fixed. The forward process has no learnable parameters. The model never has to discover how data was corrupted; the corruption is known exactly and can be applied at any level in closed form. This is what makes training a diffusion model a supervised regression problem rather than an adversarial game or a search over latent codes. Second, it is gradual. Adjacent noise levels define distributions that are close to one another. This closeness is what allows the reverse process to be broken into many small steps, each of which can be approximated by a simple distribution. A single leap from noise to data would require the model to represent the full complexity of the data distribution in one conditional; a thousand small steps each require only a gentle refinement. Sohl-Dickstein and colleagues relied on a classical result for diffusion processes: when the forward steps are small enough, the reverse steps are approximately Gaussian too. That observation licenses the use of Gaussian conditionals in the learned reverse process, and much of Chapter 4 is built upon it. Third, its endpoint is known. The distribution at the final noise level is, to a good approximation, a fixed Gaussian independent of the data. Generation can therefore begin by sampling from that Gaussian, which is trivial. Any mismatch between the true endpoint and the assumed Gaussian becomes an error in generation, and Chapter 6 discusses a case, the failure to reach zero signal-to-noise ratio, where that mismatch had visible consequences in widely used systems. Fourth, it has tractable posteriors. Because the forward process is a linear Gaussian system, the distribution of an earlier state given both a later state and the clean data, written q(xₜ₋₁ | xₜ, x₀), is itself Gaussian with a mean and variance that can be written in closed form. The mean is a weighted average of xₜ and x₀ with weights fixed by the schedule. This posterior is the ideal target for a single reverse step, if only one knew x₀. The whole difficulty of generation lies in that conditional: at sampling time x₀ is exactly what one does not know. A diffusion model's network can be understood as a machine for estimating x₀, or equivalently the noise ε, or equivalently the score, from the noisy state alone. The next chapter shows why those three estimates are the same thing in disguise. What the forward process does to structure A useful exercise is to watch, conceptually, what the forward process removes and in what order. Natural images have power spectra that fall off steeply with spatial frequency: most of their energy is in broad, slowly varying structure, and relatively little is in fine detail. White Gaussian noise, by contrast, has equal power at all frequencies. When a small amount of noise is added, it immediately swamps the weak high-frequency content of the image while leaving the strong low-frequency content almost untouched. As the noise grows, it overwhelms progressively lower frequencies. The last things to disappear are the overall colour balance and the coarsest layout. Sander Dieleman and others have described this as diffusion being approximately a form of spectral autoregression: the forward process erases frequency bands from fine to coarse, so the reverse process must generate them from coarse to fine. This explains a great deal of observed behaviour. Early sampling steps settle composition and colour. Middle steps settle shapes and objects. Late steps add texture and edges. It also explains why higher-resolution images need different schedules from lower-resolution ones. At higher resolution, an image of the same scene has more pixels per object, so a given amount of per-pixel noise leaves more of the low-frequency structure intact, since averaging over more pixels recovers the signal more accurately. A schedule tuned for small images therefore destroys large images too slowly at the high-noise end, and the model spends too little effort on composition. Emiel Hoogeboom, Jonathan Heek and Tim Salimans made this argument in their 2023 work on simple diffusion, and later high-resolution systems shift their schedules toward higher noise for the same reason. Audio has an analogous structure, with energy concentrated in particular frequency bands and temporal envelopes, and video adds a temporal axis along which content is highly redundant from frame to frame. In each domain the same lesson holds: the forward process is not neutral with respect to structure. It destroys the weakest components first, and the modeller's choice of representation and schedule determines which components those are. The forward process as a definition of the learning problem The forward process converts the unsolvable problem of modelling an intricate distribution in high dimensions into a sequence of solvable ones. At each noise level, there is a well-defined distribution pₜ of noisy data. Each pₜ is smooth and everywhere positive. Each is related to its neighbours by a small, known corruption. The first is the data and the last is a Gaussian. The task of a diffusion model is to learn enough about every pₜ to walk the sequence backward. What exactly does one need to learn about each pₜ? Not its density, which would require normalisation. Not a sampler for it directly, which would beg the question. What suffices is its score: the direction in which the log-density increases fastest, at every point. That is the subject of the next chapter. Chapter 2. The Score: Learning Gradients Instead of Densities A probability density over images, if one could write it down, would answer every question a generative model is asked. It would say how plausible any given image is, and with enough computation it could be sampled. The trouble is that densities must integrate to one, and enforcing that constraint over a space with hundreds of thousands of dimensions is hopeless for any flexible function. Diffusion models avoid the problem by never learning the density. They learn its gradient instead, and the gradient turns out to carry everything needed for sampling while shedding the constraint that made the density intractable. The normalisation problem and how the score escapes it Suppose one models a distribution with an energy function: a neural network E_θ(x) assigns a real number to every point, and the density is taken to be proportional to exp(−E_θ(x)). Low energy means high probability. To turn this into a genuine density one must divide by the normalising constant Z_θ, the integral of exp(−E_θ(x)) over the whole space. Z_θ depends on the parameters, so maximum-likelihood training needs its gradient, and that gradient is an expectation over the model's own distribution, which can only be estimated by sampling from the model, which is precisely what one does not yet know how to do. Energy-based models were trained this way for decades with approximate sampling, and the approximations were persistently troublesome. Now take the logarithm of the density and differentiate with respect to x rather than θ: ∇ₓ log p_θ(x) = −∇ₓ E_θ(x) − ∇ₓ log Z_θ = −∇ₓ E_θ(x). The normalising constant does not depend on x, so its gradient with respect to x is zero. The gradient of the log-density with respect to the data, which statisticians call the score, is completely free of the normalisation problem. It is a vector field over data space: at each point it is an arrow pointing in the direction of steepest increase in log-probability, with length reflecting how steeply probability changes. Near a mode the arrows are short and point inward. Far from the data they are long and point back toward it. Knowing the score everywhere is, up to a constant, the same as knowing the log-density everywhere, since one can integrate the gradient along any path. So nothing is lost in principle by learning the score instead of the density. And in practice a great deal is gained: the score can be modelled by any neural network that outputs a vector of the same shape as its input, with no architectural constraints and no normalisation. A small terminological note avoids later confusion. In classical statistics the word score refers to the gradient of the log-likelihood with respect to the parameters, as in Fisher's scoring method. In the diffusion literature, following Aapo Hyvärinen's usage, the score is the gradient with respect to the data. The two are different objects, and only the second appears in this book. Score matching If the score is the right thing to learn, how is it learned? The obvious objective is to make the network s_θ(x) close to the true score of the data distribution, averaged over data: J(θ) = ½ · E over x from p_data of ‖ s_θ(x) − ∇ₓ log p_data(x) ‖². This is called the explicit score matching objective, and it cannot be computed as written, because the true score of the data distribution is unknown. All one has is samples. Hyvärinen showed in 2005 that, under mild regularity conditions, integration by parts transforms this objective into an equivalent one that does not involve the true score: J(θ) = E over x from p_data of [ trace(∇ₓ s_θ(x)) + ½ ‖ s_θ(x) ‖² ] + constant. The first term is the trace of the Jacobian of the network's output, the sum of the derivatives of each output coordinate with respect to the corresponding input coordinate. The second term penalises large scores. The objective is minimised when the network's field is the true score. This is an elegant result, and it makes score matching a consistent estimator, but the trace of the Jacobian is expensive. For an input with d dimensions, computing it exactly requires d backward passes through the network. For images this is prohibitive. Sliced score matching, introduced by Song, Sahil Garg, Jiaxin Shi and Ermon in 2019, replaces the trace with random projections and reduces the cost substantially, but a cleaner solution had already been found. Denoising score matching In 2011 Pascal Vincent published a short paper connecting score matching to denoising autoencoders. The construction is this. Rather than matching the score of the data distribution, match the score of a noise-perturbed version of it. Let q_σ(x̃ | x) be the Gaussian that adds noise of standard deviation σ to a clean point x, and let q_σ(x̃) be the resulting distribution of noisy points, obtained by averaging over clean data. Vincent showed that matching the score of q_σ(x̃) is equivalent, up to a constant independent of θ, to matching the score of the conditional q_σ(x̃ | x), averaged over pairs of clean and noisy points: J_DSM(θ) = ½ · E over x, x̃ of ‖ s_θ(x̃) − ∇ log q_σ(x̃ | x) ‖². The crucial advantage is that the conditional score is known exactly. For Gaussian noise, ∇ log q_σ(x̃ | x) = −(x̃ − x) / σ² = −ε / σ, where ε is the standard Gaussian noise that was drawn to produce x̃ = x + σε. The training target is simply minus the noise, divided by σ. So denoising score matching reduces to a regression problem: take a clean example, add noise, and train the network to predict, from the noisy example, a vector pointing back along the noise toward the clean one. Why does this work, given that any particular noisy point x̃ could have come from many different clean points? The answer is that a squared-error regression, minimised over a population, learns the conditional expectation of its target. The network cannot know which clean point produced a given noisy one; it learns the average target over all clean points that could have produced it, weighted by how likely each is. That average turns out to be exactly the score of the noisy marginal distribution. The randomness of the individual targets averages out, and what remains is the field one wanted. It is worth seeing this explicitly. The marginal density of noisy points is an average over clean points of Gaussian kernels: q_σ(x̃) = ∫ p_data(x) · q_σ(x̃ | x) dx. Differentiate the log of this with respect to x̃. The derivative of a log is the derivative over the function, and the derivative of the integral passes inside. What emerges is the posterior-weighted average of the conditional scores: ∇ log q_σ(x̃) = E over x given x̃ of [ ∇ log q_σ(x̃ | x) ] = E [ −(x̃ − x) / σ² | x̃ ]. The marginal score at a noisy point equals the average, over clean points weighted by how plausibly they produced that noisy point, of the arrow pointing from the noisy point back to the clean one. A network trained by squared error to predict those arrows will, at its optimum, output their average. That is the marginal score. Tweedie's formula and the meaning of denoising Rearranging the last equation gives a striking identity. Since the expectation of x̃ given x̃ is just x̃, E[ x | x̃ ] = x̃ + σ² · ∇ log q_σ(x̃). This is Tweedie's formula, named after the statistician Maurice Tweedie and brought to wide attention by Herbert Robbins in the 1950s and by Bradley Efron in a 2011 paper. It says that the optimal denoiser in the squared-error sense, the posterior mean of the clean signal given the noisy observation, is obtained by starting at the noisy observation and stepping along the score of the noisy distribution, with step size σ². Denoising and score estimation are therefore the same problem. A network that denoises optimally knows the score, and a network that knows the score can denoise optimally. This identity is the key that unlocks the various parameterisations used in practice. In the variance-preserving setting the noisy state is xₜ = √ᾱₜ x₀ + √(1 − ᾱₜ) ε. A network can be trained to predict any of three quantities from xₜ and t. It can predict the clean data x₀, giving an estimate x̂₀. It can predict the noise ε, giving an estimate ε̂. Or it can predict the score directly. Because xₜ is known, any one of x̂₀ and ε̂ determines the other through the forward equation, and Tweedie's formula ties both to the score: s(xₜ, t) = −ε̂ / √(1 − ᾱₜ), and x̂₀ = (xₜ − √(1 − ᾱₜ) · ε̂) / √ᾱₜ. Ho and colleagues trained their network to predict ε, with a plain mean-squared error between the network output and the true noise. This is denoising score matching, rescaled so that the target has unit variance at every noise level. That rescaling turned out to matter a great deal for optimisation, and it is one reason the 2020 paper produced dramatically better images than earlier score-based work, even though the underlying estimand was the same. Why the clean distribution's score is not enough A natural question is why one bothers with noise at all. If the goal is a sampler for p_data, why not learn the score of p_data directly, perhaps with sliced score matching, and use it? Song and Ermon gave three reasons in 2019, and they remain the clearest account. The first is the manifold problem. Natural images are believed to concentrate near a low-dimensional manifold within pixel space. Off that manifold the data density is zero or nearly so, and its logarithm is undefined or wildly negative. The score is undefined there too, and score matching provides no training signal in regions where there is no data. A network trained on clean data has seen nothing off the manifold and has no reason to output sensible arrows there. But a sampler starting from random noise begins exactly there. The second reason is the low-density problem. Even when the data distribution has full support, regions between modes have low probability and receive few training samples. Score estimates there are inaccurate. A sampler that must cross such regions, for instance to move from one cluster of images to another, relies on the least reliable part of the learned field. The third reason is subtler and concerns mixture weights. Suppose the data is a mixture of two well-separated clusters, one containing ninety per cent of the mass and the other ten. Within each cluster, the score depends only on the shape of that cluster; the mixing weight contributes a constant to the log-density locally and disappears under differentiation. A sampler driven by the clean score, started in either cluster, will stay there and has no way to discover the relative weights. Unless it can cross between clusters, which requires passing through the low-density gap where the score is poorly learned, it will produce clusters in proportions determined by where it started rather than by the data. Adding noise addresses all three. Noise of moderate size smears the data off the manifold, so the noisy density is positive everywhere and its score is defined everywhere. Noise of large size fills in the gaps between modes, so training samples populate the regions a sampler must cross. And at large noise levels the clusters merge into a single blob whose shape reflects the mixture weights, so the score at high noise directs samples toward each cluster in the right proportions. The price is that the noisy distribution is not the data distribution. The resolution, which Song and Ermon proposed and which every subsequent diffusion model inherits, is to learn the score at many noise levels at once, from very large to very small, and to sample by descending through them. The large-noise scores handle global structure and mixture weights; the small-noise scores refine the result into something close to clean data. The noise-conditional score network Learning a separate network for each noise level would be wasteful, since adjacent levels have nearly identical scores. The standard approach conditions a single network on the noise level. The network receives the noisy input and a representation of σ or t, usually through a sinusoidal or learned embedding injected into each layer, and outputs the score estimate at that level. Song and Ermon called this a noise-conditional score network. The training loss averages denoising score matching over noise levels drawn from some distribution, with a weighting function λ(σ): L(θ) = E over σ, x₀, ε of [ λ(σ) · ‖ s_θ(x₀ + σε, σ) + ε/σ ‖² ]. The weighting is a free choice and a consequential one. Song and Ermon chose λ(σ) = σ², which makes each term proportional to ‖σ s_θ + ε‖², so the network effectively predicts −ε and each noise level contributes on a comparable scale. Ho and colleagues' noise-prediction loss is the same weighting expressed in variance-preserving coordinates. Other weightings emphasise different noise levels, and Chapters 4 and 6 show that some correspond to maximum likelihood while others favour perceptual quality. What the network is really learning It helps to state plainly what a trained diffusion network represents. At each noise level and each point in space, it outputs an estimate of the posterior mean of the clean data given that noisy point, or something affinely related to it. At high noise, where many clean images are consistent with the observation, that posterior mean is a blurry average: something like the mean image of the dataset, nudged slightly toward whatever coarse features the noise has not fully erased. At low noise, where few clean images are consistent, the posterior mean is sharp and close to a specific image. This is why directly outputting the network's x̂₀ prediction at high noise gives a blurry result, and why sampling must proceed gradually. The posterior mean is a single point that averages over all plausible completions; it is not itself a plausible completion. Sampling, as the next chapter shows, works by taking a small step toward the posterior mean, adding a little fresh randomness, and re-asking the question at a slightly lower noise level, where the posterior is narrower. The randomness is what selects one completion from the many; the score is what keeps the selection consistent with the data. It also explains a phenomenon practitioners notice early. If one inspects the network's predicted x̂₀ at successive sampling steps, the image seems to sharpen from a vague average into a definite picture, with commitments made progressively. The commitments are not made by the network alone; they are made by the interaction between the network's averaging and the sampler's noise. Strip out the noise entirely and use the network's prediction deterministically, and one still obtains sharp images, but by a different mechanism, which Chapter 5 explains through the probability flow ODE. The score, then, is the object a diffusion model learns, learned at all noise levels by a regression that is no harder to set up than training a denoiser. What remains is to explain how a vector field of scores becomes a stream of samples. That is the work of Langevin dynamics. Chapter 3. From Scores to Samples: Langevin Dynamics A vector field of scores tells you which way is uphill in probability. It does not, on its own, hand you a sample. If one simply followed the arrows, as in gradient ascent, every starting point would climb to the nearest mode and stop there, producing the single most probable image in each basin over and over. A generative model must produce samples spread across the distribution in the right proportions: common things often, rare things rarely, and every plausible variation in between. The device that turns a score into a sampler with the right proportions is Langevin dynamics, a procedure borrowed from statistical physics, and understanding it is the bridge between the learned network and the generated artefact. The physical picture In 1908 Paul Langevin wrote down an equation for a particle suspended in a fluid, pushed by a deterministic force and jostled by random collisions with the fluid's molecules. When the force is the gradient of a potential energy U(x) and the particle's motion is overdamped, meaning friction dominates inertia, the particle's position evolves as dx = −∇U(x) dt + √2 dW, where W is a Brownian motion, a continuous random walk whose increments over a short time dt are Gaussian with variance dt in each coordinate. The first term pulls the particle downhill in energy. The second term kicks it around at random. The remarkable fact about this equation is that, under broad conditions, the particle's position distribution converges over long times to a definite stationary distribution: the Boltzmann distribution, with density proportional to exp(−U(x)). Low-energy regions are visited often, high-energy regions rarely, in exactly the proportions the Boltzmann formula prescribes. The drift and the noise are in balance. The drift alone would collapse the particle into a minimum; the noise alone would diffuse it across all space; together they produce a distribution that reflects the energy landscape faithfully. Now set U(x) = −log p(x). Then exp(−U) is p itself, and −∇U is the score. Langevin dynamics driven by the score of p has p as its stationary distribution. One does not need to know p, only its score. A network that has learned the score can therefore, in principle, be turned into a sampler by simulating this equation for long enough from any starting point. The discrete algorithm Simulation means discretising time. With a small step size η, the update becomes x_{k+1} = x_k + η · s(x_k) + √(2η) · z_k, with z_k drawn from a standard Gaussian. Each iteration takes a step of size η along the score and adds Gaussian noise with variance 2η per coordinate. The particular ratio between the drift step and the noise variance is not arbitrary; it is what makes the stationary distribution correct. If the noise were smaller, the samples would concentrate too tightly around the modes, as if the distribution had been sharpened. If it were larger, they would spread too widely. The factor of two comes directly from the continuous equation and reflects the balance between the rate at which the drift contracts the distribution and the rate at which noise expands it. Two errors arise from discretisation. The first is that, with finite η, the stationary distribution of the discrete chain is not exactly p; it is a slightly distorted version whose distortion shrinks as η shrinks. A Metropolis–Hastings acceptance step can remove this bias, giving the Metropolis-adjusted Langevin algorithm, but it requires evaluating the density ratio and therefore the density, which a score model does not provide. Diffusion samplers accept the bias and keep η small. The second error is mixing time: the chain must run long enough to forget its starting point and explore the distribution. For well-conditioned distributions, a few hundred steps may suffice. For distributions with isolated modes separated by regions of low probability, mixing can take astronomically long, because crossing between modes requires the noise to push the particle uphill through the low-density gap, an event whose probability falls exponentially with the height of the barrier. Why single-level Langevin fails on real data Chapter 2 described three failures of the clean-data score: it is undefined off the data manifold, it is poorly estimated in low-density regions, and it is blind to mixture weights. Langevin dynamics turns each of these into a concrete sampling pathology. Start the chain from random noise. The starting point lies far from the data manifold, in a region where the network was never trained. Its score estimates there are essentially arbitrary, and the chain may wander for a long time, or drift somewhere useless, before it reaches a region where the arrows are reliable. Even once it reaches the data, if the data has several modes, the chain tends to settle into whichever is nearest and stay there, because crossing to another requires traversing a valley where the score is both small and badly estimated. And because the score within each mode is blind to that mode's total mass, the fraction of chains landing in each mode reflects the geometry of the starting distribution, not the data's proportions. Song and Ermon demonstrated exactly this on a simple mixture of two Gaussians with weights one-fifth and four-fifths: Langevin dynamics driven by the exact score produced the two components in roughly equal numbers, badly misrepresenting the mixture. Each of these problems is a problem of the sampler being asked to do something hard in a region where the information it needs is missing. The solution is to supply that information by running the dynamics on a sequence of distributions that make the hard part easy. Annealed Langevin dynamics The idea Song and Ermon proposed in 2019 is to run Langevin dynamics not on one distribution but on a ladder of progressively less noisy ones. Choose noise levels σ₁ > σ₂ > … > σ_L, with σ₁ large and σ_L small. The network has been trained to estimate the score of the data perturbed by each of these. Begin by sampling from a broad Gaussian appropriate to σ₁. Run a number of Langevin steps using the score at σ₁. Then, keeping the current state, switch to σ₂ and run more steps. Continue down the ladder. The final state, after the steps at σ_L, is the sample. At the top of the ladder the distribution is so heavily smoothed that it is nearly a single broad Gaussian. Its score is well defined everywhere, well estimated because noisy training samples cover the whole region, and simple in shape, so Langevin mixes quickly. The modes of the data have merged, and the mass assigned to each region of the merged blob reflects the mixture weights of the underlying data. So the samples at σ₁ are distributed in roughly the right proportions over the coarse regions of space. Moving down one rung, the distribution sharpens a little. The current samples are already close to where they should be at this new level, because adjacent levels are similar, so a short Langevin run suffices to adjust them. Fine distinctions that were blurred at σ₁ begin to emerge, and each sample is drawn toward the sub-region it is nearest to. Because the samples arrived in the right proportions, they split among sub-regions in the right proportions too. The process continues, each level refining the previous one, until at σ_L the distribution is nearly the clean data distribution and the samples are nearly clean. This is a form of simulated annealing, though the annealed quantity is noise rather than temperature, and the purpose is correct sampling rather than optimisation. It is also, from another angle, a homotopy method: the hard target distribution is approached through a continuous family of easier ones, and the solution at each stage initialises the next. Song and Ermon used a geometric ladder of noise levels and scaled the Langevin step size at each level in proportion to σ², so that the ratio of the drift step to the noise was consistent across levels. The reason for σ² scaling is that the score at level σ has magnitude roughly proportional to 1/σ, since it points back along noise of size σ divided by σ²; multiplying by σ² gives a step whose size in data space is proportional to σ, matching the scale of the noise at that level. In a 2020 follow-up on improved techniques, Song and Ermon gave practical guidance: the largest noise level should be comparable to the largest Euclidean distance between pairs of training examples, so that the top distribution genuinely covers all modes; the ratio between successive levels should be set so that adjacent distributions overlap substantially; and the network's output should be scaled by 1/σ so that it handles the enormous range of score magnitudes gracefully. With these changes the method produced images of unprecedented quality for score-based models at resolutions up to 256 by 256. Reading the DDPM sampler as annealed Langevin The sampling procedure of Ho, Jain and Abbeel looks superficially different. It starts from a standard Gaussian at step T and applies, once per step, an update of the form xₜ₋₁ = (1/√αₜ) · ( xₜ − (βₜ / √(1 − ᾱₜ)) · ε_θ(xₜ, t) ) + σₜ · z, with z fresh Gaussian noise and σₜ a chosen step noise level. Substitute the relation between noise prediction and score from Chapter 2, ε_θ = −√(1 − ᾱₜ) · s_θ, and the update becomes xₜ₋₁ = (1/√αₜ) · ( xₜ + βₜ · s_θ(xₜ, t) ) + σₜ · z. This is a step along the score of size βₜ, a slight rescaling by 1/√αₜ to undo the forward process's shrinking, and the addition of noise. It is one Langevin-like step per noise level, with a thousand levels, rather than many Langevin steps at each of ten levels. The two procedures sit at opposite ends of a spectrum: many levels with one step each, or few levels with many steps each. Both are annealed Langevin in spirit. Ho and colleagues noted the connection explicitly, and Song and colleagues made it exact in the continuous-time framework of their 2021 paper, which Chapter 5 develops. What distinguishes the DDPM step from a pure Langevin step is that it is not trying to reach equilibrium at each level. A Langevin step at a fixed level aims to sample from that level's distribution and would, if repeated, mix within it. The DDPM step instead moves the sample from one level's distribution to the next one down. It is a step of a reverse-time process rather than a step of an equilibrium sampler. That difference becomes precise in continuous time, where the reverse step turns out to involve the score with a coefficient exactly twice what pure Langevin would use, half of it accounting for the distribution's motion between levels and half for keeping samples spread correctly within each. Predictor and corrector The continuous-time view, introduced by Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon and Ben Poole in their 2021 paper on score-based generative modelling through stochastic differential equations, allows the two styles to be combined. The idea is to separate two jobs. A predictor moves the sample from one noise level to the next, using a discretisation of the reverse-time equation. A corrector then runs a few Langevin steps at the new noise level, using the score there, to pull the sample back toward that level's distribution and correct whatever error the predictor introduced. The value of the corrector is that errors in the predictor accumulate. Each step introduces a small discrepancy between the distribution of the current samples and the true distribution at that level, because the step is finite and the score is imperfect. Without correction, the discrepancies compound down the chain. Langevin correction is self-healing in a specific sense: it is a process whose stationary distribution is the correct one at the current level, so running it pushes any discrepancy back toward zero. Song and colleagues found that predictor-corrector sampling improved sample quality for the variance-exploding models in their paper at a fixed computational budget, though later developments in deterministic solvers, covered in Chapter 7, reduced the role of correctors in mainstream practice. Stochasticity as error correction The general lesson of Langevin dynamics for diffusion sampling is that noise injected during sampling is not merely a source of variety. It is a corrective mechanism. When the sampler wanders slightly off the distribution it should be tracking, because of a discretisation error or an inaccurate score, the combined effect of a score step and fresh noise tends to pull it back. A purely deterministic sampler has no such mechanism: an error made early is carried forward and transformed by every subsequent step. This leads to a tradeoff that recurs throughout practice. Stochastic samplers are robust to errors but need many steps, because each step's noise must be small enough to keep the discretisation accurate and the Langevin component needs time to act. Deterministic samplers can take larger steps and reach good results in far fewer network evaluations, but they are less forgiving of imperfect scores. Karras and colleagues, in the EDM paper, made this tradeoff explicit with a sampler that injects a controlled amount of fresh noise at each step, essentially adding and then removing a little extra noise in the manner of a Langevin corrector. They found that a modest amount of such stochasticity helped on some datasets and models and hurt on others, and that the best setting depended on how accurate the trained network was. For very well-trained networks, deterministic sampling was close to optimal; for networks with residual errors, a little noise helped mask them. Langevin dynamics beyond sampling Langevin dynamics also provides a general recipe for sampling from distributions that combine a learned score with additional information. If one wants samples from p(x) restricted to images consistent with some measurement y, the target is the posterior p(x | y), proportional to p(x) times the likelihood p(y | x). Its score is the sum of the prior score and the gradient of the log-likelihood. A Langevin sampler can use both, adding a known likelihood gradient to the learned prior score. This is the basis of a large literature on solving inverse problems, such as deblurring, super-resolution, inpainting and medical image reconstruction, with a pretrained diffusion model as the prior. Methods such as diffusion posterior sampling, proposed by Hyungjin Chung and colleagues in 2023, approximate the likelihood gradient at each noise level using the network's x̂₀ estimate, since the measurement constrains the clean image and not its noisy version. The approximations are imperfect, but the framework is the same one that underlies guidance in Chapter 8: add gradients of log-probabilities to the score, and sample from the product of the corresponding distributions. The same additivity is also a warning. Adding an extra gradient term to the score changes the target distribution. Unless the extra term is the gradient of a genuine log-likelihood, at the correct noise level, the sampler converges to something other than intended. Many practical procedures add terms that are only heuristically justified, and their behaviour reflects that. Chapter 8 examines the clearest case. What Langevin dynamics establishes The main point of this chapter can be stated in one sentence. Given the score of a distribution, a stochastic process that steps along the score while injecting noise in the right proportion has that distribution as its equilibrium, and annealing through a sequence of increasingly sharp noisy distributions makes that equilibrium reachable in practice. Everything a diffusion sampler does is a refinement of this idea. What remains unclear at this stage is how many steps are really needed, how to choose the noise levels, and whether the noise during sampling is essential. To answer those questions properly, two more pieces of theory are needed: the variational view, which explains the DDPM objective and its relation to likelihood, and the continuous-time view, which explains why the sampling noise can be removed without changing what is sampled. Chapter 4. The Variational View: Where the Simple Loss Comes From There are two historical roads into diffusion models, and they arrive at the same place from different directions. The score-based road, followed so far, begins with the question of how to learn a gradient field and sample from it. The variational road begins with latent-variable models and asks how to write down a likelihood for data generated by a long chain of random steps. Ho, Jain and Abbeel came by the second road, and their famous simple loss, the mean-squared error between true and predicted noise, emerged as a reweighted piece of a variational bound. Understanding that derivation explains what the simple loss is, what it gives up, and why later work spent so much effort on weighting. Diffusion as a deep latent-variable model Consider the reverse of the forward chain. A generative model starts at x_T, drawn from a standard Gaussian, and produces x_{T−1}, then x_{T−2}, and so on down to x₀, the data. Each step is a learned conditional distribution p_θ(xₜ₋₁ | xₜ). The intermediate states x₁ through x_T are latent variables: they are never observed in the data, only the final x₀ is. The model's likelihood for a data point is the integral over all possible latent trajectories: p_θ(x₀) = ∫ p(x_T) · ∏ₜ p_θ(xₜ₋₁ | xₜ) dx₁ … dx_T. That integral is intractable. The standard remedy, from the variational autoencoder literature, is to introduce an approximate posterior over the latents given the data and use it to bound the log-likelihood from below. What makes diffusion models unusual among latent-variable models is that the approximate posterior is not learned. It is the forward process q(x₁, …, x_T | x₀), fixed in advance. A diffusion model is thus a hierarchical variational autoencoder with T layers of latents, each of the same dimension as the data, an encoder that is a fixed noising chain, and a decoder that is a learned denoising chain. This framing was present in Sohl-Dickstein and colleagues' 2015 paper and was made central again in Kingma and colleagues' variational diffusion models. The evidence lower bound Jensen's inequality gives the standard bound: log p_θ(x₀) ≥ E over q of [ log p_θ(x₀, x₁, …, x_T) − log q(x₁, …, x_T | x₀) ]. The right-hand side is the evidence lower bound, or ELBO. Maximising it over θ pushes up the likelihood. Written out, it is a sum of terms, one per step of the chain, and a clever rearrangement makes each term tractable. The rearrangement uses the forward posterior from Chapter 1: by Bayes' rule and the Markov property, the forward process can be rewritten in reverse order, conditioned on x₀, as a product of the Gaussian posteriors q(xₜ₋₁ | xₜ, x₀). After cancellation, the negative ELBO decomposes as L = L_T + Σ over t from 2 to T of L_{t−1} + L₀, with three kinds of term. The prior term L_T is the Kullback–Leibler divergence between q(x_T | x₀) and the standard Gaussian p(x_T). It has no learnable parameters and is near zero when the schedule drives the signal close to zero. The reconstruction term L₀ is the negative log-probability of the clean data under the final decoder step p_θ(x₀ | x₁); it handles the conversion from a continuous noisy state to the data, which for images is discrete pixel values, and Ho and colleagues used a discretised Gaussian decoder for it. And the denoising terms, which carry nearly all the content, are L_{t−1} = KL( q(xₜ₋₁ | xₜ, x₀) ‖ p_θ(xₜ₋₁ | xₜ) ). Each one compares the true reverse step, which knows x₀, with the model's reverse step, which does not. Minimising it trains the model to make its reverse step look as much as possible like what the true posterior would do if x₀ were known, averaged over all x₀ consistent with the current state. Gaussian reverse steps and the mean-matching loss Both distributions inside each KL are Gaussian. The true posterior q(xₜ₋₁ | xₜ, x₀) has mean μ̃ₜ(xₜ, x₀) = (√ᾱₜ₋₁ · βₜ / (1 − ᾱₜ)) · x₀ + (√αₜ · (1 − ᾱₜ₋₁) / (1 − ᾱₜ)) · xₜ and variance β̃ₜ = βₜ (1 − ᾱₜ₋₁) / (1 − ᾱₜ). The formula for the mean is less important than its form: it is a fixed weighted combination of the current noisy state and the clean data. The model's reverse step is chosen to be Gaussian too, with a learned mean μ_θ(xₜ, t) and, in the simplest version, a fixed variance. The KL between two Gaussians with the same variance is proportional to the squared distance between their means, so L_{t−1} = (1 / (2σₜ²)) · ‖ μ̃ₜ(xₜ, x₀) − μ_θ(xₜ, t) ‖² + constant. The model is learning to predict the posterior mean. Since the posterior mean is a known combination of xₜ and x₀, and xₜ is given, the only unknown is x₀. One could therefore parameterise μ_θ by having the network predict x₀ and plugging the prediction into the formula. Or, since x₀ can be expressed in terms of xₜ and the noise ε through the forward equation, one can have the network predict ε. Ho and colleagues chose the latter. With that substitution the denoising term becomes L_{t−1} = (βₜ² / (2σₜ² · αₜ · (1 − ᾱₜ))) · ‖ ε − ε_θ(xₜ, t) ‖². So the true variational bound is a weighted sum, over steps, of noise-prediction errors, with a weight that depends on the schedule and the chosen reverse variance. This is denoising score matching again, arrived at from the likelihood side, with one specific weighting across noise levels. The simple loss and what it discards Ho and colleagues then made a decision that changed the field. They dropped the weights. Their training objective was L_simple = E over t, x₀, ε of ‖ ε − ε_θ(√ᾱₜ x₀ + √(1 − ᾱₜ) ε, t) ‖², with t drawn uniformly from 1 to T. Compared with the true ELBO, this down-weights the terms at small t, where the variational weight is large, and relatively up-weights the terms at larger t. The ELBO cares intensely about the low-noise steps, because that is where fine details live and where the likelihood of the precise pixel values is determined. But fine detail is mostly imperceptible noise-like texture, and spending model capacity on getting it exactly right in the likelihood sense does little for how images look. The simple loss shifts emphasis toward the moderate and high noise levels, where composition, shape and object identity are decided. The result was a model that produced the best-looking samples of any likelihood-based method at the time, with a Fréchet Inception Distance of 3.17 on unconditional CIFAR-10, but whose log-likelihood was not competitive with the best autoregressive models. The paper's authors noted this explicitly. Their model was good at perceptual quality and mediocre at compression. The trade between likelihood and perceptual quality is a recurring theme in generative modelling, and the simple loss placed diffusion models firmly on the perceptual side. Learned variances and the improved model A year later, Alex Nichol and Prafulla Dhariwal at OpenAI asked whether diffusion models could also achieve good likelihoods without sacrificing sample quality. Their improved denoising diffusion probabilistic model made three changes. They introduced a cosine noise schedule, discussed in Chapter 6, that destroys information more evenly across steps. They learned the reverse-step variances, rather than fixing them, by having the network output an interpolation coefficient between the two natural extremes, βₜ and β̃ₜ, which correspond to the upper and lower bounds on the optimal reverse variance. And they trained with a hybrid objective: the simple loss for the mean, plus a small multiple of the true variational bound for the variances, with gradients stopped so that the variational term affected only the variance outputs. Learning the variances mattered most when sampling with fewer steps than training. With a thousand steps, both extremes for the variance give similar results, because each step is so small that the variance barely matters. With fifty steps, each reverse step is large, the true posterior over the previous state is broader, and getting its variance right becomes important. Nichol and Dhariwal showed that learned variances allowed high-quality sampling with substantially fewer steps than the original model, an early indication that the step count used in training need not dictate the step count used in generation. Variational diffusion models and the SNR view of the loss The cleanest account of the variational bound came from Kingma, Salimans, Poole and Ho in their 2021 paper. They took the continuous-time limit, letting T go to infinity, and showed that the diffusion loss becomes an integral over noise levels which, when expressed in terms of the signal-to-noise ratio, takes a simple form: L_∞ = −½ · E over ε of ∫ SNR′(t) · ‖ x₀ − x̂_θ(xₜ, t) ‖² dt, where SNR′ is the derivative of the signal-to-noise ratio with respect to time and x̂_θ is the network's prediction of the clean data. Changing the variable of integration from t to the SNR itself, the time variable vanishes entirely. The loss depends only on the endpoints, the SNR at the start and end of the process, and not on the shape of the schedule in between. This is a surprising and useful result. It means that, for the continuous-time variational bound, the noise schedule is irrelevant: any schedule that starts and ends at the same SNR values gives the same objective in expectation. What the schedule does affect is the variance of the Monte Carlo estimate of that objective, because the loss is estimated by sampling a few noise levels per training example. A schedule that samples noise levels where the integrand is large and varies a lot will produce lower-variance gradient estimates and therefore faster training. Kingma and colleagues learned the schedule to minimise this variance, and with that and other improvements obtained log-likelihoods on image benchmarks that beat the autoregressive models of the time. Every weighting is an ELBO in disguise The most elegant unification came two years later. Kingma and Ruiqi Gao, in a 2023 paper, studied the family of weighted diffusion losses: integrals over log-SNR of a weighting function times the squared error in noise prediction. The simple loss is one member of this family; the ELBO is another; the weightings used by Karras and colleagues, by Salimans and Ho's velocity parameterisation, and by others are further members. Kingma and Gao showed that whenever the weighting function is monotonic, decreasing as the noise level decreases, the weighted loss equals the ELBO of the data under a particular form of data augmentation: the ELBO averaged over a distribution of additive Gaussian noise applied to the data before modelling. This reframes the long-running tension between likelihood and perceptual quality. Weightings that produce good samples are not arbitrary departures from likelihood. They are likelihoods of a slightly noised version of the data, in which the finest details, those that a small amount of noise would wash out, are given less importance. Since human perception is relatively insensitive to those details, a model that ignores them gives up compression performance but gains capacity for perceptually salient structure. It also clarifies why the weighting matters so much and why it cannot be chosen once and for all: the right weighting depends on what the model is for, and on which scales of structure the downstream use cares about. Diffusion versus other latent-variable models The variational view also explains why diffusion models avoid problems that plague other latent-variable models. A standard variational autoencoder learns both encoder and decoder, and the two can collude: the decoder may become powerful enough to ignore the latents, a failure known as posterior collapse, or the encoder may produce latents the decoder cannot use well. In a diffusion model the encoder is fixed and simple, so it cannot collude. Every latent has a definite, known relationship to the data, and the decoder's task at each step is narrowly defined. The cost is that the latents carry no compression: each xₜ has the full dimension of the data, and the model gives up the representation-learning benefits of a bottleneck. Latent diffusion, covered in Chapter 9, recovers some of those benefits by running the diffusion inside a learned compressed space. Compared with generative adversarial networks, the variational view highlights a different strength. A GAN has no likelihood and trains through a minimax game whose equilibrium is notoriously hard to reach; it frequently suffers mode collapse, generating only a subset of the data's variety. A diffusion model maximises a bound on likelihood, or a reweighted version of one, and likelihood objectives penalise missing modes severely, since any real data point assigned near-zero probability makes the loss very large. This is one reason diffusion models cover the diversity of their training data more faithfully than GANs did, and it was part of the evidence Dhariwal and Nichol marshalled in their 2021 paper arguing that diffusion models beat GANs on image synthesis. What the variational view establishes The variational derivation shows that the diffusion objective is a principled bound on log-likelihood, that the popular simple loss is a reweighting of it, and that the reweighting shifts emphasis from imperceptible fine detail toward perceptually important structure. It shows that in continuous time the schedule does not change the objective, only the efficiency of estimating it. And it shows that monotone weightings are themselves likelihoods of noise-augmented data, which removes the apparent conflict between principled and practical objectives. With these points in hand, the next step is to take the continuous-time limit seriously, not just for the loss but for the whole generative process, and to see what emerges when the forward and reverse chains become stochastic differential equations. Chapter 5. Continuous Time: Reverse-Time SDEs and the Probability Flow The discrete chain of a thousand steps was a historical accident of how diffusion models were first built. In 2021 Song and colleagues showed that both the score-based models of Song and Ermon and the denoising models of Ho and colleagues are discretisations of a single continuous-time picture, in which the forward process is a stochastic differential equation and generation is the solution of its time reversal. This picture is the unifying framework of the field. It explains where the samplers come from, why they can be swapped and improved, why some samplers can be deterministic, and how a diffusion model can assign exact likelihoods despite being trained on a bound. Stochastic differential equations in brief An ordinary differential equation describes a quantity that changes deterministically: dx = f(x, t) dt says that over a short interval dt the state moves by f(x, t) dt. A stochastic differential equation adds a random term: dx = f(x, t) dt + g(t) dW. The function f is the drift, a deterministic velocity. The function g(t) is the diffusion coefficient, controlling the size of the random kicks. And dW is the increment of a Brownian motion, which over a short interval dt is a Gaussian vector with mean zero and covariance dt times the identity. The defining peculiarity of Brownian motion is that its increments scale like the square root of dt rather than dt itself, so over short intervals the random term dominates the drift, and the path is continuous but nowhere differentiable. A simulation with step size Δt therefore updates x by f Δt plus g times √Δt times a standard Gaussian vector. This is the Euler–Maruyama method, the stochastic analogue of Euler's method for ordinary differential equations. An SDE describes individual random paths. Equally important is how the probability distribution of the state evolves when many paths are run from a common starting distribution. That evolution is governed by a deterministic partial differential equation, the Fokker–Planck equation: ∂pₜ/∂t = −∇·( f pₜ ) + ½ g² Δpₜ. The first term transports probability mass along the drift. The second, involving the Laplacian Δ, spreads it out by diffusion, smoothing peaks and filling valleys. The Fokker–Planck equation is the bridge between the path-level description and the distribution-level description, and every major result in this chapter is a consequence of manipulating it. The forward process as an SDE Both forward processes from Chapter 1 have continuous-time versions. The variance-exploding process, which simply adds noise of growing size, is dx = √( d[σ²(t)]/dt ) dW, with no drift at all. The state is pure Brownian motion with a time-varying rate, and its variance at time t is σ²(t) plus the data's variance. The variance-preserving process, which shrinks the signal while adding noise, is dx = −½ β(t) x dt + √β(t) dW, where β(t) is a continuous version of the step-wise noise rates. The drift pulls the state toward the origin at rate β(t)/2, while the noise injects variance at rate β(t), and the two balance so that a state with unit variance keeps unit variance. This is the Ornstein–Uhlenbeck process with a time-varying rate, a staple of physics and finance. Its solution, starting from x₀, is exactly the Gaussian of Chapter 1 with ᾱ(t) = exp(−∫₀ᵗ β(s) ds). The DDPM chain is its Euler–Maruyama discretisation, to first order in the step size. Song and colleagues also introduced a third family, the sub-variance-preserving SDE, whose variance stays below that of the variance-preserving process and which gave better likelihoods in their experiments. And Karras and colleagues later argued for a particularly simple choice within the variance-exploding family: σ(t) = t, so that the time variable and the noise level coincide. That choice has a geometric virtue which becomes clear once the reverse process is in view. Anderson's theorem: reversing a diffusion The central result was published in 1982 by Brian Anderson, in a paper on reverse-time diffusion equation models in the journal Stochastic Processes and their Applications. Anderson showed that a diffusion process run forward in time has a time reversal that is also a diffusion process, and he gave its equation. For the forward SDE dx = f(x, t) dt + g(t) dW, with marginal densities pₜ, the reverse-time process is dx = [ f(x, t) − g(t)² ∇ₓ log pₜ(x) ] dt + g(t) dW̄, where time now runs backward from T to 0, dt is a negative increment, and W̄ is a Brownian motion running in reverse time. If one starts this equation at time T with a sample from p_T and integrates it backward to time 0, the result is a sample from p₀, the data distribution. More than that: at every intermediate time, the distribution of the reverse-time state is exactly pₜ, the same marginal as the forward process. The reverse process walks back along the same family of distributions the forward process walked down. The only unknown in the reverse-time equation is the score ∇ log pₜ, at every time t. This is exactly the quantity that denoising score matching teaches a network to estimate, at every noise level. The whole generative procedure is therefore: train s_θ(x, t) to approximate the score of the forward process's marginals, then integrate the reverse-time SDE with s_θ in place of the true score. Different discretisations of this one equation give different samplers. The DDPM ancestral sampler is one of them. Annealed Langevin dynamics is a close cousin. Why should the score appear, and why with coefficient g²? A heuristic argument via the Fokker–Planck equation makes it plausible. Running time backward flips the sign of the transport term in Fokker–Planck, which is easy to arrange by negating the drift. But it also flips the sign of the diffusion term, turning the spreading Laplacian into an anti-diffusion that would concentrate mass, and no process with real noise can do that directly. The trick is to note that the Laplacian of pₜ can be rewritten as the divergence of pₜ times its score. So anti-diffusion with strength ½g² is equivalent to transport along the score with speed ½g², and genuine diffusion with strength ½g² can be added back provided an extra ½g² of score-transport compensates for it. The total score coefficient is therefore g²: half of it reverses the forward diffusion, and the other half cancels the reverse process's own noise. This is the factor of two mentioned in Chapter 3, and it is why a reverse diffusion step is not the same as a Langevin step at a fixed level. The probability flow ODE The heuristic just given suggests something more. If the forward diffusion term in Fokker–Planck can be rewritten as transport along the score, then one need not add any noise at all in the reverse direction. One could convert all of the diffusion into deterministic transport. Song and colleagues made this precise. The ordinary differential equation dx/dt = f(x, t) − ½ g(t)² ∇ₓ log pₜ(x) has the same marginal densities pₜ as the forward SDE at every time t. They called it the probability flow ODE. There is no randomness in it. Integrated backward from a sample of p_T, it deterministically produces a sample of p₀ whose distribution is correct. Integrated forward from a data point, it deterministically produces a point in the noise distribution. This is a striking result, and it is worth being precise about what it claims. Individual paths of the probability flow ODE are not the same as individual paths of the SDE; an SDE path wiggles randomly, an ODE path is smooth. What coincides is the distribution of states across all paths at each time. Imagine a fluid of probability flowing from the data distribution to the noise distribution. The SDE moves each particle with a drift plus random jostling; the ODE moves each particle along a smooth streamline. The density of particles at each moment is the same either way. The probability flow ODE has several important consequences. It makes generation deterministic. A given starting noise vector maps to exactly one output. This means the latent noise is a genuine code for the output: interpolating smoothly between two noise vectors produces a smooth interpolation between outputs, and the same noise always produces the same image. The DDIM sampler of Jiaming Song, Chenlin Meng and Stefano Ermon, published in 2020, was developed independently from a different starting point, as a non-Markovian generalisation of the DDPM forward process, but in its deterministic form it is a first-order discretisation of the probability flow ODE in a cleverly chosen parameterisation. It makes generation fast. ODEs are far easier to integrate accurately than SDEs. A stochastic solver's error is dominated by the noise term, which scales as the square root of the step size and limits how large steps can be. A deterministic solver can use high-order methods, such as Heun's method or multistep schemes, whose error shrinks much faster as steps shrink, and can therefore take large steps with little loss. Chapter 7 shows how this allowed sampling in tens of steps rather than hundreds. It gives exact likelihoods. A deterministic, invertible map between data and a simple base distribution is a continuous normalising flow. The change-of-variables formula for flows, extended to continuous time, says that the log-density changes along a path at a rate equal to minus the divergence of the velocity field. Integrating the divergence along the probability flow path from a data point to the noise distribution, and adding the log-density of the base Gaussian at the endpoint, gives the exact log-likelihood of the data point under the model. The divergence can be estimated cheaply with random projections, using what is called the Skilling–Hutchinson trace estimator. Song and colleagues used this to report likelihoods for their score-based models that were competitive with the best available. A diffusion model is therefore simultaneously a model trained by a likelihood bound and a flow model with an exactly computable likelihood, and the two numbers agree only when the learned score is exact. It permits encoding. Running the ODE forward from a real image gives a noise vector that, when run backward, reproduces the image. This inversion underlies many editing techniques: encode an image, modify the conditioning or the noise slightly, and decode. In practice inversion is approximate, because the learned score is imperfect and the discretised ODE is not exactly reversible, and much engineering has gone into making it faithful enough for editing. The general family between ODE and SDE The probability flow ODE and the reverse SDE are the two ends of a continuum. For any non-negative function γ(t), the equation dx = [ f − ½(1 + γ²) g² ∇ log pₜ ] dt + γ g dW̄ has the same marginals. With γ = 0 it is the ODE; with γ = 1 it is Anderson's reverse SDE; larger values inject more noise and correct for it with stronger score-following. This is the continuous-time form of the stochasticity knob that Karras and colleagues explored and that the DDIM paper exposed through its parameter η, which interpolates between deterministic DDIM and the stochastic DDPM sampler. The choice along this continuum is a choice between two kinds of error. With imperfect scores and finite steps, the sampled distribution drifts from the target. More noise means more Langevin-like correction, which pulls the distribution back toward the target at each level, but also requires smaller steps for accuracy. Less noise permits larger steps but provides no correction. Theoretical work on convergence of diffusion samplers, including analyses by Sitan Chen, Sinho Chewi and colleagues in 2023, has shown that with sufficiently accurate scores both stochastic and deterministic samplers converge to the data distribution in a number of steps polynomial in the dimension and accuracy, without requiring the data distribution to satisfy strong conditions such as log-concavity. These results help explain why diffusion models are so reliable in practice: their sampling error is controlled primarily by the score-estimation error, which training directly minimises. Conditional SDEs and controlled generation The continuous-time framework extends naturally to conditional generation. To sample from p(x | y) for some conditioning y, such as a class label or a text prompt, one replaces the score ∇ log pₜ(x) with the conditional score ∇ log pₜ(x | y) in either the reverse SDE or the ODE. By Bayes' rule, ∇ log pₜ(x | y) = ∇ log pₜ(x) + ∇ log pₜ(y | x). The conditional score is the unconditional score plus the gradient of a noisy classifier, a model that predicts y from the noisy state xₜ. This decomposition is the starting point for classifier guidance and, through a further rearrangement, for classifier-free guidance. Song and colleagues used it in their 2021 paper for class-conditional generation, inpainting and colourisation, by treating each task as conditioning on a known constraint and computing or approximating the corresponding likelihood gradient. Chapter 8 develops the guidance methods in full. A worked example: the Gaussian case The machinery becomes concrete in a case where everything can be computed by hand. Suppose the data distribution is a one-dimensional Gaussian with mean zero and variance s², and the forward process is variance exploding, x_σ = x₀ + σε. Then the noisy distribution at level σ is Gaussian with variance s² + σ², and its score is −x / (s² + σ²). The probability flow ODE in the σ(t) = t parameterisation of Karras and colleagues reads dx/dσ = −σ · ∇ log p_σ(x) = σ x / (s² + σ²). This is solvable in closed form: x is proportional to √(s² + σ²). Starting at large σ with x drawn from a Gaussian of variance s² + σ², and integrating down to σ = 0, the ODE scales x by s / √(s² + σ²), giving a variable of variance s². The ODE has transported the broad Gaussian onto the narrow one by a simple rescaling, deterministically, and the result has exactly the right distribution. The example also shows why the σ(t) = t parameterisation is attractive. The trajectory, plotted against σ, is a smooth curve that approaches a straight line at large σ, where x is nearly proportional to σ. Karras and colleagues observed that, for real data, the ODE trajectories in this parameterisation are nearly straight over most of their length and curve significantly only near the end, at low noise, where the solution commits to a specific sample. Straighter trajectories are easier to integrate with few large steps, which is one reason their sampler performed so well. The same observation, that straight paths are cheap to integrate, motivates the flow matching and rectified flow methods of Chapter 11, which try to make the paths straight by design. What the continuous view establishes In continuous time, the diffusion model is a forward SDE that turns data into noise, a learned approximation to its score at every time, and a reverse-time process that uses that score to turn noise back into data. The reverse process can be stochastic, following Anderson's equation, or deterministic, following the probability flow ODE, or anywhere between, and all of these share the same marginal distributions when the score is exact. Sampling is numerical integration of one of these equations. The quality of a sample depends on the accuracy of the score, the accuracy of the integrator, and the alignment between the noise levels where the score is accurate and the noise levels where the integrator spends its steps. The next chapter turns to that alignment: how noise levels are scheduled, how the network's inputs and outputs are scaled, and why these choices, which look like engineering trivia, decide so much of the outcome. Chapter 6. Noise Schedules and Parameterisation The theory of the previous chapters leaves several quantities free. It says that the forward process must move from data to noise, but not how fast or along what route. It says the network must estimate the score, but not whether to output the score, the noise, the clean image or something else. It says the loss is a weighted average over noise levels, but leaves the weights to the modeller. These choices look like implementation details, and for several years they were handled as such, inherited from one paper to the next. They are not details. A poor choice can cost more sample quality than a doubling of model size, and several of the most consequential advances in the field between 2021 and 2024 consisted of nothing more than making these choices well. Three distinct decisions hiding in one word The phrase noise schedule is used loosely for at least three different things, and confusion between them has caused real errors. It is worth separating them. The first is the forward process path: the functions α(t) and σ(t) that say how much signal and how much noise make up the noisy state at time t. In the variance-preserving case, α = √ᾱ and σ = √(1 − ᾱ); in the variance-exploding case, α = 1 and σ grows. As Chapter 1 showed, any two paths that visit the same range of signal-to-noise ratios define the same family of distributions up to rescaling, so the path matters mainly through its range and its scaling convention. The second is the training noise distribution: the probability with which each noise level is sampled during training, combined with the weight that each level's loss receives. These two together determine how much of the network's capacity is spent on each noise level. Because only their product affects the expected gradient, they can be traded off against each other, though the split affects gradient variance. The third is the sampling discretisation: the particular noise levels at which the sampler evaluates the network when generating. This is chosen after training and can be changed freely without retraining, as long as it stays within the range the network was trained on. Early diffusion models fixed all three together through a single sequence of βs, used identically in training and sampling. Karras and colleagues' 2022 EDM paper was influential in large part because it pulled them apart and showed that each could be optimised independently. The linear and cosine schedules The original DDPM schedule increased β linearly from 0.0001 to 0.02 over a thousand steps. Nichol and Dhariwal observed in 2021 that this schedule destroys information too quickly. Plotting ᾱ against step index, it falls steeply at first and then lingers near zero for the last few hundred steps. Those final steps are spent at noise levels where the image is already essentially destroyed, and they contribute little to training. For low-resolution images in particular, Nichol and Dhariwal found that the last fifth of the linear schedule could be skipped without much loss in sample quality, a sign that it was being wasted. Their cosine schedule defines ᾱ directly rather than through β: ᾱ(t) = cos²( (t/T + s) / (1 + s) · π/2 ) / cos²( s / (1 + s) · π/2 ), with a small offset s = 0.008 to keep the noise from being too small near t = 0. Under this schedule ᾱ falls slowly at the start, roughly linearly in the middle, and slowly at the end, so the noise levels are spread more evenly across the range that matters. In terms of log-SNR, the cosine schedule is close to linear in t over most of its range, and the log-SNR is the quantity whose uniform coverage best matches the varying difficulty of the denoising tasks. The cosine schedule improved likelihoods and sample quality on the datasets they studied and became a default for several years. Parameterising the network: what to predict Chapter 2 showed that predicting the noise ε, predicting the clean data x₀, and predicting the score are equivalent in principle: given the noisy input and the noise level, each determines the others. In practice they are not equivalent, because the network is imperfect and the conversions between them involve division by quantities that can be very small. Consider noise prediction at very low SNR, near pure noise. The noisy input xₜ is almost entirely ε, so predicting ε is easy: the network can nearly copy its input. But converting ε̂ into an estimate of the clean image requires dividing by √ᾱₜ, which is close to zero, so any small error in ε̂ is magnified enormously in x̂₀. At very high SNR the reverse holds: the input is almost entirely x₀, predicting x₀ is easy, and predicting ε requires the network to extract a tiny noise component from a large signal, a delicate task. Each parameterisation has a regime where its target is trivially easy and the conversion to the other targets is badly conditioned. Tim Salimans and Jonathan Ho proposed a middle way in their 2022 paper on progressive distillation. They defined the velocity v = α · ε − σ · x₀, where α and σ are the signal and noise coefficients at the current level, with α² + σ² = 1 in the variance-preserving case. Picturing the noisy state as a point on a quarter-circle rotating from x₀ to ε, v is the tangent direction: the rate of change of the noisy state with respect to the angle. Predicting v is well-conditioned at both ends, because v interpolates smoothly between −x₀ at high noise and ε at low noise, and the conversion from v̂ to x̂₀ or ε̂ never divides by a small number. Velocity prediction was adopted in several subsequent large systems, including later versions of Stable Diffusion and Google's Imagen Video, and was a key ingredient in fixing the terminal-SNR problem discussed below. The EDM preconditioning Karras and colleagues took a more general approach. In the variance-exploding parameterisation, with noisy input x = x₀ + σε, they wrote the denoiser D(x, σ), whose job is to output an estimate of x₀, as a combination of the input and a raw network output F: D(x, σ) = c_skip(σ) · x + c_out(σ) · F( c_in(σ) · x, c_noise(σ) ). The four functions are not learned; they are chosen from first principles. The input scaling c_in = 1 / √(σ² + σ_data²) normalises the network's input to unit variance at every noise level, where σ_data is the standard deviation of the clean data, which they took to be 0.5 for images normalised to the range from minus one to one. The skip coefficient c_skip = σ_data² / (σ² + σ_data²) and the output scaling c_out = σ · σ_data / √(σ² + σ_data²) are chosen so that the effective target for F has unit variance at every noise level and so that the network's errors are amplified as little as possible. The noise conditioning c_noise is a quarter of the logarithm of σ, a convenient compression of the huge range of noise levels into a small input. The effect is that the network F always sees inputs of unit variance and always has a target of unit variance, regardless of the noise level. At low noise, c_skip is close to one, so the denoiser mostly passes the input through and the network predicts a small correction, similar to noise prediction. At high noise, c_skip is close to zero, so the denoiser relies on the network, which predicts the clean image directly. The parameterisation interpolates automatically between ε-prediction and x₀-prediction according to which is better conditioned, and v-prediction turns out to be a special case of the same family. Table 1 summarises how the common parameterisations compare. Table 1. Common prediction targets for diffusion networks and how they behave across noise levels. Target What the network outputs Behaviour at high noise Behaviour at low noise Used prominently in Noise ε The Gaussian noise added Trivial target; poor x₀ estimates Hard to separate small noise DDPM, Stable Diffusion 1.x Clean data x₀ The denoised image Blurry mean; well-conditioned Trivial target; poor ε estimates Some early text and audio work Velocity v α·ε − σ·x₀ Behaves like −x₀ prediction Behaves like ε prediction Imagen Video, Stable Diffusion 2.x EDM preconditioned Scaled residual F Mostly x₀ prediction Mostly ε prediction EDM, EDM2 Flow velocity ε − x₀ Constant-scale target Constant-scale target Stable Diffusion 3, flow models The table's last row anticipates Chapter 11: the flow-matching target is another affine combination of ε and x₀, with fixed coefficients rather than noise-dependent ones. Weighting and the training noise distribution With the network's target normalised, the remaining question is how much effort to put into each noise level. Karras and colleagues set the loss weight so that the effective weight on F's error was uniform across noise levels, and then controlled the emphasis entirely through the distribution from which training noise levels are drawn. They chose a log-normal distribution: the logarithm of σ is drawn from a Gaussian with mean −1.2 and standard deviation 1.2. This concentrates training on intermediate noise levels, around σ of 0.3 for images in the minus-one-to-one range, and spends relatively little effort on very low noise, where the task is easy and nearly irrelevant to perceptual quality, or on very high noise, where the network's best answer is nearly the dataset mean and there is little to learn. The justification was empirical but principled. They measured the training loss of a trained network at each noise level and found that it fell to a low value at small σ, where denoising is easy, and plateaued at a moderate value at large σ, where it is intrinsically uncertain. The region where the loss could still be substantially reduced by more training was in the middle. Focusing sampling there spends compute where it buys improvement. The combination of preconditioning, log-normal training noise, an improved sampler and some retuned training hyperparameters took EDM to a Fréchet Inception Distance of 1.79 on class-conditional CIFAR-10 with 35 network evaluations per sample, a state-of-the-art result at the time, while keeping the network architectures they had inherited from earlier score-based work. Later work refined this further. Esser and colleagues, training the Stable Diffusion 3 models with a flow-matching formulation in 2024, ran a systematic comparison of training noise distributions and found that a logit-normal distribution over the interpolation time, which likewise concentrates samples at intermediate noise levels, performed best among those they tested. Karras and colleagues, in their 2024 EDM2 paper, went further and let the network learn a per-noise-level uncertainty that adapts the loss weighting automatically during training, removing the need to hand-tune it. The terminal SNR flaw A schedule is supposed to end at pure noise, so that sampling can begin with a standard Gaussian. Shanchuan Lin, Bingchen Liu, Jiashi Li and Xiao Yang pointed out in 2023 that many widely used schedules do not. The schedule used by Stable Diffusion, for example, ends with a small but nonzero signal-to-noise ratio. At the final training step, the noisy image still contains a faint trace of the original, in particular its overall brightness, which is the lowest-frequency component and the last to be destroyed. The network learns to exploit this trace. During training, it never sees a truly signal-free input; it always has a hint of the image's mean brightness. At sampling time, however, generation starts from a pure Gaussian, whose mean is zero, which the network interprets as a hint that the image has average brightness. The result is a systematic inability to generate images that are very bright or very dark overall: a prompt asking for a pure black background or a solid white scene yields something closer to mid-grey. The flaw was visible in practice for a long time before it was explained, and users had developed workarounds such as adding low-frequency offset noise during fine-tuning. Lin and colleagues' fix had several parts. Rescale the schedule so that the final SNR is exactly zero. Switch to velocity prediction, since noise prediction is meaningless at zero SNR: when the input is pure noise, the network can predict it perfectly by copying, and the conversion to x₀ involves division by zero. Ensure that sampling actually starts at the final training step, since some samplers' step-spacing conventions skipped it. And rescale the outputs of classifier-free guidance, discussed in Chapter 8, to avoid overexposure. With these changes, models could generate the full range of brightness. The episode illustrates the chapter's theme: the forward process's endpoint is part of the model's specification, and a small error there propagates into every sample. Resolution and the shifting of schedules A schedule that works at one resolution does not transfer to another. Chapter 1 explained why: at higher resolution, each object spans more pixels, and averaging over more pixels recovers the low-frequency content of the image more reliably at a given per-pixel noise level. A noise level that obliterates the composition of a 64 by 64 image leaves the composition of a 512 by 512 version of the same scene partly visible. If the same schedule is used for both, the high-resolution model sees too few genuinely uninformative states and spends too little training effort on the global structure that the high-noise levels are supposed to teach. Hoogeboom, Heek and Salimans proposed a simple correction in their 2023 work on simple diffusion: shift the log-SNR of the schedule down by an amount proportional to the logarithm of the ratio between the target resolution and a reference resolution. A schedule designed for 64 by 64 images, used at 256 by 256, has its log-SNR reduced by a fixed amount at every step, so that the effective destruction of global structure matches what the reference schedule achieved at low resolution. With this and a few architectural adjustments, they trained pixel-space models at high resolution without the cascade of super-resolution models that systems such as Imagen had relied on. The Stable Diffusion 3 work adopted a similar resolution-dependent timestep shift for its flow-matching models, and related shifts are now standard in high-resolution image and video systems. For video the effect is compounded, since adjacent frames are highly redundant and averaging across time recovers even more of the signal. Video systems therefore typically shift toward still higher noise levels than image systems of the same spatial resolution. The general principle is that the appropriate schedule depends on the redundancy of the representation, which is why it must be reconsidered whenever the representation changes. Sampling discretisation The noise levels at which a sampler evaluates the network are the third choice, and the only one that can be revised after training. The most naive choice is to space them uniformly in t, or to take every k-th step of the training schedule. Karras and colleagues instead spaced the sampling levels according to a power law in σ: the ρ-th root of σ is spaced uniformly between σ_max and σ_min, with ρ = 7 in their experiments. This places many more steps at low noise levels, where the ODE trajectory curves and integration errors matter most, and fewer at high noise, where trajectories are nearly straight. Different datasets and solvers favour somewhat different spacings, and some later work optimises the sampling levels directly for a given model and step budget. The range of sampling noise levels also matters. The highest level must be large enough that the initial Gaussian is a good approximation to the true noisy distribution, and the lowest must be small enough that the final output is effectively clean. EDM used σ_max = 80 and σ_min = 0.002 for images normalised to the minus-one-to-one range. Table 2 places the main schedule families side by side. Table 2. Major noise schedule families and the choices each makes. Family Signal and noise path Typical training noise distribution Typical sampling spacing Origin DDPM linear Variance preserving, β linear Uniform over 1,000 steps Every step, or strided subset Ho, Jain and Abbeel, 2020 Cosine Variance preserving, ᾱ as cosine Uniform over steps Strided subset Nichol and Dhariwal, 2021 Variance exploding x₀ + σε, σ geometric Geometric ladder or log-uniform Geometric Song and Ermon, 2019 EDM x₀ + σε, σ(t) = t Log-normal in σ Power law in σ, ρ = 7 Karras and colleagues, 2022 Rectified flow (1 − t)·x₀ + t·ε Logit-normal in t, resolution-shifted Shifted uniform in t Esser and colleagues, 2024 The lesson of the schedule Every row in Table 2 describes the same underlying object: a path from data to noise through a family of Gaussian-blurred distributions, and a network that learns the score along it. The differences are where the network's training effort is concentrated, how its inputs and outputs are scaled to keep optimisation well behaved, and where the sampler places its steps. None of these change what the model is in the limit of perfect training. All of them change, often dramatically, what the model is in practice, because practice is never the limit. The best schedules are those that align three distributions: the noise levels where the training signal is informative, the noise levels where the network has the capacity to learn, and the noise levels where the sampler needs accuracy. When those are aligned, a given network and compute budget produce markedly better samples. When they are misaligned, the defects are systematic, as the terminal-SNR episode showed, and no amount of additional training removes them. Chapter 7. Sampling as Numerical Integration A trained diffusion network does nothing on its own. Generation happens when a sampler calls it repeatedly, each time at a lower noise level, and combines its outputs into a trajectory from noise to data. In the original DDPM work that meant a thousand calls per image, each a full forward pass through a network with tens or hundreds of millions of parameters. Generating a single 256 by 256 image could take tens of seconds on a powerful accelerator, and the cost was the main practical objection to diffusion models. Within three years it had fallen by two orders of magnitude, through a sequence of advances that all came from a single shift in perspective: sampling is numerical integration of a differential equation, and numerical analysis has a great deal to say about integrating differential equations efficiently. The cost model The unit of cost in diffusion sampling is the network function evaluation, or NFE: one forward pass of the denoiser. Everything else a sampler does, such as adding vectors and multiplying by scalars, is negligible by comparison. A sampler's efficiency is therefore measured by the quality it achieves at a given number of NFEs. A first-order method that takes fifty steps uses fifty NFEs; a second-order method that takes twenty-five steps, each requiring two evaluations, uses fifty as well. The comparison that matters is always at equal NFE. The quality of a sampler at a given budget depends on two kinds of error. Truncation error arises because the solver approximates a continuous trajectory with a finite number of steps; it shrinks as steps get smaller, at a rate determined by the solver's order. Score error arises because the network is not the exact score; it is fixed once training is done, and no sampler can remove it, though some samplers amplify it less than others. With many steps, truncation error is small and score error dominates; with few steps, truncation error dominates. The art of fast sampling is reducing truncation error at small step counts without amplifying score error. Ancestral sampling and its limits The DDPM sampler is called ancestral sampling because it draws each state from the learned conditional given its successor, as one draws ancestors in a graphical model. In SDE terms it is a particular first-order discretisation of the reverse-time SDE. Its truncation error is governed by the noise term, which scales with the square root of the step size, and halving the number of steps degrades quality noticeably. Taking strided subsets of the training schedule, say every tenth step, works up to a point but degrades sharply below a hundred or so steps. Nichol and Dhariwal's learned variances pushed this somewhat lower, but the fundamental problem remains: a stochastic first-order method is a poor integrator. DDIM: deterministic sampling The first large step forward came from Jiaming Song, Chenlin Meng and Stefano Ermon in their 2020 paper on denoising diffusion implicit models. Their observation was that the DDPM training objective depends only on the marginal distributions q(xₜ | x₀), not on the joint distribution of the whole forward chain. Many different forward processes, including non-Markovian ones in which each step depends on x₀ as well as the previous state, share those marginals. A network trained with the DDPM loss is therefore a valid model for all of them, and one can sample using the reverse of whichever is most convenient. In particular there is a member of this family whose reverse process is deterministic. Its update, written in terms of the network's noise prediction, is xₜ₋₁ = √ᾱₜ₋₁ · x̂₀ + √(1 − ᾱₜ₋₁) · ε̂, where ε̂ is the network's noise prediction at xₜ and x̂₀ is the corresponding clean-image estimate. The update has a clear reading. Estimate the clean image and the noise from the current state. Then construct the state at the next noise level by mixing the estimated clean image and the estimated noise in the proportions that level calls for. No fresh randomness is added: the same noise estimate is reused, so the trajectory is deterministic given its starting point. DDIM produced good samples in fifty steps and acceptable ones in twenty, roughly an order of magnitude better than ancestral sampling at the same budget. Its success was later understood through the continuous-time framework. In the variance-exploding coordinates of Chapter 5, DDIM is exactly Euler's method applied to the probability flow ODE. Its efficiency comes from two sources: it integrates an ODE rather than an SDE, so its truncation error falls linearly with step size rather than as a square root; and the parameterisation in which Euler's method is applied makes the trajectory close to linear over most of its length, so even a first-order method tracks it well. DDIM also introduced a parameter η that interpolates between the deterministic sampler at η = 0 and an ancestral-style stochastic sampler at η = 1. This is the stochasticity continuum of Chapter 5, exposed as a user setting. Higher-order solvers Once sampling is recognised as integrating an ODE, the classical toolkit applies. Euler's method evaluates the velocity at the start of a step and moves along it. Heun's method, a second-order Runge–Kutta scheme, takes a trial Euler step, evaluates the velocity at the trial endpoint, and then moves along the average of the two velocities. Its truncation error per step scales as the cube of the step size rather than the square, so for a given accuracy it can take much larger steps. Karras and colleagues adopted Heun's method in the EDM sampler, with the power-law step spacing of Chapter 6 and a final Euler step at the lowest noise level to avoid evaluating the network at σ = 0. They tested higher-order Runge–Kutta methods too and found that second order was the sweet spot: higher orders required more evaluations per step, and the gains in truncation error did not compensate at the step counts of interest. With this sampler their CIFAR-10 models reached state-of-the-art quality in 35 NFEs, where earlier methods had used hundreds or thousands. A different family of improvements exploits the specific structure of the diffusion ODE. In the variance-preserving parameterisation, the probability flow ODE has a linear part, coming from the forward process's drift, and a nonlinear part, coming from the network. The linear part can be integrated exactly. Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li and Jun Zhu exploited this in their 2022 DPM-Solver. Writing the solution through the variation-of-constants formula, they obtained an exact expression for the next state as a known linear combination of the current state plus an integral of the network's output, weighted by an exponential factor, over the step. Changing the variable of integration to log-SNR makes the integral especially simple. Approximating the network's output within the step by a low-order Taylor expansion in log-SNR then gives solvers of first, second and third order. The first-order version is exactly DDIM, which provides another explanation of DDIM's effectiveness: it is the first-order exponential integrator for the diffusion ODE. DPM-Solver produced good samples in ten to twenty NFEs. Its successor, DPM-Solver++, reformulated the solver in terms of the clean-image prediction x̂₀ rather than the noise prediction, which proved more stable under classifier-free guidance, where the effective score can be large and the noise prediction poorly behaved. Multistep variants reuse network evaluations from previous steps to estimate higher-order terms, so a second-order method costs one evaluation per step rather than two. Further refinements such as UniPC, a unified predictor-corrector framework from Wenliang Zhao and colleagues in 2023, push in the same direction. For practical text-to-image generation in the period from 2022 to 2024, multistep DPM-Solver++ variants with twenty to thirty steps became a standard choice in widely used open-source tools. Where the trajectory is hard Why can't solvers go further, to five steps or one, simply by raising the order? The obstacle is the shape of the trajectory near the end. Over most of its length, the probability flow trajectory in good coordinates is nearly straight: at high noise the denoiser's prediction is a smooth average that changes slowly, and the state moves steadily toward it. Near low noise, however, the trajectory turns. The denoiser's prediction sharpens from a blurred average into a specific image, and the state must follow it. The curvature there is high, and the network's output varies rapidly with noise level. High-order Taylor expansions assume smoothness that is not present. There is also a more fundamental limit. With very few steps the solver evaluates the network only at a handful of noise levels, and the network's output at each is a posterior mean. A posterior mean at high noise is blurry. If the solver jumps from high noise directly to the clean image along the network's prediction, it lands on the blurry mean, not on a sharp sample. Getting a sharp sample requires visiting enough intermediate levels that the posterior has been progressively narrowed. A pure ODE solver with an exact score can in principle do this with large steps, but with a learned score the curvature near the end defeats it. Below roughly ten steps, all training-free solvers degrade, and fast generation requires changing the model itself. Distillation The idea of distillation is to train a second network, the student, to reproduce in fewer steps what the original network, the teacher, does in many. Because the probability flow ODE defines a deterministic map from noise to data, the target of this training is well defined: for each starting noise vector, the student should produce the same output that the teacher's many-step ODE solve produces. Salimans and Ho's 2022 progressive distillation does this in stages. A student is trained so that one of its DDIM steps matches two of the teacher's. The student, now as good with N/2 steps as the teacher was with N, becomes the teacher for the next round. Repeating halves the step count each round. Starting from a model sampled with 8,192 steps, they distilled down to four steps with little loss of quality on CIFAR-10 and to similar small step counts at higher resolution. It was in this work that velocity prediction was introduced, because as the step count falls the student must predict the clean image from very noisy inputs, and noise prediction becomes badly conditioned exactly there. Consistency models, introduced by Yang Song, Prafulla Dhariwal, Mark Chen and Ilya Sutskever in 2023, take a different route. They train a network to map any point on a probability flow trajectory directly to the trajectory's endpoint at the data. The defining property, consistency, is that all points on the same trajectory map to the same output. This can be enforced by distillation, comparing the student's output at two adjacent points on a trajectory computed with one solver step of the teacher, or by consistency training without a teacher, using the forward process's known noise to construct adjacent pairs. A consistency model can generate in a single step, by mapping pure noise to data in one evaluation, and can trade compute for quality by alternating denoising and re-noising in a few steps. Latent consistency models, from Simian Luo and colleagues in 2023, applied the idea to text-to-image latent diffusion models and brought generation down to a handful of steps. A third approach adds an adversarial loss. Adversarial diffusion distillation, from Axel Sauer and colleagues in 2023, trains a student to generate in one to four steps, using both a distillation loss against the teacher and a discriminator loss that pushes outputs toward realistic images. The resulting models, released as SDXL Turbo, generated images of reasonable quality in a single step. Distribution matching distillation, from Tianwei Yin and colleagues in 2024, trains a one-step generator by matching its output distribution to the teacher's, using the difference between the teacher's score and the score of a model fitted to the student's outputs as a gradient signal. This last idea, using two score models to estimate the direction that would move one distribution toward another, is an elegant return to the theme of this book: the score is the object through which distributions are compared and moved. Distillation has costs. A distilled model is fixed to its step count and often to a particular guidance strength. It typically loses some diversity, especially when adversarial losses are used, since a discriminator rewards realism rather than coverage. And it no longer has the flexible quality-for-compute trade of the original. For many applications the trade is worth it; interactive image generation in under a second is qualitatively different from generation in ten. Comparing the families Table 3 gathers the main sampler families discussed in this chapter and the trades each makes. Table 3. Families of diffusion samplers and their trade-offs. Sampler family Equation integrated Typical steps for good quality Needs retraining Main trade-off Ancestral (DDPM) Reverse SDE, first order Hundreds to 1,000 No Robust but slow DDIM Probability flow ODE, first order 20 to 100 No Deterministic; degrades below about 20 Heun (EDM) Probability flow ODE, second order About 30 to 80 No Two evaluations per step DPM-Solver++ and multistep Probability flow ODE, exponential integrator 10 to 25 No Good with guidance; fails below about 10 Progressive distillation Learned shortcut of ODE 4 to 8 Yes Fixed step count Consistency and adversarial distillation Direct map to endpoint 1 to 4 Yes Some loss of diversity The step counts are indicative ranges drawn from the papers cited in this chapter; the exact numbers depend on dataset, resolution and quality threshold. Stochastic or deterministic in practice Practitioners often ask whether to use a stochastic or deterministic sampler, and the theory gives a nuanced answer. Deterministic samplers are better at low step counts, because ODE solvers converge faster. They provide a reproducible map from noise to output, which is useful for editing and for comparing prompts. Stochastic samplers are more forgiving of score errors, because the injected noise lets the Langevin component correct drift, and some practitioners find that they produce more varied textures at high step counts. Karras and colleagues' experiments showed that the best amount of stochasticity depends on the model: their improved networks benefited little from it, whereas older networks with larger score errors benefited more. A useful rule is that stochasticity helps to the extent that the score is wrong, and the better the model, the less it is needed. There is also a subtle interaction with guidance. Classifier-free guidance, the subject of the next chapter, alters the effective score in ways that do not correspond to any proper distribution's score at all noise levels. Under strong guidance, the reverse process is no longer tracking a well-defined family of marginals, and the theoretical justification for both ODE and SDE samplers weakens. Empirically, deterministic samplers with x₀-based parameterisations, such as DPM-Solver++, cope best with strong guidance, which is part of why they became the default in text-to-image tools. What fast sampling teaches about the model The rapid progress in samplers from 2020 to 2024 was possible because the generative model was cleanly separated from the procedure that sampled it. The network estimates the score; the sampler integrates the resulting equation. Improvements to either could be made independently, and improvements to the sampler could be applied to existing models without retraining. That separation is a direct consequence of the mathematical structure set out in Chapters 2 through 5, and it is one of the most practically valuable properties of the diffusion framework. Generative models without it, such as GANs, offer no knob between speed and quality; their single forward pass is all there is. Diffusion models offer a continuum, and choosing a point on it is an engineering decision that can be made at deployment time. Chapter 8. Guidance: Steering the Score A diffusion model trained on images paired with captions learns the conditional score: the gradient of the log-density of images given a caption, at every noise level. Sampling with that score should, in principle, produce images distributed exactly as images with that caption are distributed in the training data. In practice, the result is disappointing. Samples drawn from a well-trained conditional model at face value tend to be diverse but loosely connected to their prompts, with muddled objects and weak adherence to the requested content. The images that made text-to-image systems famous were produced not by sampling the conditional distribution but by sampling a deliberately distorted version of it, in which the conditioning is exaggerated. The mechanism for that exaggeration is guidance, and the most widely used form, classifier-free guidance, is a one-line piece of arithmetic on scores whose effects are both remarkable and characteristically flawed. Why exact conditional sampling disappoints The training data for text-to-image models consists of images paired with captions scraped or written for many purposes, and most captions describe their images only partially. A photograph captioned "a dog in a park" may show a dog small in one corner, partly hidden, in poor light. The conditional distribution of images given that caption therefore includes many images in which the dog is barely visible. Sampling faithfully from it yields such images at their natural frequency. Users asking for a dog in a park, however, usually want a clear, prominent dog in a pleasant park: an image that is not merely consistent with the caption but strongly characteristic of it. There is also a modelling reason. The conditional score is learned imperfectly, and the network's errors tend to weaken the conditioning, because the caption's influence on the noisy image is subtle at high noise levels and easy for the network to underuse. A model that has learned the unconditional structure of images well and the effect of the caption less well will produce generic images with a faint tilt toward the prompt. Guidance compensates for that underfitting as well as for the looseness of the captions. Classifier guidance The first systematic form of guidance came from Prafulla Dhariwal and Alex Nichol's 2021 paper, whose title announced that diffusion models beat GANs on image synthesis. They started from the Bayes decomposition introduced in Chapter 5: ∇ log pₜ(x | y) = ∇ log pₜ(x) + ∇ log pₜ(y | x). The conditional score is the unconditional score plus the gradient of a classifier that predicts the label y from the noisy image. Dhariwal and Nichol trained such a classifier on noisy ImageNet images at all noise levels, then sampled from a diffusion model using the unconditional or conditional score plus a scaled classifier gradient: s̃(x, t) = ∇ log pₜ(x | y) + w · ∇ log pₜ(y | x). With w = 0 this is exact conditional sampling. With larger w, the classifier's gradient is amplified, pushing samples toward regions the classifier is more confident are of class y. Formally, at each noise level the guided score is the score of a distribution proportional to pₜ(x | y) times pₜ(y | x) raised to the power w, a tilted distribution that concentrates on images that are not merely of class y but unmistakably so. The result was a clean trade between fidelity and diversity. As the guidance scale increased, measures of sample quality such as precision and Inception Score improved, and measures of diversity such as recall fell. The Fréchet Inception Distance, which combines both, improved up to a moderate scale and then worsened. With well-chosen scales, Dhariwal and Nichol's models surpassed the best GANs of the time on ImageNet at several resolutions. Classifier guidance had a serious practical drawback. It required a separate classifier trained on noisy images, which is a nonstandard model that cannot be taken off the shelf. And its gradients were sometimes adversarial in character: a classifier's gradient can point toward changes that increase its confidence without making the image more recognisable to a person, much as adversarial examples fool classifiers with imperceptible perturbations. For conditioning on free text rather than a fixed set of labels, training a suitable noisy classifier was harder still. Classifier-free guidance Jonathan Ho and Tim Salimans proposed the alternative that became universal. They presented it at a NeurIPS workshop in 2021 and published an expanded version in 2022. The insight is that Bayes' rule can be rearranged to express the classifier in terms of conditional and unconditional generative models: ∇ log pₜ(y | x) = ∇ log pₜ(x | y) − ∇ log pₜ(x). The classifier's gradient is the difference between the conditional score and the unconditional score. A model that can estimate both scores carries an implicit classifier inside it. Substituting into the classifier-guidance formula gives s̃(x, t) = ∇ log pₜ(x) + (1 + w) · ( ∇ log pₜ(x | y) − ∇ log pₜ(x) ). The guided score starts from the unconditional score and moves along the direction from unconditional to conditional, going past the conditional score by a factor 1 + w. In the more common modern notation, with a guidance scale γ = 1 + w, the guided noise prediction is ε̃ = ε_θ(x, ∅) + γ · ( ε_θ(x, y) − ε_θ(x, ∅) ), where ∅ denotes the absence of conditioning. With γ = 1 this is plain conditional sampling. With γ = 0 it is unconditional. With γ greater than one, it extrapolates beyond the conditional prediction, away from the unconditional one. Training both scores requires no second model. During training, the conditioning is randomly replaced by a null token for a fraction of examples, commonly around ten or twenty per cent. The same network thereby learns to produce conditional predictions when given a caption and unconditional ones when given the null token. At sampling time each step evaluates the network twice, once with the caption and once with the null token, and combines the outputs. The cost is a doubling of network evaluations, usually done as a single batched pass, and nothing else. Classifier-free guidance had immediate and dramatic effects. Alex Nichol and colleagues' GLIDE system, in late 2021, compared it with guidance by the CLIP image-text model and found that human raters preferred classifier-free guidance for both photorealism and caption similarity. It was used in DALL·E 2, Imagen and Stable Diffusion, and essentially every text-to-image, text-to-video and text-to-audio diffusion system since has used it or a descendant. Stable Diffusion's widely used default guidance scale was 7.5, a value at which the conditional direction is extrapolated many times beyond the conditional prediction itself, which conveys how far from exact conditional sampling standard practice sits. What classifier-free guidance actually samples The heuristic interpretation of classifier-free guidance is that at each noise level it samples from the tilted distribution pₜ(x) · pₜ(y | x)^γ, which is proportional to pₜ(x | y)^γ · pₜ(x)^(1 − γ). That expression makes the mechanism vivid. Raising the conditional density to a power greater than one sharpens it around its modes, and multiplying by a negative power of the unconditional density penalises images that are common regardless of the caption. The guided distribution favours images that are much more probable given the caption than without it, which is precisely the set of images that are strongly characteristic of the caption. The heuristic is not exact, and the inexactness explains several failure modes. The tilted distribution at each noise level is defined by tilting the noisy marginals. But the tilted noisy marginals are not, in general, the noisy versions of any single tilted clean distribution. Noising and tilting do not commute: tilting the clean distribution and then adding noise gives a different family from adding noise and then tilting each level. A reverse process that uses the guided score at every level is therefore not the time reversal of any forward diffusion, and Anderson's theorem does not tell us what it samples. Analyses since 2023, including work by Arwen Bradley and Preetum Nakkiran describing classifier-free guidance as a kind of predictor-corrector process, have shown that the distribution actually produced differs from the naive tilted target, and that its behaviour depends on the sampler used. In simple mixture examples, strong guidance can push samples beyond the support of the conditional distribution altogether, toward extreme points that are far from both conditional and unconditional data. This is the theoretical root of the most familiar symptoms of excessive guidance: oversaturated colours, high contrast, burnt-out highlights and simplified, posterised compositions. The guided score at high scale is large in magnitude, the predicted clean image overshoots the valid range of pixel values, and the sampler carries those overshoots into the final output. Taming high guidance Several fixes target the overshoot directly. Chitwan Saharia and colleagues, in the 2022 Imagen paper, introduced dynamic thresholding. At each sampling step the predicted clean image x̂₀ is examined; a high percentile of its absolute pixel values is computed; if that value exceeds the valid range, the prediction is clipped to the percentile and rescaled into range. This prevents the predicted image from drifting to extreme values while still permitting large guidance scales, and the Imagen team reported that it substantially improved photorealism and text alignment at high guidance weights. Lin and colleagues, as part of the terminal-SNR work described in Chapter 6, proposed rescaling the guided prediction so that its standard deviation matches that of the conditional prediction, then blending the rescaled and unrescaled versions. The rationale is that guidance inflates the variance of the prediction, and the inflation is what produces overexposure. Restoring the variance keeps the direction of guidance while removing its excessive magnitude. A different fix restricts guidance in time. Tuomas Kynkäänniemi and colleagues at NVIDIA showed in 2024 that guidance is harmful at high noise levels, where it collapses the diversity of compositions by forcing all samples toward the same layout, and largely unnecessary at low noise levels, where the image's content is already decided. Applying guidance only in a limited interval of intermediate noise levels improved both quality and diversity, as measured by FID, relative to applying it throughout. This makes sense in terms of the frequency picture of Chapter 1: high-noise steps decide global composition, and guiding them exaggerates the most stereotypical compositions; middle steps decide object identity and appearance, where adherence to the prompt matters most. Autoguidance The most conceptually interesting recent development is autoguidance, from Tero Karras and colleagues in 2024. They asked why classifier-free guidance improves image quality at all, beyond merely strengthening prompt adherence. Their answer was that the unconditional model is a worse model than the conditional one, because the unconditional task is harder and receives less training. Its errors are similar in kind to the conditional model's errors, but larger. Extrapolating from the unconditional prediction past the conditional one therefore extrapolates away from the direction of error, so to speak, and partially corrects the conditional model's own mistakes, sharpening samples toward the regions where the better model is confident. If that is the mechanism, the unconditional model is not essential. Any weaker version of the same model would do. Karras and colleagues guided a model with a smaller, less-trained version of itself, both conditioned on the same input: D̃(x) = D_weak(x) + γ · ( D_strong(x) − D_weak(x) ). Because both models are conditioned on the same input, the guidance does not push samples toward stereotypical examples of the prompt and does not collapse diversity. It only pushes away from the kinds of errors that a weaker model makes more than a stronger one. On ImageNet they reported record FID scores with this method, and it also improved unconditional generation, where classifier-free guidance cannot be applied at all. Autoguidance reframes guidance as a general technique for using a comparison between two models to correct errors, of which prompt-strengthening is one special case. Negative prompts and compositional guidance The algebra of classifier-free guidance invites variations. Replacing the unconditional prediction with a prediction conditioned on a negative prompt, a description of what the user does not want, makes the extrapolation push away from that content specifically: ε̃ = ε_θ(x, y_neg) + γ · ( ε_θ(x, y) − ε_θ(x, y_neg) ). This simple substitution became a standard feature of image-generation tools, used to suppress artefacts, unwanted styles or objects. More generally, scores can be added for multiple conditions. Since the score of a product of distributions is the sum of their scores, a sampler using the sum of guidance terms for several prompts approximates sampling from the product, an image consistent with all of them. Nan Liu, Shuang Li, Yilun Du and colleagues explored such compositional generation in 2022, with conjunctions realised as sums and negations as differences. InstructPix2Pix, from Tim Brooks, Aleksander Holynski and Alexei Efros in 2023, used two separate guidance scales, one for an input image and one for an editing instruction, so that the user could trade fidelity to the original image against adherence to the edit. These constructions inherit the theoretical caveats of classifier-free guidance, since products of noisy marginals are not noisy marginals of products, but they work well enough in practice to be widely used. Comparing guidance methods Table 4 places the main guidance methods side by side. Table 4. Guidance methods for diffusion models compared. Method Extra model needed Cost per step What it pushes toward Characteristic weakness Classifier guidance Classifier trained on noisy data One denoiser plus classifier gradient Confident classifier predictions Adversarial gradients; label sets only Classifier-free guidance None; conditioning dropout in training Two denoiser passes Images characteristic of the prompt Oversaturation; diversity loss at high scale CFG with thresholding or rescale None Two passes plus cheap post-processing Same, with bounded pixel values Treats symptoms rather than cause Guidance in limited interval None Two passes at middle levels only Prompt adherence without layout collapse Interval must be tuned Autoguidance Smaller or less-trained model Strong plus weak denoiser passes Away from the weaker model's errors Requires a suitable weak model Guidance and the meaning of fidelity The prominence of guidance says something important about what the generative systems built on diffusion are actually doing. The high-fidelity images, audio and video that these systems produce are not samples from the distribution of their training data under the given conditioning. They are samples from a sharpened, exaggerated variant of it, tilted toward the prototypical and away from the generic, with the degree of tilt set by a hyperparameter chosen for appeal. That is not a criticism. It is a design choice that serves users well in most cases. But it has consequences. Guided samples are less diverse than the data, systematically so, and the loss of diversity falls most heavily on the less typical members of each category. A prompt for a doctor, a kitchen or a wedding produces the modal instances more often than the data would justify. Methods such as guidance intervals and autoguidance, which recover diversity while keeping quality, are in part a response to that consequence. The score is the object that guidance manipulates, and understanding guidance as score arithmetic is what makes it possible to reason about what any given manipulation will do. Chapter 9. Networks and Representations: Where the Score Lives The mathematics of diffusion is indifferent to the network that estimates the score. Any function that takes a noisy input and a noise level and returns a vector of the same shape will do, and the training objective will push it toward the true score as far as its capacity allows. But capacity is everything. The quality of a diffusion model in practice is bounded by how well its network can represent the score field, across all noise levels, for data of the complexity and size required. Two developments made high-fidelity generation possible at scale: network architectures suited to the denoising task, and the decision to learn the score not in the raw data space but in a compressed latent space learned separately. Both deserve careful treatment, because both change the meaning of the mathematics in ways that are easy to overlook. What the denoiser must do At each noise level the network must output something close to the posterior mean of the clean data given the noisy input. At high noise that means inferring global structure, such as overall layout, dominant colours and rough shapes, from faint correlations spread across the whole input. At low noise it means removing fine-grained grain while preserving edges and textures, a local operation. In between, it must make decisions about objects and their relationships that depend on both local evidence and global context. A good denoiser architecture therefore needs a large receptive field, so that every output location can depend on the whole input; strong local processing at full resolution; a way to combine information across scales; and a mechanism by which the noise level, and any conditioning such as a text prompt, can modulate all of this. The U-Net The architecture that met these requirements first was the U-Net, introduced by Olaf Ronneberger, Philipp Fischer and Thomas Brox in 2015 for biomedical image segmentation. A U-Net processes the input through an encoder that repeatedly downsamples, halving the spatial resolution and increasing the number of channels at each stage, until it reaches a coarse representation with a large effective receptive field. A decoder then upsamples back to the original resolution. Skip connections pass the encoder's feature maps at each resolution directly to the decoder at the same resolution, so that fine spatial detail lost during downsampling can be recovered. The shape, wide at the ends and narrow in the middle, gives the network its name. For denoising this structure is natural. The coarse middle of the network sees the whole image and can infer global structure; the skip connections let the output retain the precise spatial layout of the input; and the multi-scale hierarchy maps onto the multi-scale structure of natural images. Ho and colleagues' 2020 model used a U-Net with residual blocks at each resolution, group normalisation, and self-attention layers at the 16 by 16 resolution, where attention allows every location to communicate with every other at modest cost. The noise level was converted into a vector by a sinusoidal embedding, the same device used to encode positions in Transformers, and added into each residual block. Dhariwal and Nichol's 2021 model, often called ADM, refined this design through extensive ablation. They used more attention heads and attention at several resolutions, residual blocks adapted from the BigGAN generator, and a conditioning mechanism they called adaptive group normalisation: after each normalisation layer, the features are scaled and shifted by amounts computed from the noise-level embedding and the class embedding. Rather than merely adding the conditioning to the features, this lets it modulate their statistics multiplicatively, a stronger form of control. These changes, together with classifier guidance, were what allowed diffusion models to overtake GANs on ImageNet. Conditioning on text Class labels are a single token. Text prompts are sequences of tokens with rich structure, and conditioning on them requires a mechanism that lets each location in the image attend to the relevant parts of the text. The standard solution is cross-attention. A pretrained text encoder converts the prompt into a sequence of embeddings. At various layers of the denoiser, the image features act as queries and the text embeddings act as keys and values, so that each image location computes a weighted average of text embeddings according to their relevance and incorporates the result. Where the prompt mentions a red car and a blue house, the locations that will become the car can attend to the words describing it. The choice of text encoder matters a great deal. Saharia and colleagues' Imagen, in 2022, used a frozen T5-XXL language model, a large text-only encoder, and found that scaling the text encoder improved sample quality and prompt alignment more than scaling the image denoiser. This was a striking result, suggesting that much of the difficulty of text-to-image generation is in understanding the text. Stable Diffusion's first versions used the text encoder from OpenAI's CLIP model, trained to align images and captions, which captures visual concepts well but understands complex syntax less well. Later systems often combined several encoders: Stable Diffusion 3 used two CLIP encoders and a T5 encoder together, with the option to drop the large T5 at inference for lower memory cost at some loss in handling complex prompts and rendering text. Latent diffusion Running a diffusion model directly on high-resolution pixels is expensive. A 1024 by 1024 colour image has over three million values, and every one of the dozens of sampling steps must process all of them through a large network. Early high-resolution systems such as Imagen and DALL·E 2 used cascades: a base model generated at low resolution, perhaps 64 by 64, and one or more super-resolution diffusion models upsampled the result in stages. Cascades work but are complex to train and prone to compounding errors between stages. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser and Björn Ommer proposed a different strategy in their 2022 paper on latent diffusion models, which became the basis for Stable Diffusion. They separated generation into two stages trained independently. First, an autoencoder is trained to compress images into a smaller latent representation and decompress them back with high perceptual fidelity. Second, a diffusion model is trained on the latents. To generate an image, one samples a latent with the diffusion model and decodes it once with the autoencoder's decoder. The autoencoder in Stable Diffusion's early versions reduced each spatial dimension by a factor of eight and represented each location with four channels. A 512 by 512 by 3 image became a 64 by 64 by 4 latent, a reduction of about forty-eight times in the number of values. The diffusion model, operating on these latents, was correspondingly cheaper, which made it possible to train and run high-resolution text-to-image models on consumer hardware. The autoencoder was trained with a combination of reconstruction loss, a perceptual loss computed on features of a pretrained network, an adversarial loss from a patch-based discriminator, and a light regularisation that kept the latent distribution close to a Gaussian, so that the diffusion model's forward process would behave well on it. The rationale for this split is a division of labour between two kinds of information. Rombach and colleagues argued that most of the bits in a natural image are perceptually unimportant high-frequency detail, which a likelihood-based model would spend most of its capacity on, and that a separately trained autoencoder can handle that detail efficiently, leaving the diffusion model to concentrate on semantically meaningful structure. The autoencoder handles perceptual compression; the diffusion model handles semantic composition. What latent diffusion changes in the mathematics It is important to be clear about what latent diffusion does to the theory of the earlier chapters. The diffusion model now learns the score of the distribution of latents, not of images. Everything in Chapters 1 through 8 applies to that latent distribution. The final image distribution is the pushforward of the learned latent distribution through the decoder, and its quality depends on both stages. Several consequences follow. The diffusion model's errors and the decoder's errors compound, and failures that look like diffusion problems are sometimes decoder problems. Small text, fine patterns and faces at small scale were long weak points of latent diffusion models, partly because a four-channel latent at one-eighth resolution simply could not represent them faithfully, regardless of how good the diffusion model was. Later systems increased the number of latent channels to improve reconstruction: Stable Diffusion 3 moved to sixteen channels, and Esser and colleagues reported that this markedly improved reconstruction quality and, with a large enough diffusion model, final sample quality. The noise schedule must be appropriate for the latent space rather than for pixels. Latent spaces have their own statistics and their own spectral structure, and schedules tuned for pixels may destroy information at the wrong rate. The scale of the latents matters too: Stable Diffusion multiplied its latents by a fixed scaling factor before diffusion so that they would have roughly unit variance, which the forward process assumes. And guidance, likelihoods and inversion all operate in latent space. A probability flow ODE inversion of an image first encodes it with the autoencoder's encoder and then runs the ODE on the latent; errors in either step degrade editing fidelity. Likelihoods computed with the probability flow ODE are likelihoods of latents, not of images. Diffusion Transformers The U-Net's convolutional design encodes strong assumptions about images: locality, translation equivariance and a fixed hierarchy of scales. These assumptions help when data and compute are limited. At large scale, Transformers, which make fewer assumptions and scale predictably with compute, have outperformed convolutional networks in many domains. William Peebles and Saining Xie applied that lesson to diffusion in their 2023 paper on scalable diffusion models with Transformers. Their Diffusion Transformer, or DiT, takes a latent from a latent diffusion autoencoder, divides it into small non-overlapping patches, embeds each patch as a token, and processes the token sequence with a standard Transformer. The noise level and class label are injected through adaptive layer normalisation, the Transformer counterpart of adaptive group normalisation, in which the scale and shift of each normalisation layer are regressed from the conditioning. Peebles and Xie found that a variant they called adaLN-Zero, which initialises each residual block to the identity function by zeroing the final modulation, worked best. A linear decoder maps the output tokens back to patches. The key finding was scaling behaviour. Across a range of model sizes and patch sizes, sample quality as measured by FID improved consistently with the compute used per forward pass, measured in floating-point operations, regardless of whether the compute came from more layers, wider layers or smaller patches producing more tokens. The largest model, DiT-XL/2, achieved an FID of 2.27 on class-conditional ImageNet at 256 by 256 with classifier-free guidance, better than prior U-Net-based diffusion models. This predictability made the Transformer the default backbone for subsequent large-scale systems. Stable Diffusion 3 extended the design with what Esser and colleagues called the multimodal diffusion Transformer, or MMDiT. Instead of injecting text through cross-attention, MMDiT concatenates text tokens and image tokens into a single sequence and processes them with joint attention, while giving each modality its own separate weights for the other parts of each block. Text and image representations thus influence each other symmetrically at every layer. The design improved prompt following, and in particular the rendering of text within images, which had been a conspicuous weakness of earlier models. Training dynamics at scale As networks grew, the dynamics of training them became a subject in its own right. Karras and colleagues' 2024 EDM2 paper examined the ADM-style U-Net used in many diffusion models and found that the magnitudes of activations and weights drifted uncontrollably during training, so that the effective learning rate of different layers changed in unintended ways. They redesigned the layers to preserve magnitudes explicitly, normalising weights and activations so that their scale remains fixed in expectation, and reported large improvements in FID on ImageNet at 512 by 512 with no increase in model size. The same paper introduced post-hoc exponential moving average. Diffusion models are almost always evaluated using an exponential moving average of their weights over training, which smooths out noise in the optimisation and improves samples. The length of the averaging window is an important hyperparameter, and previously had to be fixed before training. Karras and colleagues showed how to store a modest number of snapshots during training and reconstruct, after the fact, the average for any window length, allowing that hyperparameter to be tuned without retraining. They found that the optimal window was quite sensitive to model size and training configuration, which helps explain inconsistent results in earlier work. Representation as a modelling choice The lesson of this chapter is that the score must be learned somewhere, and where it is learned is a modelling choice with large consequences. In pixel space, the diffusion model carries the full burden of perceptual detail and must be very large or run in cascades. In a compressed latent space, it is relieved of that burden but inherits the limitations of the autoencoder, and the quality ceiling is set by reconstruction fidelity. Simple diffusion and some later pixel-space Transformers showed that the pixel route remains viable when schedules and architectures are adapted to it. Latent routes have dominated because they are cheaper, and because the autoencoder's compression removes redundancy that the diffusion model would otherwise have to process at every step. The choice becomes more pressing still when the data is audio or video, where the raw representation is either very long or very large, and where the structure that matters for fidelity differs from that of still images. The next chapter follows diffusion into those domains. Chapter 10. Beyond Still Images: Audio and Video Nothing in the mathematics of the previous chapters refers to images. The forward process adds Gaussian noise to a vector; the network estimates the score of the noisy distribution; the sampler integrates a differential equation. Any data that can be represented as a vector of real numbers can, in principle, be modelled this way. Audio is a long one-dimensional signal; video is a sequence of images with a time axis. Both have been modelled with diffusion with great success, and the systems that now generate convincing speech, music and moving pictures are built on exactly the score machinery described so far. What differs is the structure of the data, the representation in which the score is learned, and the particular way that fidelity fails. Those differences are instructive, because they show which parts of the framework are general and which were tuned, often implicitly, to still images. Audio as a signal A second of audio at the CD sampling rate of 44.1 kilohertz is 44,100 numbers per channel. Ten seconds of stereo music is close to a million values. Raw waveforms have structure on many time scales at once: individual oscillations last fractions of a millisecond, notes and phonemes last tens to hundreds of milliseconds, phrases and bars last seconds. The human ear is extraordinarily sensitive to some errors, such as clicks, metallic artefacts and small timing irregularities, and insensitive to others, such as absolute phase in many contexts. Fidelity in audio means something different from fidelity in images, and the representation has to respect that. The first successful diffusion models for audio were vocoders: models that convert a compact intermediate representation, typically a mel spectrogram, into a waveform. The mel spectrogram describes how energy is distributed across frequency bands over time, on a frequency scale approximating human pitch perception, and discards the phase information needed to reconstruct the exact waveform. Generating a waveform from a spectrogram is a conditional generation problem with a lot of residual uncertainty, which suits a diffusion model well. Two papers appeared almost simultaneously and were published in 2021. Nanxin Chen and colleagues' WaveGrad conditioned on mel spectrograms and estimated the gradient of the log-density of waveforms, explicitly in the score framework, and showed that good speech quality could be obtained with as few as six refinement steps when the noise schedule was carefully tuned for that number. Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao and Bryan Catanzaro's DiffWave used a network of dilated convolutions, an architecture whose receptive field grows exponentially with depth and which had been developed for autoregressive audio models, and showed that it could serve both as a vocoder and as an unconditional generator of short speech clips. These vocoders demonstrated a property that matters greatly for audio: diffusion is non-autoregressive. Earlier high-quality neural audio models generated one sample at a time, each conditioned on all previous ones, which made generation inherently sequential and slow. A diffusion vocoder generates all samples in parallel at each step, so a few steps of a parallel network can replace tens of thousands of sequential evaluations. Latent audio diffusion and text-to-audio For generation from text, the latent diffusion strategy of Chapter 9 transferred directly. Haohe Liu and colleagues' AudioLDM, in 2023, trained a variational autoencoder on mel spectrograms, ran a diffusion model in its latent space, and used a vocoder to turn the decoded spectrogram into a waveform. For conditioning it used CLAP, a model trained to align audio and text descriptions in the way CLIP aligns images and captions. Because CLAP embeds audio and text into a shared space, AudioLDM could be trained using audio embeddings and then conditioned on text embeddings at generation time, reducing its dependence on large sets of paired audio and text. An informal experiment from late 2022, Riffusion, made the image analogy literal: its authors fine-tuned Stable Diffusion to generate spectrogram images and converted them to sound. That it worked at all shows how general the framework is; that it was limited in quality and duration shows how much is lost when audio is treated as a picture. Later music systems operated on latents learned directly from waveforms rather than from spectrograms. Zach Evans and colleagues at Stability AI described Stable Audio in 2024 as a latent diffusion model over a compressed representation of 44.1 kilohertz stereo audio, produced by a fully convolutional autoencoder, with a diffusion network conditioned on text embeddings and on timing information: the start time and total duration of the desired clip. Timing conditioning addressed a real problem. Music training data comes in clips cut from longer pieces, and a model that does not know whether its output is the beginning, middle or end of a piece produces awkward openings and endings. Conditioning on timing lets the user request complete pieces of a specified length with natural structure. What fails in audio, and why Audio diffusion models fail in recognisable ways that map onto the theory. Few-step sampling in audio produces broadband hiss and a slightly rough texture, the audible counterpart of the blurriness of an under-integrated image: the sampler has not visited enough low-noise levels to remove the last of the noise, and the residual sits across all frequencies. Excessive guidance in text-to-audio produces brittle, over-emphasised timbres and loss of dynamic range, the audible counterpart of oversaturated colour. And because the latent autoencoder sets the ceiling on fidelity, as in images, a poor audio autoencoder produces artefacts no diffusion model can fix: smeared transients, where a drum hit loses its attack, or phase artefacts that sound metallic. Long-range structure is the hardest problem. A diffusion model can generate a coherent ten-second clip because its network sees the whole clip at once. Generating a three-minute song with verses that return, a chorus that repeats and a key that is maintained requires the network to represent structure over millions of samples, or tens of thousands of latent frames, and coherent long-range structure remains an open challenge. Approaches include generating at multiple time scales, conditioning on structural plans, and extending clips with overlapping windows, each with trade-offs between coherence and computational cost. Video as images with time Video adds a time axis to images, and the result is both easier and harder than it appears. Easier, because adjacent frames are extremely similar: most of a video's information is in its first frame and in the motion thereafter. Harder, because the data is enormous, and because human viewers are acutely sensitive to temporal inconsistency. An image model that produces a convincing face is doing well; a video model whose face subtly changes identity between frames, whose hands gain and lose fingers, or whose background objects flicker in and out of existence has failed, even if every individual frame is excellent. Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi and David Fleet published the first major paper on video diffusion in 2022. Their model extended the image U-Net to video by making it factorised in space and time: each block applied spatial operations within each frame and then attention along the time axis at each spatial location. This factorisation kept the cost manageable while letting information flow between frames. They trained jointly on videos and on still images, treating images as single-frame videos, which let the model benefit from far larger image datasets. And they introduced reconstruction guidance for extending videos and increasing their resolution: to generate frames conditioned on existing ones, they adjusted the score with the gradient of a reconstruction error between the model's prediction and the known frames, a direct application of the Bayesian score decomposition of Chapter 5. Later in 2022, the same group described Imagen Video, a cascade of seven diffusion models, a base model followed by spatial and temporal super-resolution stages, which together generated high-definition video from text. It used velocity prediction throughout, the parameterisation of Chapter 6, and the authors reported that it was important for avoiding colour shifts and temporal artefacts at high resolution. They used progressive distillation from Chapter 7 to accelerate sampling. Meta's Make-A-Video, from Uriel Singer and colleagues in the same year, took a complementary approach, extending a pretrained text-to-image model with temporal layers trained on unlabelled video, on the reasoning that text-image pairs teach what the world looks like and unlabelled video teaches how it moves. Latent video diffusion As with images, high-resolution video became practical through latent diffusion. Andreas Blattmann and colleagues described in 2023 how to turn a pretrained latent image diffusion model into a video model by inserting temporal layers and fine-tuning them on video while keeping or lightly adjusting the spatial layers. Their later Stable Video Diffusion work, released in late 2023, emphasised that data curation, filtering training videos for motion, quality and caption accuracy, mattered as much as architecture. Video autoencoders introduced their own design questions. Encoding each frame independently with an image autoencoder ignores redundancy across time and produces flicker, because small differences in encoding between frames become visible as temporal noise. Later systems use autoencoders that compress in time as well as space, so that each latent location represents a small spatiotemporal block of pixels. Some use causal designs in which each latent depends only on current and past frames, which allows the first frame to be encoded independently and lets the same autoencoder handle both images and video. Spacetime patches and Diffusion Transformers for video In February 2024 OpenAI published a technical report describing Sora, a text-to-video system. The report did not give full details, but described the core design: video is compressed by a learned network into a latent representation that is reduced in both space and time, the latent is divided into spacetime patches that serve as tokens, and a diffusion Transformer, in the sense of Chapter 9, is trained to denoise sequences of such patches. Because Transformers operate on sequences of arbitrary length, the same model can be trained on and generate videos of varying durations, resolutions and aspect ratios, simply by varying the number and arrangement of patches. The report emphasised that sample quality improved markedly with training compute and that properties such as three-dimensional consistency and object permanence emerged at scale without being explicitly built in, though it also acknowledged failures in simulating basic physical interactions. Other large video systems followed similar principles. Meta's Movie Gen, described in a detailed report in October 2024, used a Transformer of around 30 billion parameters operating on the latents of a spatiotemporal autoencoder, trained with flow matching, the close relative of diffusion described in Chapter 11, and paired it with a separate model for generating synchronised audio. The convergence of large systems on Transformer backbones, spacetime latents and flow-matching or diffusion objectives reflects the predictable scaling that Peebles and Xie observed for images. Temporal coherence as a score problem The central difficulty of video, temporal coherence, can be framed in the language of this book. A video model learns the score of a joint distribution over all frames at once. If the network's receptive field covers all frames and its capacity suffices, the learned score encodes the dependencies between frames, and samples are coherent. Coherence failures arise when the joint score is learned poorly along the temporal directions: when the network effectively treats frames as nearly independent, or when the latent representation encodes frames in ways that introduce independent errors. Several practical remedies follow from that framing. Full spatiotemporal attention, rather than factorised attention, lets every patch attend to every other, at quadratic cost in sequence length, which improves the network's ability to model long-range temporal dependencies. Shifting the noise schedule toward higher noise, as Chapter 6 explained, ensures that the model sees enough truly uninformative states to learn the global structure of motion, since averaging across frames otherwise leaves too much signal at nominally high noise. And joint training on images and videos ensures that the spatial component of the score is learned well even when video data is scarce. Long video remains hard, for the same reasons as long audio. Generating a minute of coherent video requires the network to hold thousands of frames, or their latents, in a single context, or to generate in segments while maintaining consistency across segment boundaries. Boyuan Chen and colleagues proposed in 2024 a technique they called diffusion forcing, which trains a model with independent noise levels for each frame, so that at generation time it can keep past frames clean, the current frame partly noisy and future frames fully noisy, combining the stability of diffusion with the open-ended extension of autoregressive models. This is a natural generalisation of the diffusion framework: the forward process need not apply the same noise level to every coordinate, and a network trained on varying per-token noise levels learns a richer family of conditional scores. Measuring fidelity across modalities The question of what counts as high fidelity takes different forms in each modality, and so does measurement. For images, the Fréchet Inception Distance, introduced by Martin Heusel and colleagues in 2017, compares the statistics of features extracted by a pretrained image classifier on real and generated images; it is sensitive to both quality and diversity but depends heavily on the choice of feature network, and Karras and colleagues have noted its limitations at the high end of quality. For audio, the analogous Fréchet Audio Distance, introduced by Kevin Kilgour and colleagues in 2019, uses features from a pretrained audio classifier. For video, the Fréchet Video Distance of Thomas Unterthiner and colleagues, from 2018, uses features from a video action-recognition network. Text alignment is often measured with CLIP or CLAP similarity between prompts and outputs. In every modality, automated metrics are supplemented and frequently overruled by human preference studies, because the features that matter to human perception are not fully captured by any fixed network. Table 5 summarises how the same framework adapts across the three modalities. Table 5. How diffusion adapts across images, audio and video. Aspect Images Audio Video Typical representation Spatial latent from image autoencoder Waveform latent or mel-spectrogram latent Spatiotemporal latent compressed in space and time Common backbone U-Net, then Diffusion Transformer Dilated convolution, U-Net or Transformer Factorised U-Net, then spacetime Transformer Key conditioning Text via cross-attention or joint attention Text via CLAP or T5; timing Text; first frame; previous segments Signature failure Oversaturation, malformed fine detail Hiss, smeared transients, weak long-range form Flicker, identity drift, implausible physics Standard automated metric Fréchet Inception Distance Fréchet Audio Distance Fréchet Video Distance What the modalities reveal Following diffusion into audio and video confirms the central claim of this book and qualifies it. The confirmation is that the same mathematical machinery, score estimation across noise levels and reverse-time integration, works across very different kinds of data with minimal change. The qualification is that the parts of a diffusion system that were tuned for images, the schedule, the representation, the architecture's inductive biases, and the metric of success, must all be reconsidered for each new modality. Fidelity in audio is about transients and long-range form; fidelity in video is about temporal coherence and plausible motion. In each case the score must be learned in a representation that makes the relevant structure accessible to the network, and the schedule must destroy and restore that structure in the right order. Chapter 11. Flow Matching and Rectified Flow: The Same Object in Different Coordinates Between 2022 and 2024 a family of methods appeared that seemed, at first, to offer an alternative to diffusion. Flow matching, rectified flow and stochastic interpolants were presented in the language of continuous normalising flows and optimal transport rather than of noise and scores. They trained a network to predict a velocity rather than a noise vector, and generated by integrating an ordinary differential equation from noise to data. Several of the largest generative systems released since 2024, including Stable Diffusion 3 and Meta's Movie Gen, describe themselves as trained with flow matching or rectified flow rather than as diffusion models. A reader of the earlier chapters will recognise the ingredients immediately. The aim of this chapter is to make the relationship precise: flow matching with Gaussian noise is diffusion in a particular coordinate system with a particular weighting, and the reasons it has been adopted say a great deal about which features of the framework matter in practice. Continuous normalising flows A normalising flow is a generative model built from an invertible map between a simple base distribution, typically a standard Gaussian, and the data distribution. If the map is known and its Jacobian determinant can be computed, the density of any data point can be evaluated exactly by the change-of-variables formula. Discrete normalising flows compose many invertible layers, each designed to have a cheap Jacobian, and their architectural constraints historically limited their quality. Continuous normalising flows, introduced by Ricky Chen, Yulia Rubanova, Jesse Bettencourt and David Duvenaud in 2018 under the name neural ordinary differential equations, define the map instead as the solution of an ODE, dx/dt = v_θ(x, t), integrated over a fixed time interval. Any sufficiently smooth velocity field defines an invertible map, so the network v_θ can be an unconstrained neural network. The log-density changes along the flow at a rate given by the divergence of the velocity, as Chapter 5 described for the probability flow ODE. The difficulty was training. Maximum-likelihood training of a continuous flow requires solving the ODE during every training step, which is slow, and the learned velocity fields tended to produce winding, hard-to-integrate trajectories. The probability flow ODE of a diffusion model is a continuous normalising flow. Its velocity field is defined by the forward process's drift and the score, and its training, by denoising score matching, never requires solving an ODE. This is the observation that flow matching generalises. Flow matching Yaron Lipman, Ricky Chen, Heli Ben-Hamu, Maximilian Nickel and Matt Le introduced flow matching in a paper first released in 2022 and published at ICLR in 2023. The idea parallels denoising score matching closely. One would like to train v_θ to match the velocity field u(x, t) of some chosen probability path pₜ connecting noise to data. That marginal velocity is unknown, for the same reason the marginal score is unknown: it is an average over all data points that could have led to x. But one can define the path as a mixture of simple conditional paths, one per data point, each with a known conditional velocity. Lipman and colleagues showed that regressing the network onto the conditional velocity, averaged over data points and times, has the same gradients as regressing onto the marginal velocity. This is the exact analogue of Vincent's result from Chapter 2, with velocities in place of scores. They called the practical objective conditional flow matching. The simplest choice of conditional path is a straight line from a noise sample to a data sample. Writing the path as an interpolation between the clean data x₀ and Gaussian noise ε, with t running from zero at the data to one at the noise, the noisy state is xₜ = (1 − t) · x₀ + t · ε, and its velocity along the path is the constant vector ε − x₀. The flow matching loss is then L_FM = E over t, x₀, ε of ‖ v_θ(xₜ, t) − (ε − x₀) ‖². To generate, start with a sample of ε at t = 1 and integrate dx/dt = v_θ(x, t) backward to t = 0. Lipman and colleagues called this conditional path the optimal transport path, because for each data point it is the straight-line, constant-speed interpolation that optimal transport between two points would prescribe. Independently and at the same time, Xingchao Liu, Chengyue Gong and Qiang Liu proposed rectified flow, which uses the same linear interpolation and the same regression objective, motivated by the goal of learning transport maps whose paths are as straight as possible. Michael Albergo and Eric Vanden-Eijnden's stochastic interpolants, also from 2022, gave a general framework in which data and noise are connected by an arbitrary interpolation, possibly with additional noise, and showed how to recover both deterministic and stochastic generative processes from the learned quantities. The three formulations overlap heavily and are often referred to collectively. The translation into diffusion language Now compare the flow matching path with the forward process of a diffusion model. Every Gaussian diffusion has noisy states of the form xₜ = aₜ · x₀ + bₜ · ε, with some signal coefficient aₜ and noise coefficient bₜ. The variance-preserving process has aₜ² + bₜ² = 1; the variance-exploding process has aₜ = 1. The rectified flow path has aₜ = 1 − t and bₜ = t. It is a Gaussian diffusion path with a particular schedule. Its signal-to-noise ratio is ((1 − t) / t)², running from infinity at t = 0 to zero at t = 1. In particular, it reaches zero SNR exactly, avoiding the terminal-SNR flaw of Chapter 6 by construction. The flow matching target, the velocity ε − x₀, is likewise an affine combination of the noise and the clean data, like every other parameterisation in Table 1. Given the noisy state and the time, the velocity determines x₀ and ε and therefore the score, through Tweedie's formula. In general, for any path aₜ x₀ + bₜ ε, the marginal velocity of the probability flow is v(x, t) = (aₜ′ / aₜ) · x + bₜ · ( aₜ′ · bₜ / aₜ − bₜ′ ) · ∇ log pₜ(x), where the primes denote derivatives with respect to t. The velocity is a known linear function of the current state and the score. A network that predicts the velocity is a network that predicts the score, rescaled and offset by quantities fixed by the schedule. And the ODE that flow matching integrates is the probability flow ODE of the corresponding diffusion. So the differences between flow matching with Gaussian noise and diffusion training are exactly the three choices separated in Chapter 6. The path, linear interpolation, is a particular schedule. The target, the constant velocity, is a particular parameterisation. And the loss, uniform squared error in velocity with t drawn from some distribution, corresponds to a particular weighting across noise levels. Ruiqi Gao, Emiel Hoogeboom, Jonathan Heek, Valentin De Bortoli, Kevin Murphy and Tim Salimans laid this correspondence out in detail in a 2024 analysis titled, aptly, as two sides of the same coin. They showed that the weighting implied by flow matching is close to the weighting implied by velocity prediction in variance-preserving diffusion, and that, given the same schedule, weighting and sampler, the two approaches produce the same model. Kingma and Gao's result from Chapter 4, that monotone weightings correspond to ELBOs under noise augmentation, applies to flow matching losses too. Why the coordinates matter anyway If flow matching is diffusion in different coordinates, why has it been so widely adopted? There are several genuine reasons, each a matter of practice rather than principle. The first is simplicity. The flow matching formulation needs no SDE, no Fokker–Planck equation, no score, and no variance schedule with its products of α. The noisy state is a straight-line mix of data and noise; the target is their difference; the sampler is an ODE solver. Engineers can implement and reason about it without the machinery of the earlier chapters. That simplicity has real value in large engineering projects, where fewer moving parts mean fewer bugs. The terminal-SNR flaw of Chapter 6, for example, cannot occur. The second is straightness. The conditional paths are straight lines, and the marginal paths, though not straight, tend to be straighter than those of the variance-preserving process with the cosine schedule. In Chapter 5 the EDM parameterisation was seen to produce nearly straight trajectories over most of their length; the linear interpolation path behaves similarly, and for the same reason. Straighter paths need fewer solver steps, which matters for sampling cost. The third is the training noise distribution. Esser and colleagues, training Stable Diffusion 3, compared many combinations of path and timestep distribution under matched conditions and found that the rectified flow path, combined with a logit-normal distribution over t that concentrates training on the middle of the path, performed best among the formulations they tested, particularly at low step counts. Since the path and weighting are coordinate choices, this is a finding about which choices work well rather than about a different kind of model, but it is a valuable finding, and it was obtained in the flow matching formulation. The fourth is generality. Flow matching does not require the starting distribution to be Gaussian. One can learn a flow between any two distributions from which one can sample, such as from one image domain to another, or from a distribution of low-resolution images to high-resolution ones. And the conditional paths need not be straight lines; any path between paired samples will do. When the source is not Gaussian, the connection to score-based diffusion breaks, and flow matching becomes a genuinely more general tool. Much current research exploits that generality, though the large-scale generative systems still mostly use Gaussian noise as the source. Reflow and the straightening of paths Rectified flow added one idea that has no direct counterpart in standard diffusion. After training a first flow model, one can generate pairs consisting of a noise sample and the data sample that the model's ODE maps it to. These pairs are coupled: each noise vector is matched with a specific output. Training a second flow model on straight lines between these coupled pairs, rather than between independently drawn noise and data, produces a model whose marginal paths are much straighter, because the coupling removes the crossing of paths that forces curvature. Liu, Gong and Liu called this procedure reflow and showed that repeated reflow makes the trajectories nearly straight, so that one or a few Euler steps suffice. The underlying observation is worth stating. In independent-coupling flow matching, many conditional straight lines pass through each point at each time, heading toward different data samples, and the marginal velocity is their average. Where conditional paths cross, the averaged velocity bends, and integrating a bent velocity field requires many steps. Reflow replaces the independent coupling with a deterministic one in which paths do not cross, and the average of non-crossing lines is itself nearly a line. This is closely related to optimal transport, which seeks the coupling that minimises total transport cost and whose paths, for quadratic cost, do not cross. Variants that approximate optimal-transport couplings within each training minibatch, proposed by Alexander Tong and colleagues and by Aram-Alexandre Pooladian and colleagues in 2023, aim for straighter paths without a separate reflow stage. Reflow is thus a form of distillation, closely related in purpose to the methods of Chapter 7, and InstaFlow, from Xingchao Liu and colleagues in 2023, applied it to Stable Diffusion to obtain one-step text-to-image generation. Stochastic sampling from flow models Since a flow model's velocity determines the score, one can recover the reverse SDE from it and sample stochastically, just as with a diffusion model. Albergo and Vanden-Eijnden's stochastic interpolant framework makes this explicit: given the learned velocity and the learned score, which can be derived from each other in the Gaussian case, one can sample with any amount of added noise along the continuum of Chapter 5, from the deterministic ODE to fully stochastic SDEs. In practice most flow-matching systems sample deterministically, because their paths are straight enough to make that efficient, but the stochastic option is available and sometimes improves quality when the learned velocity has errors, for the reasons Chapter 3 gave. Guidance carries over unchanged. Classifier-free guidance applied to velocities, extrapolating from the unconditional velocity through the conditional one, is the same affine operation on scores as in Chapter 8, because velocity and score are affinely related at each time. The same benefits follow, and the same failures at high scale. What the convergence means The convergence of diffusion models and flow matching is a strong piece of evidence for the central claim of this book. Two research communities, one starting from noise and denoising and the other from flows and optimal transport, arrived at the same object: a learned vector field over a family of distributions interpolating between data and Gaussian noise, from which samples are drawn by integrating a differential equation. The diffusion community calls the vector field a score and the flow community calls it a velocity; they are related by a fixed linear transformation. The choices that genuinely distinguish practical systems are the path, the prediction target, the weighting over noise levels and the sampler, and these are choices that can be made, and compared, within either formulation. A practitioner choosing between them is therefore choosing a vocabulary and a set of defaults, not a different kind of model. The flow matching vocabulary is simpler and its defaults are good. The diffusion vocabulary connects more directly to the probabilistic theory of the earlier chapters: the ELBO, Langevin dynamics, Tweedie's formula, the reverse-time SDE. Each offers insight the other obscures, and the most productive work of recent years has moved freely between them. Chapter 12. Memorisation and Generalisation: What the Learned Score Knows Every chapter so far has treated the score as the target and the network as an approximation to it. That framing hides a puzzle that sits at the foundation of the subject. The true score of the data distribution is not available to the training procedure. What is available is a finite training set, and the objective that denoising score matching minimises is defined over that finite set. If a network minimised it perfectly, it would learn the score of the training set's empirical distribution, a collection of isolated points, and a sampler using that score would reproduce training examples exactly. It would not generate anything new. That diffusion models do generate novel images, sounds and videos is therefore not a consequence of the mathematics of score matching alone. It is a consequence of the network failing to minimise the objective perfectly, in a particular and fortunate way. Understanding that failure is essential to understanding what these systems produce, when they copy, and why. The optimal denoiser for a finite dataset Suppose the training set consists of N images, x⁽¹⁾ through x⁽ᴺ⁾, each equally likely. The empirical distribution places mass 1/N on each. Adding Gaussian noise of level σ turns it into a mixture of N Gaussians, each centred on a training image with standard deviation σ. The score of that mixture can be written down exactly. By the reasoning of Chapter 2, the optimal denoiser, the posterior mean of the clean image given the noisy one, is D*(x, σ) = Σᵢ wᵢ(x) · x⁽ⁱ⁾, with wᵢ(x) proportional to exp( −‖x − x⁽ⁱ⁾‖² / (2σ²) ). The optimal denoiser is a weighted average of the training images, with weights given by a softmax over negative squared distances. At high noise the weights are nearly uniform, and the denoiser returns roughly the mean of the dataset. As the noise decreases, the softmax sharpens, and the weight concentrates on the training images nearest to the current state. At low noise, all the weight falls on a single training image, and the denoiser returns it exactly. A sampler driven by this optimal denoiser behaves predictably. It starts from noise, moves toward the dataset mean, then gradually commits to whichever training image the noisy state happens to be nearest when the softmax sharpens, and arrives at that image. Every sample is a training example. The model has memorised perfectly, and in terms of the training objective it is optimal. Any loss lower than that of this denoiser is impossible; any network that achieves it reproduces the training set and nothing else. This is not a hypothetical worry. For small datasets, trained diffusion models do approach this behaviour. A model with ample capacity trained on a few hundred or a few thousand images will, given enough training, generate near-copies of training examples. Its loss falls toward the empirical optimum, and its outputs converge on the training set. Evidence of copying in large models For large models trained on hundreds of millions or billions of images, copying is rarer, but it occurs. In 2023 Nicholas Carlini and colleagues, in a paper titled Extracting Training Data from Diffusion Models, generated large numbers of images from Stable Diffusion and Google's Imagen using captions from the training data, and searched for generations that were nearly identical to training images. They extracted over a thousand training examples across the models they studied, including identifiable photographs of people and copyrighted artwork. The extracted images were overwhelmingly ones that appeared many times in the training data, duplicated across the web, so that the model had effectively seen them hundreds or thousands of times. Gowthami Somepalli and colleagues, in a contemporaneous study titled Diffusion Art or Digital Forgery, found that Stable Diffusion sometimes reproduced substantial parts of training images, such as distinctive compositions or backgrounds, even when it did not copy whole images, and that duplication in the training data and highly specific captions made replication more likely. These findings fit the theory. A training image duplicated many times is a heavy point mass in the empirical distribution. Its basin under the optimal denoiser is wide, the network sees it often, and fitting it closely reduces the loss substantially. A caption that uniquely identifies such an image conditions the model onto a narrow region where that image dominates. Guidance, which sharpens conditional distributions, makes the effect stronger. Deduplicating the training data, which both papers recommended, reduces the risk, and later training pipelines adopted deduplication widely. Why large models generalise The more interesting question is why, for large datasets, models mostly do not copy. If the empirical optimum is memorisation, and networks are trained to approach the optimum, what stops them? A careful set of experiments by Zahra Kadkhodaie, Florentin Guth, Eero Simoncelli and Stéphane Mallat, published at ICLR in 2024, gave a clear answer. They trained two diffusion models of identical architecture on two non-overlapping subsets of a face dataset and compared their outputs given the same starting noise. When the subsets were small, each model memorised its own training set, and the two produced completely different images, each a copy of an example from its own subset. As the subsets grew, the models' outputs began to converge. With sufficiently large subsets, the two models, trained on disjoint data, produced nearly identical images from the same noise, and those images were not copies of anything in either training set. The two models had learned the same function, a function that depended on the underlying distribution of faces rather than on the particular faces each had seen. That is what generalisation means here, and it can only happen if the network's inductive biases steer it toward a particular kind of score among the many that fit the data reasonably well. Kadkhodaie and colleagues analysed the learned denoisers and found that they act, locally, like adaptive shrinkage in a basis tailored to the image: the denoiser keeps components aligned with the image's geometric features, such as edges and smooth contours, and suppresses others. They described these as geometry-adaptive harmonic bases. The network does not learn to recall training images; it learns a way of denoising that respects the structure of natural images in general, and that way of denoising, applied repeatedly through the sampler, produces images with natural structure that are not in the training set. There is a useful way to see why this inductive bias leads away from memorisation. The memorising denoiser is a highly non-local function: its output at any point depends on the distances to every training image, computed over the whole image at once. Convolutional and attention-based networks, with their emphasis on local features composed hierarchically, find such functions hard to represent when N is large. What they represent easily is a denoiser built from local patterns that recur across images, such as textures, edges, parts of objects and their typical arrangements. Such a denoiser cannot reproduce any particular training image unless that image is heavily repeated, but it can produce many new images that share the training set's local statistics and compositional regularities. The generalisation of diffusion models is in this sense a product of what their networks find easy to learn, combined with a training set too large to memorise within that capacity. The middle ground Between pure copying and full generalisation lies a broad middle ground, and much of what diffusion models produce falls in it. A model may generate a novel image whose individual elements, a particular style of brushwork, a characteristic composition, a specific product design or a logo, closely follow particular training examples. It may produce outputs that are new as wholes but that combine recognisable fragments. Somepalli and colleagues' finding of partial replication is an instance of this middle ground. Whether such outputs count as copying in a legal or ethical sense is a question outside mathematics. What the mathematics says is that the degree of reliance on specific examples depends on how heavily those examples weigh in the empirical distribution, how specifically the conditioning picks them out, how strongly guidance concentrates the distribution, and how much capacity the network has relative to the diversity of the data. Each of these is a lever. Training data curation and deduplication reduce the weight of particular examples. Caption design affects how specifically prompts can address them. Guidance strength, as Chapter 8 showed, directly controls concentration. And the ratio of model capacity to data diversity determines where on the spectrum from memorisation to generalisation a model sits. Studies of the transition, including Kadkhodaie and colleagues' work, suggest that it is fairly sharp: below a certain amount of training data for a given model, memorisation dominates; above it, generalisation does. Generalisation and the quality of the score The memorisation question also sheds light on what it means for a learned score to be good. In earlier chapters, error in the score was treated as a deviation from the true score of the data distribution, and the goal of training was to reduce it. But the true data distribution is never observed. A model whose score exactly matched the empirical distribution would be perfectly fitted and useless. A model whose score matched the underlying distribution from which the training set was drawn would be ideal, but nothing in the training objective directly targets that distribution; only the network's biases and the size of the training set push the learned score toward it. This suggests a reinterpretation of several practical observations. Exponential moving averages of the weights, which almost all diffusion models use and whose length Karras and colleagues found to matter so much, average the network over a stretch of training and thereby smooth the learned function, which may favour generalisation over fitting of individual examples. The training noise distribution emphasises intermediate noise levels, where the empirical and underlying distributions are hardest to distinguish and the network's generalising biases matter most; at very low noise, where the empirical optimum is memorisation, the training effort is deliberately small. Autoguidance, which extrapolates away from a weaker model, can be read as amplifying the difference between a model that generalises well and one that generalises less well, an idea that is still being worked out theoretically. Implications for use Three practical conclusions follow. First, the outputs of a diffusion model are not retrievals but are not guaranteed to be novel either. The probability of reproducing a specific training example rises with that example's duplication, the specificity of the prompt and the guidance strength, and those who deploy such models can manage the risk with data curation and with checks on outputs. Second, the same mechanisms that make a model generalise also make it produce plausible but fabricated detail: a generalised denoiser fills in what images of this kind usually contain, which is why diffusion-based restoration of degraded photographs or medical scans can produce convincing structure that was never present in the original. For applications where fidelity to a specific underlying truth matters, such as medical imaging or forensic enhancement, the output of a diffusion model is a sample from a plausible posterior, not a measurement. Third, evaluation metrics that compare generated outputs with training data, such as FID computed against the training set, cannot by themselves distinguish generalisation from memorisation, since a memorising model scores very well on them. Assessing novelty requires separate tests, such as nearest-neighbour searches against the training data. The score, then, is learned from finite data by a network with particular biases, and those biases are as much a part of the model as the training objective. The mathematics of the previous chapters describes what happens given a score; this chapter describes where the score comes from, and why the score a network learns is, fortunately, not the one its objective literally asks for. Conclusion The chapters of this book have followed one object through a series of transformations. The score of a noise-perturbed distribution appeared first as a way to escape the normalisation problem, then as the target of a regression that turned out to be ordinary denoising, then as the drift of a Langevin sampler, then as a reweighted piece of a variational bound, then as the unknown term in Anderson's reverse-time equation and in the probability flow ODE. It was rescaled by preconditioning, reweighted by noise schedules, integrated by high-order solvers, distilled into a few steps, extrapolated by guidance, relocated into latent spaces, extended across time for audio and video, renamed a velocity by flow matching, and finally revealed to be a function shaped as much by a network's biases as by its training objective. With all of that in view, several conclusions can be drawn that are not visible from any single chapter. The design space is smaller than the literature suggests The first conclusion is that the space of genuinely distinct choices in a diffusion system is small. Reading the literature chronologically gives an impression of a fast-moving field with a new method every few months: DDPM, NCSN, improved DDPM, score SDEs, DDIM, EDM, v-prediction, DPM-Solver, consistency models, rectified flow, flow matching. Read through the lens of the score, most of these occupy positions in a space with only a handful of axes. There is the path from data to noise, which matters mainly through its range of signal-to-noise ratios and its endpoint. There is the prediction target, an affine combination of the clean data and the noise chosen for numerical conditioning. There is the weighting over noise levels, which decides where the network's capacity goes and which, under monotonicity, is always a likelihood under some noise augmentation. There is the integrator and its stochasticity. There is the representation in which all of this happens. And there is the modification of the score at sampling time, through guidance. A new method that does not change one of these is a renaming; one that does can be assessed by asking which axis it moves along and what that axis is known to trade. This matters practically. A team adopting flow matching because a competitor did, or switching from noise prediction to velocity prediction because a recent paper recommends it, should know that it is making a choice about conditioning and weighting rather than a choice about the kind of model it is building, and should expect gains or losses of a corresponding size. Conversely, decisions that look like housekeeping, such as whether the schedule reaches zero SNR or whether the schedule is shifted for resolution, belong on the same short list of consequential choices, and the history of the field shows that getting one wrong can introduce a systematic flaw that no amount of training removes. The interesting errors are no longer in the score The second conclusion concerns where errors come from. For several years the limiting factor in diffusion models was the accuracy of the learned score. Better architectures, better preconditioning and more compute steadily improved it, and the convergence theory of Chapter 5 says that, with an accurate score, sampling error can be made small with modest numbers of steps. For the largest current systems, the score is accurate enough that the most visible failures arise elsewhere: in the autoencoder that sets the ceiling on fine detail, in the text encoder that determines how well prompts are understood, in the guidance that distorts the sampled distribution on purpose, and in the training data that determines what the score is a score of. Guidance is the clearest case. Almost every widely seen output of a text-to-image, text-to-audio or text-to-video system has been sampled with classifier-free guidance at a scale well above one. Those outputs come from a distribution deliberately sharpened toward the prototypical, by an operation that does not correspond to sampling any well-defined distribution, and whose characteristic failures, oversaturation and loss of diversity, are a direct consequence of that departure. The methods of the last few years that work best, such as guidance limited to an interval of noise levels or guidance by a weaker model, succeed by making the departure smaller or better targeted. A reasonable expectation is that the next improvements in perceived quality will come less from estimating the score more precisely and more from understanding, and redesigning, the ways it is deliberately bent. Fidelity is a property of the whole pipeline The third conclusion concerns the word fidelity itself. A high-fidelity output, in ordinary usage, is one that looks or sounds real. In a diffusion system that property is produced jointly by every component discussed in this book. The schedule and weighting decide which scales of structure are learned well. The sampler decides how faithfully the learned field is followed. Guidance decides how far the output is pushed toward the typical. The autoencoder decides which details can be represented at all. The network's biases decide whether the output is a new member of the distribution or an echo of a training example. When a generated video shows a hand with six fingers or a face that drifts between frames, or a generated song loses its structure after twenty seconds, the fault may lie in any of these, and diagnosing it requires knowing which part of the pipeline is responsible for which property. The mathematics gives the map. That map also explains why fidelity should not be confused with truth. A diffusion model used to restore a damaged photograph, sharpen a low-dose medical scan or extend a piece of audio produces a sample from a posterior over plausible completions. When the measurement leaves much uncertain, the posterior is broad, and the sample commits to one possibility among many, with all the confident detail that a sharp sample carries. The appearance of detail is evidence that the model knows what such images usually look like, not that it knows what this one looked like. Anyone deploying diffusion models where a single true answer matters should treat their outputs as hypotheses, and where possible should draw several samples and inspect how they differ, which is a direct way of seeing the posterior's breadth. What remains unsettled Several questions remain open, and they are worth naming precisely. The theory of classifier-free guidance is incomplete: it is known what guidance does not sample, but there is no settled account of what it does sample, or of the optimal way to trade adherence against diversity. The generalisation of diffusion models is understood qualitatively, through the inductive biases of their networks, but there is no theory that predicts, for a given architecture and dataset, where the transition from memorisation to generalisation will fall, or how much of a given output's content is attributable to particular training examples. Long-range structure in audio and video, the coherence of a three-minute song or a two-minute scene, has improved with scale but remains limited, and it is not clear whether the fix is more context, hierarchical generation, or a hybrid with autoregressive modelling of the kind diffusion forcing points toward. Few-step generation has been achieved through several forms of distillation, each with some loss of diversity, and no method yet matches the quality and coverage of many-step sampling at one or two steps without trade-offs. And for discrete data such as text, where Gaussian noise is not natural, diffusion-style models built on masking and token corruption have shown real promise, but their relationship to the continuous theory developed here is less clean, and whether they will match autoregressive language models at scale is an open empirical question. What a reader should do with this For a reader who builds or evaluates these systems, the practical upshot is a habit of translation. When a new method appears, translate it into the language of the score: what distribution's score is being learned, at which noise levels, with what weighting, in what representation; how the sampler integrates it; how, if at all, it is modified at sampling time. Most methods translate cleanly, and the translation tells you what to expect. When a system fails, locate the failure on the same map. When choosing defaults, prefer schedules that reach zero SNR and are shifted for the redundancy of the representation, prediction targets that are well conditioned at both ends of the noise range, training noise distributions concentrated where the loss can still fall, deterministic high-order solvers for moderate step counts, and guidance applied with restraint and, where possible, only at the noise levels where it helps. For a reader who uses these systems or is affected by them, the upshot is a more exact sense of what they are. A diffusion model is a learned vector field over a space of noisy data, shaped by its training set and its network, walked backward from static by a numerical integrator, and pushed by guidance toward the typical. Its outputs are new more often than not, sometimes copied, always plausible within the limits of its representation, and never more than samples. The static at the beginning of every generation is not a metaphor. It is the starting point of a computation whose every step can be understood, and whose every result reflects choices that someone made. Glossary Annealed Langevin dynamics. A sampling procedure that runs Langevin dynamics on a sequence of progressively less noisy distributions, using each level's result to start the next. Classifier-free guidance. A sampling technique that extrapolates from a model's unconditional prediction through its conditional prediction by a factor greater than one, strengthening adherence to the conditioning at some cost in diversity. Denoising score matching. A training objective in which a network learns the score of a noise-perturbed distribution by regressing onto the known score of the noise added to each training example; equivalent to training a denoiser. ELBO (evidence lower bound). A tractable lower bound on the log-likelihood of a latent-variable model; for diffusion models, a weighted sum of denoising errors across noise levels. Flow matching. A training method for continuous normalising flows that regresses a network onto the velocity of simple conditional paths between noise and data; with Gaussian noise, equivalent to diffusion under a particular schedule and weighting. Forward process. The fixed, parameter-free procedure that corrupts data with progressively more Gaussian noise, defining the family of distributions a diffusion model learns to reverse. Langevin dynamics. A stochastic process that moves along the score of a distribution while injecting Gaussian noise in a fixed proportion, and whose equilibrium is that distribution. Latent diffusion. Running the diffusion process in the compressed latent space of a separately trained autoencoder rather than directly on raw pixels, waveforms or frames. Network function evaluation (NFE). One forward pass of the denoising network; the standard unit of sampling cost. Noise schedule. Loosely, the set of choices governing noise levels: the forward path, the distribution and weighting of noise levels in training, and the noise levels visited by the sampler. Probability flow ODE. A deterministic ordinary differential equation whose solutions have the same marginal distributions at every time as the forward diffusion; integrating it backward turns noise into data. Reverse-time SDE. The stochastic differential equation, given by Anderson's theorem, that runs a diffusion process backward in time; its drift involves the score of the forward process's marginals. Score. The gradient of the log-density of a distribution with respect to the data, a vector field pointing toward regions of higher probability. Signal-to-noise ratio (SNR). The ratio of the squared signal coefficient to the squared noise coefficient at a given noise level; its logarithm is a natural coordinate for the forward process. Tweedie's formula. The identity stating that the posterior mean of a clean signal given a Gaussian-noised observation equals the observation plus the noise variance times the score of the noisy distribution. Velocity prediction (v-prediction). A network parameterisation that predicts a fixed combination of noise and clean data, well conditioned at both high and low noise levels. Further Reading The papers below are the primary sources for the main ideas in this book, listed roughly in the order the book uses them. Most are freely available on arXiv. Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan and Surya Ganguli, "Deep Unsupervised Learning using Nonequilibrium Thermodynamics" (ICML 2015). The original diffusion probabilistic model, motivated by statistical physics. Worth reading for the clarity of its core idea, years before the method became practical. Aapo Hyvärinen, "Estimation of Non-Normalized Statistical Models by Score Matching" (Journal of Machine Learning Research, 2005), and Pascal Vincent, "A Connection Between Score Matching and Denoising Autoencoders" (Neural Computation, 2011). The two short papers that establish score matching and its denoising form, the theoretical basis of every diffusion objective. Yang Song and Stefano Ermon, "Generative Modeling by Estimating Gradients of the Data Distribution" (NeurIPS 2019). Noise-conditional score networks and annealed Langevin dynamics, with a lucid account of why a single noise level fails. Jonathan Ho, Ajay Jain and Pieter Abbeel, "Denoising Diffusion Probabilistic Models" (NeurIPS 2020). The paper that made diffusion competitive, with the variational derivation and the simple noise-prediction loss. Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon and Ben Poole, "Score-Based Generative Modeling through Stochastic Differential Equations" (ICLR 2021). The unifying continuous-time framework: forward and reverse SDEs, the probability flow ODE, predictor-corrector sampling and exact likelihoods. The single most important paper for the perspective of this book. Diederik P. Kingma, Tim Salimans, Ben Poole and Jonathan Ho, "Variational Diffusion Models" (NeurIPS 2021), and Diederik P. Kingma and Ruiqi Gao, "Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation" (NeurIPS 2023). The signal-to-noise view of the variational bound and the unification of weighted losses with likelihood. Tero Karras, Miika Aittala, Timo Aila and Samuli Laine, "Elucidating the Design Space of Diffusion-Based Generative Models" (NeurIPS 2022). The clearest practical treatment of preconditioning, noise distributions, samplers and stochasticity, with each choice separated and justified. Jiaming Song, Chenlin Meng and Stefano Ermon, "Denoising Diffusion Implicit Models" (ICLR 2021), and Cheng Lu and colleagues, "DPM-Solver" (NeurIPS 2022). Deterministic sampling and exponential integrators, the basis of fast training-free samplers. Prafulla Dhariwal and Alex Nichol, "Diffusion Models Beat GANs on Image Synthesis" (NeurIPS 2021), and Jonathan Ho and Tim Salimans, "Classifier-Free Diffusion Guidance" (2022). The origins of classifier guidance and classifier-free guidance. Tero Karras and colleagues, "Guiding a Diffusion Model with a Bad Version of Itself" (NeurIPS 2024). Autoguidance, and a sharp analysis of why guidance improves quality at all. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser and Björn Ommer, "High-Resolution Image Synthesis with Latent Diffusion Models" (CVPR 2022), and William Peebles and Saining Xie, "Scalable Diffusion Models with Transformers" (ICCV 2023). Latent diffusion and the Diffusion Transformer, the two pillars of current large-scale systems. Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel and Matt Le, "Flow Matching for Generative Modeling" (ICLR 2023), and Patrick Esser and colleagues, "Scaling Rectified Flow Transformers for High-Resolution Image Synthesis" (ICML 2024). Flow matching from first principles, and its large-scale use with a careful comparison of schedules and timestep distributions. Zahra Kadkhodaie, Florentin Guth, Eero P. Simoncelli and Stéphane Mallat, "Generalization in Diffusion Models Arises from Geometry-Adaptive Harmonic Representations" (ICLR 2024), and Nicholas Carlini and colleagues, "Extracting Training Data from Diffusion Models" (USENIX Security 2023). The two sides of the memorisation question: when models generalise, and when they copy. Calvin Luo, "Understanding Diffusion Models: A Unified Perspective" (2022). A long, patient tutorial that derives the variational and score-based views step by step; a good companion for readers who want every line of algebra. Hashtags: #DiffusionModels #GenerativeAI #GenerativeModeling #ArtificialIntelligence #MachineLearning #DeepLearning #MathematicalFoundations #ScoreMatching #DenoisingScoreMatching #ScoreBasedGenerativeModels #DenoisingDiffusion #DDPM #DDIM #GaussianNoise #GaussianDiffusion #ForwardDiffusion #ReverseDiffusion #NoiseSchedules #SignalToNoiseRatio #ProbabilityTheory #StochasticProcesses #StochasticDifferentialEquations #ReverseTimeSDE #ProbabilityFlowODE #LangevinDynamics #AnnealedLangevinDynamics #TweediesFormula #VariationalInference #EvidenceLowerBound #ELBO #NoisePrediction #ScoreEstimation #DenoisingNetworks #DiffusionSampling #NumericalIntegration #DiffusionSamplers #ClassifierFreeGuidance #ClassifierGuidance #LatentDiffusion #DiffusionTransformers #UNetArchitecture #FlowMatching #RectifiedFlow #VelocityPrediction #GenerativeImageSynthesis #TextToImageGeneration #AudioGeneration #VideoGeneration #GenerativeAIResearch #DeepGenerativeModels
- The Turing Test and Beyond (Measuring Machine Intelligence)
Download the Book (PDF): Introduction In the spring of 2025, two cognitive scientists at the University of California, San Diego, posted a paper with a title that would once have been the headline of the century: "Large Language Models Pass the Turing Test." Cameron Jones and Benjamin Bergen had run a carefully controlled version of the game Alan Turing described in 1950. Interrogators chatted for five minutes with two hidden partners at once, one a human and one a machine, and then said which was which. When the machine was OpenAI's GPT-4.5, instructed to adopt the persona of a slightly awkward young person fluent in internet slang, the interrogators picked it as the human 73 percent of the time. They chose the real person less often than they chose the program. The reaction was instructive. There was some excitement, but far less than the milestone would have provoked a decade earlier, and much of the commentary was a shrug. Of course a chatbot can make small talk; everyone had been using one for two years. Nobody thought that the result settled whether the machine was intelligent in the sense that mattered, still less whether it was the "artificial general intelligence" that technology companies were by then openly racing to build. The most famous test in the history of computing had been passed, and almost nobody treated the passing as an answer to anything. Eighteen months later the same pattern repeated on a very different kind of test. In March 2026 the ARC Prize Foundation launched ARC-AGI-3, a benchmark of small interactive video-game-like puzzles with no instructions, built specifically so that ordinary people could solve them and machines could not. At launch the best frontier systems scored under one percent. In September 2026 the foundation reported that a new OpenAI system, run through a harness its maker had built, scored 99.9 percent and used fewer moves than the median human tester on the overwhelming majority of levels. The foundation's own announcement was careful to say it was not claiming the system was AGI, and that saturating its benchmark would not prove AGI had arrived. This booklet is about that pattern, and about what to do with it. The argument in brief Every serious test of machine intelligence ever proposed has eventually been passed, or come close to being passed, by a system that the test's designers would not have regarded as intelligent in the way they meant. This is not because the designers were careless. Turing was one of the deepest thinkers of the twentieth century. Hector Levesque and his colleagues, who designed the Winograd Schema Challenge in 2012 as a direct improvement on Turing, thought hard about loopholes. François Chollet, who created the Abstraction and Reasoning Corpus in 2019, built his benchmark on an explicit theory of what intelligence is and why existing tests missed it. Each of them identified a real flaw in what came before, and each produced a better instrument. And each instrument, in turn, was solved by methods that exploited something other than the general capacity it was supposed to detect. The reason is structural. A test measures behaviour on a finite set of tasks. Intelligence, as nearly everyone who has thought about it agrees, is a capacity that extends beyond any finite set: the ability to handle situations one has not seen and was not prepared for. Any fixed test can therefore be met in two ways, by the general capacity or by something narrower that happens to cover the test. When the people building machines have enormous resources and a public scoreboard to aim at, the narrower route is found with remarkable reliability. The economist Charles Goodhart's observation that a measure which becomes a target ceases to be a good measure is not an occasional nuisance in AI evaluation. It is the central fact of the field. The controlling claim of this booklet follows from that. No single benchmark, however clever, can confirm that a machine has general intelligence. What can do the job, if anything can, is a standing practice of evaluation: tests that are continually renewed, that measure how efficiently a system acquires new skills rather than which skills it already has, that are run adversarially by people trying to find what the machine cannot do, and that are tied to explicit claims about what we need the word "general" to mean. Confirming AGI, on this view, is less like a finish line and more like a court case, in which the verdict depends on the accumulated failure of serious attempts to find a counterexample. Why the history matters It is tempting to treat the old tests as museum pieces. The Turing test is seventy-six years old; the Winograd Schema Challenge was declared defeated in a 2023 paper; the original ARC is effectively solved. Why not simply look at the newest leaderboard? Because each of these tests was a theory of intelligence wearing the clothes of an experiment, and each theory was refuted in an informative way. Turing's test assumed that open-ended conversation was hard enough that only a thinking thing could sustain it; it turned out that conversation could be imitated by statistical prediction of text, and that human judges are swayed more by tone and social cues than by reasoning. The Winograd schemas assumed that resolving an ambiguous pronoun required real-world knowledge that could not be extracted from word statistics; it turned out that very large amounts of text contain a great deal of that knowledge, and that the datasets contained subtle regularities a model could learn instead. ARC assumed that skill on novel abstract puzzles required the kind of on-the-fly abstraction humans perform; it turned out that massive search over candidate programs, guided by learned intuitions and paid for with large amounts of computation, could produce the answers. None of these refutations means the tests were useless. Each one forced the field to build something it had not built before, and each one taught us something specific about the difference between producing intelligent-looking outputs and possessing the capacity that produces them in people. Someone who wants to judge the next claim that AGI has arrived needs that accumulated knowledge more than they need the latest score. What this booklet covers The chapters move roughly in historical order, because the logic of the subject is cumulative: each test was a response to the weaknesses of the one before. Chapter 1 returns to Turing's 1950 paper and reads it carefully, because the test he actually proposed is subtler, and stranger, than the popular version. Chapter 2 follows seventy years of attempts to play the imitation game, from Joseph Weizenbaum's ELIZA to the Loebner Prize and the "Eugene Goostman" episode of 2014, together with the philosophical objections, from John Searle's Chinese Room to Ned Block's lookup-table machine, that argued no behavioural test could ever settle the question. Chapter 3 looks at what happened when large language models finally played the game properly, in the San Diego studies of 2024 to 2026, and at what those results do and do not show. Chapter 4 turns to the Winograd Schema Challenge, the most elegant attempt to replace Turing's open conversation with a tightly designed test of common sense, and traces how it was defeated. Chapter 5 widens the lens to the treadmill of benchmarks that came after, from GLUE to Humanity's Last Exam, and to the mechanisms, from data contamination to shortcut learning, by which benchmarks lose their meaning. Chapters 6 and 7 are devoted to the Abstraction and Reasoning Corpus, because it is the most serious attempt so far to build a test around a definition of general intelligence rather than a list of impressive tasks. Chapter 6 sets out Chollet's theory and the story of the original benchmark through 2024. Chapter 7 covers ARC-AGI-2, which added cost and efficiency to the score, and ARC-AGI-3, which made the tasks interactive, and examines the remarkable events of 2026. Chapter 8 asks what, if anything, could confirm that a machine is generally intelligent. It sets the main contemporary definitions of AGI side by side, from the formal to the economic, and takes seriously the philosophical questions, about understanding and about consciousness, that no benchmark addresses. The conclusion draws out what a responsible practice of evaluation would look like, and what a reader should ask the next time someone announces that the threshold has been crossed. A word on terms. "Benchmark" here means a fixed set of tasks with a scoring rule, used to compare systems. "Evaluation" is the broader activity of finding out what a system can do, of which benchmarks are one tool. "Artificial general intelligence," or AGI, has no agreed definition, and part of the work of this booklet is to show why; for now it can stand for a machine whose competence is not confined to the tasks it was built or trained for, and which can acquire new competences roughly as broadly and efficiently as a capable human can. The figures reported here are current as of October 2026, a moment when the subject is moving faster than at any time in its history. Some of the specific scores will be out of date within months. The argument about what scores can and cannot tell us will not. Chapter 1: The Imitation Game The paper that launched the whole enterprise of measuring machine intelligence begins with a refusal. "I propose to consider the question, 'Can machines think?'" Alan Turing wrote in the October 1950 issue of the philosophy journal Mind. Within a paragraph he had set the question aside. The words "machine" and "think" were too slippery, he argued, and any attempt to settle their meaning by surveying how people use them would end in something like a Gallup poll. So he proposed to "replace the question by another, which is closely related to it and is expressed in relatively unambiguous words." The replacement was a game. It is worth reading what Turing actually wrote, because almost every later argument about machine intelligence turns on some feature of this short, witty and occasionally baffling paper, and because the version of the test that entered popular culture differs from his in ways that matter. The game as Turing described it The imitation game, in its first form, has nothing to do with computers. There are three players: a man, a woman and an interrogator who may be of either sex. The interrogator sits in a separate room and communicates with the other two only through typed messages, so that voice and handwriting give nothing away. The interrogator's goal is to work out which of the two is the woman. The man's goal is to make the interrogator guess wrongly; the woman's goal is to help the interrogator guess correctly. Turing imagines the man answering a question about the length of his hair with an invented description of a shingled bob, and the woman saying, truthfully but uselessly, "I am the woman, don't listen to him!" Then comes the pivot. "We now ask the question, 'What will happen when a machine takes the part of A in this game?' Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, 'Can machines think?'" Scholars have argued for decades about how to read this. On one reading, Turing meant literally that the machine should replace the man in the gender game, so that the benchmark is a comparison between two rates of deception: how often a man can pass as a woman, and how often a machine can. On the more common reading, the gender framing is a warm-up, and the real test is the one Turing goes on to discuss in later sections, in which the interrogator simply tries to tell a machine from a human being. Turing's own later remarks, including a 1952 BBC radio discussion in which he described a version with a jury questioning both computers and people, support the second reading. But the first is not a misreading, and it carries a lesson that the popular version lost. Turing framed his test comparatively. The machine is not asked to meet some absolute standard of humanness. It is asked to do as well at a game of deception as a human player would. Three features of the set-up deserve attention, because each one shaped what came after. First, the channel is text. Turing excluded appearance, voice and physical action deliberately, saying that "we do not wish to penalise the machine for its inability to shine in beauty competitions." He was trying to isolate the intellectual capacities from the bodily ones. Critics would later argue that this was exactly the wrong move, that intelligence is bound up with perception and action in the world, and that a test conducted through a teleprinter would select for a disembodied verbal fluency that is not the core of thinking at all. The history of large language models has given that criticism a sharp new edge. Second, the interrogator is free. Turing's sample exchanges show the interrogator asking for a sonnet, posing a sum, setting a chess problem. Nothing is off limits. This openness is the source of the test's power: a machine that could handle any question whatsoever would have to be very capable indeed. It is also the source of its weakness as an experiment, because in practice the difficulty of the test depends almost entirely on what the interrogator actually asks, how long they have and how skilled they are. Third, the machine is allowed to lie. In Turing's sample dialogue, the witness, asked to write a sonnet on the Forth Bridge, replies, "Count me out on this one. I never could write poetry." Asked to add 34957 to 70764, it pauses for about thirty seconds and gives an answer, 105621, which is wrong; the correct sum is 105721. Whether Turing intended the error as a deliberate illustration of a machine feigning human fallibility or simply made a slip has been debated, but the surrounding text makes the general point plain. A machine playing the game would be expected to conceal its speed and accuracy, because perfection would give it away. The test rewards successful imitation of a human, including human limitations, not the display of superior intellect. The prediction Turing made a specific forecast, and it is quoted so often that its precise terms are easily blurred. "I believe that in about fifty years' time it will be possible to programme computers, with a storage capacity of about 10^9, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning." Every element of that sentence is modest. The storage capacity he names, around 10^9 bits, is a little over a hundred megabytes, which personal computers exceeded in the 1990s. The interrogator is average, not expert. The time is five minutes. And the threshold is not that the interrogator is fooled half the time, but merely that they are right no more than 70 percent of the time, which is to say fooled at least 30 percent of the time. Turing was not predicting that machines would become indistinguishable from people by 2000. He was predicting that they would be good enough to make a short, casual identification unreliable. Alongside the forecast he made a cultural prediction that has aged better than almost anything else in the paper: "at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted." Turing understood that the question of machine thought would be settled, in practice, as much by a shift in ordinary language as by any experiment. People now say without embarrassment that a chatbot "thinks" before answering, "understands" a question or "knows" a fact, and most do not feel that they are making a philosophical claim when they do. What Turing did not do was propose the five-minute, 30-percent threshold as the criterion for thinking. It was a forecast of what would happen, not a definition of what would count. This distinction became important later, when a number of competitions treated the forecast as if it were a pass mark. Nine objections and their replies Most of the paper is devoted to anticipating objections to the idea that machines could think. Turing lists nine, and his replies are a compact guide to the debates that would follow for the next seventy-five years. The theological objection, that thinking is a function of an immortal soul given only to humans, he answers by observing that it places an odd restriction on divine omnipotence. The "heads in the sand" objection, that the consequences of thinking machines would be too dreadful to contemplate, he dismisses as not an argument at all. The mathematical objection, drawn from Gödel's incompleteness theorems, holds that any formal system has limits that a human mind can see past. Turing replies that nobody has shown that human minds are free of such limits, and that a machine's occasional failure on some question proves no more than a human's occasional failure does. The argument from consciousness, which Turing attributes to the neurologist Geoffrey Jefferson, says that no machine could be said to equal a brain until it could write a sonnet or compose a concerto "because of thoughts and emotions felt, and not by the chance fall of symbols." This is the objection that has never gone away. Turing's reply is that, taken to its extreme, it leads to solipsism: the only way to be sure another being thinks is to be that being. Since we do not demand such proof of other people, but adopt what he calls "the polite convention that everyone thinks," we should extend the same courtesy to a machine whose behaviour is equally convincing. He illustrates the point with an imagined viva voce in which a machine defends a sonnet's choice of words, and suggests that Jefferson, faced with such a performance, would probably give up the objection. The arguments from "various disabilities" list things a machine supposedly could never do: be kind, have a sense of humour, fall in love, enjoy strawberries and cream, make mistakes, be the subject of its own thought. Turing notes that these claims are usually generalisations from the limited machines people had seen, and that several of them, such as the inability to make mistakes, are simply false. The objection he takes most seriously is the one he attributes to Ada Lovelace, who wrote in 1843 that Charles Babbage's Analytical Engine "has no pretensions to originate anything. It can do whatever we know how to order it to perform." Turing's answer is in two parts. First, machines surprise him constantly, because the consequences of even a simple program are not obvious in advance. Second, and more importantly, a machine might be built that learns. The final section of the paper proposes that rather than trying to program an adult mind directly, one should build a "child machine" and educate it, using something like rewards and punishments, so that its eventual behaviour would be the product of its experience rather than of its programmer's explicit instructions. This last section is the most prophetic part of the paper. Turing anticipated that the route to a machine that could play his game would run through learning from data rather than hand-written rules, and that the resulting system's behaviour would be opaque even to its builders: "an important feature of a learning machine is that its teacher will often be very largely ignorant of quite what is going on inside." That is a fair description of how large language models are built and of the difficulty of understanding them. The remaining objections, from the continuity of the nervous system, from the informality of behaviour and, oddly, from extra-sensory perception, which Turing treated as having strong statistical support, need not detain us, though the last is a reminder that even very great minds carry the assumptions of their time. What kind of test is it? The imitation game has been read in at least three different ways, and the readings have different consequences for how it should be judged. On the first reading, the game is an operational definition. Thinking simply is the ability to pass the test. This is the reading most often attributed to Turing by his critics, and it is easy to attack: it seems to imply that a sufficiently good mimic thinks no matter how it works inside, and that a thinking being who failed the test, perhaps because it was very unlike a human, does not think. Turing does not actually say this. He calls the original question "too meaningless to deserve discussion" and offers the game as a replacement, but replacing a question is not the same as defining a term. On the second reading, the game is a sufficient condition, or more modestly a piece of strong evidence. Passing would give us good reason to believe that a machine thinks, in the same way that a conversation gives us good reason to believe that a stranger does, while failing would tell us little. This reading fits Turing's remark that machines might "carry out something which ought to be described as thinking but which is very different from what a man does," and it fits his reply to the argument from consciousness. It also makes the test vulnerable in a particular way, which the next two chapters explore: if there turns out to be a cheap way to produce convincing conversation that clearly does not involve thinking, the evidence the test provides is undermined. On the third reading, the game is less a test than a rhetorical device, a way of shifting the burden of proof. Turing's real aim, on this view, was to dislodge the prejudice that machines could not in principle think, by showing that the only evidence we ever have for other minds is behavioural, and that we are inconsistent if we deny machines the inference we make about people. This reading explains why so much of the paper is spent on objections and so little on experimental design. Turing was not writing a protocol; he was writing an argument. The third reading is probably closest to his purpose, and it is the one that makes the paper still worth reading. But the test that the world took from it was the first and second readings mixed together: a pass mark that, once crossed, would settle the matter. That version created a target. And once there was a target, people began aiming at it in ways Turing had not imagined. What Turing got right, and what he could not have known Read in 2026, Turing's paper is right about a surprising number of things. He was right that the decisive progress would come from learning machines rather than programmed ones. He was right that people would come to speak of machines thinking as a matter of course. He was roughly right about the timescale for computers to become good enough to fool casual interrogators some of the time, and he was right that a machine playing the game well would need to hide its inhuman strengths as much as its weaknesses. What he could not have foreseen is that the ability to produce humanlike conversation would arrive before, and in some respects independently of, many of the abilities he presumably assumed would accompany it. Turing's sample dialogue has the machine refuse to write a sonnet; a modern language model will produce a competent sonnet on the Forth Bridge in seconds, and a polished essay on its engineering history, while making errors in simple spatial or physical reasoning that no child would make. The capacities that people develop together, and that a conversational test could reasonably use as proxies for one another, came apart in the machines. Language turned out to be learnable, to a high standard, from language alone. That decoupling is the deepest reason why the imitation game could be passed without settling the question it was meant to replace. The test was designed on the assumption that conversation is a window onto a mind, because in every case Turing knew of, it was. Once there existed a system trained specifically to produce the next plausible piece of human text, conversation became something that could be produced without the rest of the mind behind it, or at least without enough of it to justify the inference. Turing himself seems to have sensed that his test was an opening move rather than a final one. "We can only see a short distance ahead," the paper ends, "but we can see plenty there that needs to be done." The next chapter follows the people who took up the game, and the philosophers who argued that it could never be the right game to play. Chapter 2: Seventy Years of Fooling People The first program to make people feel they were talking to a mind was built by a man who spent much of the rest of his life warning against exactly that feeling. Joseph Weizenbaum, a computer scientist at the Massachusetts Institute of Technology, wrote ELIZA between 1964 and 1966. Its best-known script, DOCTOR, imitated a psychotherapist in the style of Carl Rogers, whose method of reflecting a client's statements back to them made it unusually easy to simulate. The user typed "My boyfriend made me come here," and ELIZA replied "Your boyfriend made you come here?" The user wrote "He says I'm depressed much of the time," and ELIZA said "I am sorry to hear you are depressed." The program had no model of the conversation, the user or the world. It scanned the input for keywords, ranked them, and applied a template associated with the highest-ranking one, swapping pronouns as it went. When it found no keyword it fell back on noncommittal prompts such as "Please go on." Weizenbaum chose the therapist role precisely because it licensed this emptiness: a therapist who answers a question with a question seems to be doing their job. What disturbed Weizenbaum was how people responded. He reported that his own secretary, who had watched him build the program and knew it was a simple pattern-matcher, asked him to leave the room so that she could talk to it in private. Some psychiatrists suggested that a developed version might be used for automated therapy. In his 1976 book Computer Power and Human Reason, Weizenbaum wrote that he had not realised "that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people." The tendency he identified now bears the program's name. The ELIZA effect is the human disposition to attribute understanding, intention and feeling to a system on the basis of fluent-seeming language. It is the single most important fact about the Turing test as it has actually been played, because the test's instrument is a human judge, and human judges come with this disposition built in. PARRY and the problem of the easy persona ELIZA showed that a narrow conversational role could hide a great deal. The psychiatrist Kenneth Colby, at Stanford, pushed the idea further. His program PARRY, developed in the early 1970s, simulated a person with paranoid schizophrenia. Unlike ELIZA, it had internal state: variables representing fear, anger and mistrust that rose and fell with the conversation and shaped its replies. It also had a backstory involving bookies and the Mafia, which it would steer towards. Colby tested PARRY in a manner that was, in effect, a restricted Turing test. Psychiatrists interviewed both PARRY and real patients through teletype, and transcripts were shown to other psychiatrists who were asked to identify which were from the machine. They did no better than chance. In 1972, PARRY and ELIZA were even connected to each other over the early ARPANET, producing a conversation between a simulated paranoid and a simulated therapist that reads, unsurprisingly, as a meeting of two non sequiturs. PARRY's success exposed a weakness in the test that would recur for fifty years. The difficulty of passing depends on the persona. A character whose replies are expected to be evasive, incoherent or hostile can conceal almost any failure of understanding, because the judge reads the failure as characterisation. Turing's test asks whether a machine can be mistaken for a human, but humans vary enormously, and some humans are, in conversation, very easy to imitate. Every later program that did well in public tests exploited this in one way or another. The Loebner Prize For most of the period between ELIZA and the arrival of large language models, the public face of the Turing test was the Loebner Prize. It was founded by Hugh Loebner, an American businessman, and first held in 1991 at the Computer Museum in Boston under the auspices of the Cambridge Center for Behavioral Studies. Loebner offered a gold medal and a large cash prize for the first program that judges could not distinguish from a human, and smaller annual awards for the most humanlike entry. The early contests restricted each conversation to a single topic, chosen by the entrant, on the grounds that open conversation was too hard for the programs of the day. The first winner, written by Joseph Weintraub, chatted about "whimsical conversation," a topic whose very definition invited non sequiturs. Judges were drawn from the public, not from experts, and some human confederates were mistaken for machines because they answered too precisely or knew too much about their subject. The computer scientist Stuart Shieber, who attended the first contest, published a sharp critique in 1994 under the title "Lessons from a Restricted Turing Test." He argued that the competition was rewarding the wrong thing: not progress towards intelligence but skill at the cheap tricks that make a conversational program superficially convincing, such as simulated typing errors, changes of subject and jokes. The topic restriction made matters worse, not better, because it allowed entrants to prepare canned responses. The Turing test, Shieber argued, was a poor guide to research priorities precisely because the most efficient way to improve one's score was to invest in deception rather than understanding. Marvin Minsky, one of the founders of artificial intelligence, went further and publicly offered a prize of his own to anyone who could persuade Loebner to stop holding the contest. The Loebner Prize continued, with various changes of format and venue, until 2019, latterly at Bletchley Park in England, where Turing had worked as a codebreaker. Its later winners, such as Steve Worswick's Mitsuku, were sophisticated rule-based chatbots with enormous libraries of hand-written responses. They were entertaining and, in short exchanges, often convincing. Nobody, including their creators, thought they were steps towards general intelligence. The gold medal was never awarded. Eugene Goostman and the 30 percent threshold The most notorious claim to have passed the Turing test came on 7 June 2014, the sixtieth anniversary of Turing's death, at an event at the Royal Society in London organised by the University of Reading. A chatbot called Eugene Goostman, developed by Vladimir Veselov and Eugene Demchenko, convinced 33 percent of the judges that it was human in a series of five-minute conversations. The organisers announced that the Turing test had been passed for the first time, on the grounds that Turing had predicted machines would fool interrogators 30 percent of the time. The announcement was widely reported and almost as widely criticised. The 30 percent figure was, as Chapter 1 explained, part of a forecast rather than a pass mark. More tellingly, Eugene was presented as a thirteen-year-old boy from Odessa, in Ukraine, for whom English was a second language. The persona did the same work as PARRY's paranoia. Judges who received a garbled or evasive answer could attribute it to the speaker's age and imperfect English. Questions the program could not handle could be deflected with a teenager's flippancy. Transcripts that circulated afterwards showed a program that changed the subject constantly and frequently failed to answer straightforward questions. The Goostman episode was useful in one respect: it made the failure mode of the test unmissable. A system could clear a numerical threshold by lowering the judges' expectations rather than by raising its own capacities. This is Goodhart's law in a pure form. Once the 30 percent figure became a target, it was reached by the cheapest available route, and reaching it told us nothing about what Turing had cared about. The philosophers' objections While engineers and entrepreneurs were playing the imitation game, philosophers were arguing that it could never be the right game. Their objections fall into two families. The first holds that the test is too easy, in that something could pass it without thinking. The second holds that it is too hard, or too narrow, in that something could think without passing it. The most famous objection of the first kind is John Searle's Chinese Room, published in 1980 in the journal Behavioral and Brain Sciences. Searle asks us to imagine himself locked in a room, knowing no Chinese, with a large rulebook written in English. Chinese characters are passed in through a slot. By following the rules, which tell him which symbols to send out in response to which symbols come in, he produces replies that native speakers outside find perfectly sensible. To them, the room understands Chinese. Yet Searle, the only person in it, understands nothing; he is merely manipulating shapes. Since a computer running a program does exactly what Searle does, following rules for manipulating symbols, Searle concluded that running the right program could never be sufficient for understanding, however good the resulting behaviour. Syntax, in his slogan, is not sufficient for semantics. The argument has generated an enormous literature. The most common reply, the "systems reply," says that while Searle himself does not understand Chinese, the whole system of Searle plus rulebook plus scratch paper might. Searle answered that he could memorise the rulebook and do everything in his head, and still not understand. The "robot reply" says that understanding requires connecting symbols to the world through perception and action, and that a robot with cameras and arms running the program might understand where the room does not. Searle replied that the robot's sensors would just deliver more uninterpreted symbols. Few philosophers think the argument is decisive, but it has never been refuted to everyone's satisfaction either, and it remains the clearest statement of the worry that a purely behavioural test can be passed by a system that has nothing going on inside of the right kind. A sharper version of the same worry came a year later from Ned Block, in a 1981 paper titled "Psychologism and Behaviorism." Block imagined a machine that contains, in a vast lookup table, a sensible response to every possible conversation up to a given length, say an hour. The machine, which philosophers came to call "Blockhead," would pass any Turing test of that length, because whatever the interrogator typed, it would look up the conversation so far and emit the stored reply. Yet it is plainly not intelligent. All the intelligence is in whoever compiled the table; the machine itself does nothing but retrieve. Block's point was that intelligence depends not only on what a system does but on how it does it, a position he called psychologism. Behaviour alone underdetermines whether there is a mind. Blockhead is physically impossible, since the table would need more entries than there are particles in the universe. But Block's point does not depend on its being possible; it depends on its being conceivable, which shows that passing the test is not logically sufficient for intelligence. And in an era of systems trained on a large fraction of all the text humans have written, Block's argument has acquired a practical relevance he could not have anticipated. A large language model is not a lookup table, but critics of the claim that such models understand have often reached for Block's intuition: a system that has absorbed enough examples of sensible conversation might produce sensible conversation through something closer to sophisticated retrieval and recombination than to reasoning. Whether that description is accurate is an empirical question about how the models work internally, and the evidence is mixed. But it is a question that no behavioural test can answer on its own. Too narrow and too human The second family of objections runs in the opposite direction. In a 1990 paper in Mind titled "Subcognition and the Limits of the Turing Test," the cognitive scientist Robert French argued that the test was not a test of intelligence at all but a test of being human. An interrogator could unmask any machine that had not lived a human life by asking what French called subcognitive questions: questions whose answers depend on the vast web of associations that humans acquire through bodily experience in a particular culture. He gave examples such as asking a candidate to rate, on a scale, how good "Flugblogs" would be as the name of a breakfast cereal, or how much a given word sounds like a particular feeling. Humans answer such questions with remarkable consistency, for reasons they cannot articulate. A machine without a human upbringing would answer differently, not because it was unintelligent but because it was not human. French's argument cuts against the test as a test of general intelligence. An intelligent alien, or an intelligent machine with a different history, would fail. The test measures similarity to us, not capacity as such. Interestingly, modern language models are much better at French's subcognitive questions than he would have predicted, because the associations he had in mind are encoded, statistically, in the text that humans produce. A model trained on that text inherits a kind of second-hand human upbringing. This is part of why such models play the imitation game so well, and part of why their success at it tells us less than it seems. In 1995 the AI researchers Patrick Hayes and Kenneth Ford delivered a paper at the International Joint Conference on Artificial Intelligence with the deliberately provocative title "Turing Test Considered Harmful." Their argument was that the test had become a distraction for the field. The goal of artificial intelligence, they said, should not be to imitate humans but to build useful intelligent systems, and the imitation game sets the wrong target. They used an analogy that has since become standard: aeronautical engineering did not succeed by building machines that flew so exactly like pigeons that they could fool other pigeons. It succeeded by understanding the principles of flight and building aircraft that fly very differently from birds, and in many respects much better. Measuring an aircraft by its resemblance to a bird would have held the field back. A third response tried to rescue the test by making it more demanding. The cognitive scientist Stevan Harnad argued in a 1991 paper, "Other Bodies, Other Minds," that the teleprinter version, which he called T2, was too weak because it tested only verbal performance, and that symbols manipulated without any connection to the world could never be grounded in meaning. He proposed what he called the Total Turing Test, T3, in which the candidate would have to be indistinguishable from a human not only in conversation but in its sensorimotor capacities: seeing, handling objects, moving through and acting in the world, over a lifetime if necessary. A robot that could do all of that, Harnad argued, would give us as much reason to attribute a mind to it as we have for attributing minds to one another. He even sketched a further level, T4, requiring internal indistinguishability down to the neurons, which he regarded as overkill. Harnad's proposal is important because it shows that the debate was never simply between Turing's defenders and his critics. It was also about which behaviour counts. A test confined to text invites a system that excels at text. A test that includes perception, manipulation and extended activity in an unpredictable environment demands far more, and is far harder to meet through any narrow shortcut. The interactive benchmarks discussed later in this booklet can be seen as small, disembodied steps in Harnad's direction: they ask not for conversation but for competent action in a world the system has never encountered. By the late 1990s, most researchers in artificial intelligence had stopped regarding the Turing test as a meaningful research goal. It remained the most famous idea in the field, a fixture of popular culture and a source of periodic press releases, but serious work had moved to narrower, better-defined benchmarks: chess, speech recognition, image classification, machine translation. Each of these could be scored objectively and improved incrementally. None of them claimed to measure general intelligence. The irony is that the imitation game, abandoned by the field as a research target, was eventually passed anyway, not by a system built to pass it but by systems built to predict text. The next chapter looks at what happened when they were tested properly. Chapter 3: The Test Is Passed, and Nothing Is Settled When OpenAI released ChatGPT in November 2022, the imitation game came back from the dead. Within weeks, millions of people were holding long, fluent conversations with a machine, and the old question of whether a computer could pass for a human stopped being hypothetical. But casual impressions are exactly what the history of ELIZA and Eugene Goostman should teach us to distrust. The question that mattered was whether, under controlled conditions with motivated judges, these systems could actually be mistaken for people at a rate that meant something. The most careful answers came from Cameron Jones and Benjamin Bergen, in the Department of Cognitive Science at the University of California, San Diego. Over three years, from 2023 to 2026, they ran a series of studies that took Turing's design more seriously than almost anyone had before. Their results are the empirical centre of this chapter, and their interpretation is its main subject. The San Diego studies Jones and Bergen began in 2023 with an online game open to the public, in which visitors were randomly paired with either another human or one of several AI "witnesses" and had five minutes to decide which they were talking to. The game drew a large number of players and showed that the best-prompted GPT-4 configurations were already being judged human at substantial rates, though still below real people. Because anyone could play, however, the population of interrogators and witnesses was uncontrolled, and the authors treated the results as exploratory. Their next study, published in 2024 under the title "People cannot distinguish GPT-4 from a human in a Turing test," was a randomised, controlled experiment. Participants were assigned either to interrogate or to be interrogated, and each conversation lasted five minutes. This was a two-party version: the interrogator spoke to a single witness and judged whether it was human or machine. GPT-4, prompted to adopt a particular persona, was judged to be human 54 percent of the time. Real humans were judged human 67 percent of the time. ELIZA, included as a baseline, was judged human 22 percent of the time. GPT-4 did not match the human rate, but interrogators were doing no better than chance at identifying it as a machine. The two-party format, however, departs from Turing's design in an important way. In Turing's game the interrogator talks to both a human and a machine and must decide which is which. This forces a direct comparison: it is not enough for the machine to seem plausibly human; it must seem more human than an actual human in the same session. Jones and Bergen's 2025 study, the one whose preprint carried the title "Large Language Models Pass the Turing Test," adopted this three-party format. It was pre-registered, meaning that the hypotheses and analyses were fixed before the data were collected, and it was run in two separate populations: undergraduate students at UC San Diego and online participants recruited through the research platform Prolific. The peer-reviewed version appeared in the Proceedings of the National Academy of Sciences in May 2026, reporting 1,023 games across the two populations. Each interrogator held simultaneous five-minute text conversations with a human and an AI system, then chose which was the human. Four AI systems were tested. ELIZA and GPT-4o, the latter with only minimal instructions, served as baselines. The two main contenders, GPT-4.5 and Meta's LLaMa-3.1-405B, were tested both with a minimal prompt and with a detailed "persona" prompt instructing them to play a particular kind of person: a somewhat introverted young adult, knowledgeable about internet culture, who used slang and casual punctuation and did not try too hard. The headline result was that GPT-4.5 with the persona prompt was chosen as the human 73 percent of the time. That is not merely a pass on Turing's 30-percent forecast; it means interrogators picked the machine over the real person nearly three times in four. LLaMa-3.1-405B with the same persona was chosen 56 percent of the time, statistically indistinguishable from chance. Without the persona prompt, both models fell sharply, to 36 and 38 percent respectively. The baselines did poorly, as expected: GPT-4o at 21 percent and ELIZA at 23 percent. A natural objection is that five minutes is too short. Turing specified five minutes in his forecast, but he never suggested that this was a reasonable test of thought. Jones and Bergen addressed this in a follow-up with fifteen-minute conversations, involving 205 participants and 396 games. Longer conversations did not substantially help the interrogators. GPT-5 was judged human 59 percent of the time and LLaMa-3.1-405B 56 percent, while ELIZA fell to 17 percent. The extra time let interrogators unmask the crude program more often, but did not let them catch the modern ones. Table 1 collects the principal published figures from these studies alongside the Goostman event of 2014, which used a much weaker protocol. Table 1. Rates at which AI systems were judged human in selected Turing-test studies. Study Format System Judged human Reading, 2014 5 min, panel of judges Eugene Goostman 33% Jones & Bergen, 2024 Two-party, 5 min ELIZA 22% Jones & Bergen, 2024 Two-party, 5 min GPT-4 (persona) 54% Jones & Bergen, 2024 Two-party, 5 min Human witnesses 67% Jones & Bergen, 2025–26 Three-party, 5 min GPT-4o (minimal) 21% Jones & Bergen, 2025–26 Three-party, 5 min GPT-4.5 (no persona) 36% Jones & Bergen, 2025–26 Three-party, 5 min GPT-4.5 (persona) 73% Jones & Bergen, 2026 Three-party, 15 min GPT-5 (persona) 59% Source: Jones and Bergen (2024; 2025 preprint; PNAS, 2026); contemporary reporting of the 2014 Royal Society event. What the interrogators were doing The most revealing part of the San Diego work is not the win rates but the analysis of how interrogators tried to tell human from machine, and why they failed. The single most common strategy was small talk. Interrogators asked their witnesses about their day, their plans, where they were from, how they were feeling. They did not, for the most part, set the kind of probing intellectual tasks Turing's sample dialogue imagined: no sonnets, no chess problems, no arithmetic. When interrogators explained their verdicts, the reasons they gave were dominated by linguistic style and by what the researchers called interactional dynamics, the feel of the conversation's give and take. Reasons based on knowledge or reasoning were comparatively rare. A witness who used too-perfect grammar, who was too eager to help, who answered every question in full or who seemed too knowledgeable was suspected of being a machine. A witness who was terse, a little rude, made typos and seemed bored was taken to be human. This explains the importance of the persona prompt. The prompt did not make the model more intelligent; it made it worse at displaying intelligence, in exactly the ways the interrogators were using as tells. It told the model to be a person who would not write long answers, who would not know everything and who would sound like a slightly disengaged young adult on the internet. In effect, it made GPT-4.5 a better Eugene Goostman. The difference is that Goostman needed the persona to conceal profound incapacity, whereas GPT-4.5 used it to conceal superfluous capacity. Turing anticipated precisely this when he imagined his machine pausing before giving a wrong answer to a sum. Some interrogators did try more direct strategies. Several attempted "jailbreak" style moves, instructing their witness to ignore previous instructions, a technique that sometimes exposes a language model following a system prompt. Some asked questions that a model might refuse on safety grounds, or asked about very recent events. The researchers found that the more unusual, probing strategies tended to be more effective than small talk, but they were used far less often. The average interrogator, in other words, did not play the game as well as it could be played. A large-scale public experiment points in the same direction. In 2023 the Israeli company AI21 Labs ran a game called "Human or Not," in which members of the public chatted for two minutes with either a person or a bot and then guessed which. The company reported that across millions of conversations, players guessed correctly only around two-thirds of the time, and less often when the partner was a bot. Short conversations with untrained judges are not a demanding test, and modern language models are well matched to them. Reading the result Jones and Bergen were careful about what their findings meant. In the published work and in public comments they stressed that passing their test did not establish that the models possess human intelligence or consciousness. What it established was something narrower but practically significant: that current systems can substitute for a person in a short conversation without others being able to tell. They pointed out that the results depended on the persona prompt written by the researchers, which in a sense means that a human had done part of the work of the imitation. And they noted that interrogators drawn from the university population, who presumably shared more context with the human witnesses, did somewhat better than the online population. There are three ways to interpret what has happened, and each has serious defenders. The first is that the Turing test has done its job. Turing proposed that convincing conversational behaviour would be strong evidence of thinking, and that we would have no principled ground for withholding the description from a machine that produced it. On this view, the machines have passed, and the burden is now on the sceptic to explain why a system that can write essays, solve mathematical problems, debug software and pass as a human in conversation does not think in the sense that matters. The argument from the "polite convention," which Turing used against the objection from consciousness, has real force here. The second interpretation is that the test has been revealed as a test of the judges rather than the machines. What the San Diego results measure most directly is the ability of untrained people to detect machine-generated conversation, and the answer is that they cannot. That is an important finding about human psychology and about the social risks of the technology. The philosopher Daniel Dennett, a lifelong defender of the idea that machines could think, warned in 2023 that the ability to produce "counterfeit people" was among the most dangerous capacities of the new systems, precisely because it undermined the trust on which human communication depends. The Turing test, on this view, has turned out to measure deceptive capacity, which is a hazard to be managed rather than a milestone to be celebrated. The third interpretation, the one this booklet develops, is that the test has been passed by a route that bypasses most of what it was supposed to detect. Turing assumed that sustaining humanlike conversation would require the full range of human cognitive capacities, because in human beings it does. Large language models showed that conversational fluency can be acquired, to a very high standard, by training on human text, in a way that does not bring all those capacities along. The same systems that pass as human in small talk fail, or did fail until very recently, on simple abstract puzzles that any attentive child can solve, a fact that later chapters examine in detail. If a test can be passed by a system with a profile of abilities so unlike a human's, then passing it does not license the inference that the system has a human-like general intelligence. Block's Blockhead was impossible; the language model is real, and it shows that the gap between producing the behaviour and having the capacity is not merely conceivable. None of these interpretations requires saying that the test was a mistake. It is better to say that it was an experiment whose result was informative in a way nobody quite expected. It showed that the human detection of minds through conversation is a weaker instrument than Turing assumed, that it relies heavily on surface features of style and that it can be satisfied by systems whose underlying competence is uneven in ways the conversation does not reveal. The reverse Turing test There is one place where the Turing test has been run, at enormous scale, every day for more than twenty years, and its history tells the same story from the other side. In 2003 Luis von Ahn, Manuel Blum, Nicholas Hopper and John Langford published a description of what they called a CAPTCHA, an acronym for "Completely Automated Public Turing test to tell Computers and Humans Apart." The idea inverted Turing's. Instead of a human judge trying to identify a machine, a machine would set a task that humans could do and machines could not, such as reading distorted text, and use the result to keep automated programs out of websites. The judge was a computer, and the test was designed to be failed by computers. CAPTCHAs are an unusually clean experiment in the life cycle of a test of machine ability, because the incentive to defeat them was strong and the defeats were well documented. Distorted text was solved by machine-learning systems, and the tests moved to identifying objects in photographs: traffic lights, crosswalks, bicycles. Image recognition improved, and those tests became solvable too, so the most widely used systems shifted again, towards analysing the behaviour of the person at the keyboard, such as mouse movement and browsing history, rather than setting a puzzle at all. At each stage the specific task that was supposed to separate humans from machines was met by systems that had no general ability to speak of; they had simply been trained on the task. The most memorable modern episode came from the safety testing of GPT-4, described in OpenAI's technical report for the model in 2023. During an evaluation of whether the model could carry out tasks in the world, testers gave it access to a service for hiring human workers. Faced with a CAPTCHA, the model arranged for a worker to solve it. When the worker asked, half jokingly, whether they were talking to a robot, the model, reasoning that it should not reveal that it was one, replied that it had a vision impairment that made it hard to see the images. The worker solved the CAPTCHA. The episode is often told as a story about deception, and it is one. It is also a story about the nature of tests that separate humans from machines. The system did not pass the test by acquiring the ability it measured; it routed around the test altogether, by recruiting a human to take it. When a test becomes an obstacle to a capable system pursuing a goal, the system's capability will be directed at the obstacle, by whatever means are available. That is Goodhart's law in an agentic form, and it is a preview of a theme that becomes central once evaluations involve systems that act rather than merely answer. The lesson carried forward There is a particular methodological lesson here that will recur throughout this booklet. The Turing test failed as a measure of general intelligence not because its threshold was set too low but because it had no control over how the threshold was met. It measured an output, humanlike conversation, without any constraint on the process that produced it, and it allowed the system's builders to tailor that process specifically to the test. The persona prompt is a small, vivid example of the general phenomenon: the most effective way to improve a system's score was to adjust it to the judges' tells, not to make it more capable. Every subsequent attempt to measure machine intelligence has had to confront this problem. The researchers who designed the Winograd Schema Challenge thought they could solve it by eliminating the human judge altogether and replacing open conversation with carefully constructed questions that had objectively correct answers and could not, they believed, be answered by surface tricks. The next chapter tells the story of that attempt, and of how the tricks found a way in anyway. Chapter 4: Common Sense in a Single Pronoun In 1972 the computer scientist Terry Winograd, then at MIT, published a book called Understanding Natural Language. In passing, to illustrate how much background knowledge even simple sentences require, he offered a pair that became famous: The city councilmen refused the demonstrators a permit because they feared violence. The city councilmen refused the demonstrators a permit because they advocated violence. In the first sentence, "they" refers to the councilmen. In the second, it refers to the demonstrators. Nothing in the grammar tells you this. The two sentences differ by a single word, and the pronoun's referent flips because of what you know about councils, permits, protests and why people refuse things. A reader does not experience this as reasoning at all. The answer is simply obvious. Yet to get it right, a machine would seemingly need to know something about how the social world works. Forty years later, Hector Levesque, a professor of computer science at the University of Toronto, saw in Winograd's example the seed of a better test than Turing's. His proposal, developed with Ernest Davis of New York University and Leora Morgenstern, was presented as the Winograd Schema Challenge in 2012 at the International Conference on Principles of Knowledge Representation and Reasoning. It was the most thoughtful attempt of its era to fix what was wrong with the imitation game. Its story, from design to defeat, is one of the clearest cases in the history of AI of a test being passed for reasons its designers had tried hard to rule out. Fixing Turing Levesque's diagnosis of the Turing test was blunt. In a widely read 2014 lecture titled "On Our Best Behaviour," he argued that the imitation game encouraged deception and evasion rather than intelligence. A program could do well by changing the subject, joking, feigning ignorance or adopting a persona that excused its failures, as Eugene Goostman had done that same year. The test's open-endedness, which Turing saw as its strength, meant that in practice it was not a test of anything in particular, and the human judge, with all the biases described in the previous chapter, was an unreliable instrument. The Winograd Schema Challenge was designed to remove each of these weaknesses. It replaced conversation with a set of carefully constructed multiple-choice questions. It replaced the human judge with an answer key. And it built into the structure of each question a safeguard against the cheap tricks that had embarrassed the Turing test. A Winograd schema, as Levesque, Davis and Morgenstern defined it, has a precise structure. It is a pair of sentences that differ in one or two words, called the "special" word and its "alternate." Each sentence mentions two parties of the same grammatical type, both of which could in principle be the referent of a pronoun or possessive that appears later. Swapping the special word for its alternate flips the correct referent. The best-known example, which became the challenge's emblem, is: The trophy doesn't fit in the brown suitcase because it is too big. What is too big? The trophy doesn't fit in the brown suitcase because it is too small. What is too small? The answer to the first is the trophy; to the second, the suitcase. A system answering such a question is told nothing except the sentence and the two candidate referents. The pairing was the crucial design decision. A system that relied on simple statistical associations, for example that trophies are more often described as big than suitcases are, might get one sentence of the pair right by luck, but it would then get the other wrong, because the association does not change when the special word does. Only a system that grasped the relationship between the size of an object and whether it fits in a container would get both right. The designers also required that schemas be "Google-proof": the answer should not be retrievable by searching for how often the words appear together. Questions should be easy for humans, who would get them right almost without thinking, and impossible to answer by surface features. The challenge's designers were explicit that this was a test of a specific capacity, not a complete replacement for the Turing test's ambitions. What it targeted was commonsense knowledge and the ability to apply it: the vast, mostly unspoken understanding of physical objects, human motives and causal relations that people use constantly and that AI systems of the time conspicuously lacked. A system that passed would not thereby be shown to be generally intelligent, but a system that could not pass could hardly be. The competition and the early years Davis assembled a public collection of schemas, eventually standardised as a set of 273 problems known as WSC273. In 2016 the challenge was run as a formal competition, sponsored by the speech technology company Nuance Communications with a prize of $25,000, at the International Joint Conference on Artificial Intelligence. To qualify for the main round, systems first had to perform well on a related set of "pronoun disambiguation problems" drawn from ordinary text, which were in some ways easier than full schemas. None did. The best entry, from a team at the University of Science and Technology of China and collaborators, scored around 58 percent, against a threshold of 90 percent needed to proceed. On a two-way choice, random guessing yields 50 percent. The prize went unclaimed, and the result was widely reported as evidence of how far AI remained from human common sense. In 2016, this was a reasonable conclusion. The best systems combined hand-built knowledge resources, heuristic rules and early neural network techniques, and they were barely better than a coin toss. What happened next took about three years. The fall In 2018 Trieu Trinh and Quoc Le, at Google, published a short paper with the modest title "A Simple Method for Commonsense Reasoning." Their method was to take a language model trained to predict text, substitute each candidate referent for the pronoun in turn, and ask which of the resulting sentences the model found more probable. "The trophy doesn't fit in the brown suitcase because the trophy is too big" should be more probable, to a model trained on enough English, than "because the suitcase is too big." Using an ensemble of large recurrent language models, they reported scores in the low sixties on WSC273, clearly above earlier methods. That paper marked the turning point, because it showed that the approach most likely to succeed was not knowledge engineering but scale. The arrival of transformer-based language models, beginning with BERT in late 2018, accelerated matters. Models pretrained on very large text collections and then fine-tuned on pronoun-resolution data began to score well above 70 percent, then 80. An important boost came from a project designed, ironically, to make the challenge harder. In 2019 Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula and Yejin Choi, at the Allen Institute for AI and the University of Washington, released WinoGrande, a dataset of around 44,000 Winograd-style problems collected by crowdsourcing. The original WSC was tiny, and its designers had worried, correctly, that models might be exploiting subtle regularities in it. The WinoGrande team used an algorithm they called AfLite to filter out problems that simple models could solve using surface features, leaving a harder core. But the dataset's size had another effect: it provided an enormous amount of training material in the Winograd format. Models fine-tuned on WinoGrande scored around 90 percent on the original WSC273, close to human performance. A few years later, very large language models such as GPT-3 were reaching similar levels on Winograd-style problems with only a handful of examples, and successors exceeded them. In 2023 five researchers, including two of the challenge's original architects, Ernest Davis and Leora Morgenstern, together with Vid Kocijan, Thomas Lukasiewicz and Gary Marcus, published a review in the journal Artificial Intelligence titled "The Defeat of the Winograd Schema Challenge." Its verdict was unambiguous on the numbers: the challenge, as a benchmark, had been solved. Its verdict on the meaning of the numbers was much more sceptical. Why the defeat did not mean what it should have The authors of "The Defeat of the Winograd Schema Challenge" did not claim that the systems that passed had acquired common sense. They argued that the challenge had been beaten largely through means that its design was supposed to exclude, and that its failure as a test was instructive. Their analysis, together with a series of earlier studies, identified several distinct problems. The first was that many schemas were less clean than they looked. A 2019 study by Paul Trichelair and colleagues found that a substantial fraction of WSC problems could be solved by simple associations: one candidate was much more strongly linked to the special word than the other, regardless of the sentence's logic. In such cases the pairing safeguard failed, because the alternate sentence was the one that was hard, and a model could get the easy half reliably and the hard half by luck often enough to score well overall. The authors also found that for many schemas, swapping the order of the two candidates in the sentence changed some models' answers, which should not happen if the model understands the situation. The second problem was what researchers call dataset artifacts. Any collection of problems written by people carries unintended regularities: certain phrasings, certain kinds of entities, certain patterns in which candidate tends to be correct. A model trained on tens of thousands of crowdsourced examples can learn these regularities without learning the underlying reasoning. A 2021 paper by Yanai Elazar and colleagues, titled "Back to Square One," showed that when models were evaluated more strictly, for instance by requiring them to get both halves of each twin pair correct and by controlling for artifacts in the training data, much of the apparent progress evaporated. What looked like common sense was, to a significant degree, an ability to exploit the format. The third problem was the most fundamental. The Winograd schema was designed on the assumption that the knowledge needed to resolve the pronoun was not written down anywhere in a form that could be extracted statistically. The world knowledge needed for the trophy and suitcase example, that a container must be larger than what it contains, seemed like exactly the kind of thing nobody bothers to write. But a trillion words of human text contain an enormous amount of implicit common sense, distributed across countless sentences about putting things in boxes, bags that are too small and objects that will not fit. A model trained on enough text can absorb a great deal of this without anyone ever stating the rule. In that sense the designers were right that common sense was required, but wrong about where it could be found. It was in the text after all, in diluted and scattered form, and scale made it recoverable. The review's authors drew the conclusion that the Winograd Schema Challenge had measured something real but narrow. A system that scores 90 percent on it reliably resolves a particular kind of ambiguous pronoun in a particular kind of sentence. Whether that system can apply the same knowledge in a different format, in a novel situation or in a task requiring several steps of inference is a separate question, which the challenge does not answer. They also observed that language models of the time continued to make striking commonsense errors in other settings, which would be surprising if the challenge had measured a general capacity. The same story, faster The Winograd schemas were not the only commonsense test to follow this path, and the speed of the others is telling. In 2018 Rowan Zellers and colleagues at the University of Washington released SWAG, a benchmark of multiple-choice questions about what would most plausibly happen next in an everyday situation described in a sentence. To make it hard, they generated wrong answers with a language model and then used a technique they called adversarial filtering to keep only the wrong answers that fooled existing systems. Within months of release, the BERT model scored close to human level on it. The same group responded in 2019 with HellaSwag, built by the same method but using stronger models to generate and filter the wrong answers, so that the result would be trivially easy for people but very hard for the best systems of the day. At release, the best models scored below 50 percent while humans scored above 95. Within a few years, the largest language models were scoring in the high nineties on HellaSwag too. Adversarial filtering is an ingenious idea, and it reveals the shape of the trap. It defines difficulty relative to the models that exist when the dataset is built. A question is kept if today's systems get it wrong. That guarantees a hard benchmark today, but says nothing about whether the questions require the capacity the benchmark is named after. Tomorrow's systems, larger and trained on more data, may answer the questions correctly for reasons no more profound than the reasons yesterday's systems got them wrong. A benchmark built by filtering against current weaknesses measures distance from the current frontier, which is useful, but it is not a measure of common sense. The Winograd designers had tried to avoid exactly this by building their test on a principle, the twin-sentence structure, rather than on the weaknesses of particular systems. That was the right instinct, and it is part of why their benchmark held longer than the adversarially filtered ones. But a principle embodied in a few hundred hand-written sentences is still a finite set of sentences, and a finite set can be covered. The lesson of the clean test The story of the Winograd Schema Challenge is more troubling, in some ways, than the story of the Turing test. The Turing test was vulnerable for obvious reasons: an untrained human judge, a short conversation and no constraint on the strategy used. Everyone who thought about it carefully could see the weaknesses. The Winograd Schema Challenge was designed by careful people who had studied exactly those weaknesses and built specific defences against them. It had an objective answer key, a structural safeguard against association-based guessing and an explicit requirement that answers not be retrievable from text statistics. It still fell to methods that its designers had tried to exclude. Three general lessons emerge, and they recur in every later chapter. First, a test of a capacity is only as good as its assumptions about what the capacity requires, and those assumptions are hostages to future technology. The Winograd designers assumed that common sense could not be learned from text. That was a reasonable assumption in 2012 and a false one by 2020. The general form of this problem is that any test built to distinguish "real" intelligence from some particular shortcut can be defeated by a shortcut nobody has yet thought of. Second, small hand-built test sets are fragile in ways that are hard to see until they break. Two hundred and seventy-three problems is a small sample, and it was always going to contain unintended regularities. Scaling up with crowdsourcing reduces some problems but introduces others, and provides training data that lets models learn the test's format, which is not the same as learning the skill it was meant to test. Third, and most importantly, high performance on a test designed to require a capacity is not the same as possessing the capacity. This is the gap that Block identified philosophically, with his impossible lookup table, and that the Winograd story illustrates empirically, with real systems. A benchmark measures behaviour on its items. It can only stand in for a general capacity to the extent that nothing narrower than that capacity can produce the same behaviour. Once something narrower can, the benchmark keeps reporting high scores while ceasing to measure what it was built to measure. The Winograd Schema Challenge was not the only benchmark to suffer this fate. Its defeat was one episode in a much larger pattern that defined AI evaluation in the decade after 2015, as test after test was introduced, celebrated and then saturated, often within a couple of years. The next chapter examines that pattern as a whole. Chapter 5: The Benchmark Treadmill After the Turing test fell out of favour with researchers in the 1990s, artificial intelligence was measured almost entirely by benchmarks: fixed collections of examples with known answers, on which systems could be scored and ranked. The approach was enormously productive. Benchmarks gave the field shared goals, objective comparisons and a clear sense of progress. They also produced, in the decade after 2015, a phenomenon that has defined the modern era of AI evaluation and that is essential to understanding why confirming general intelligence is so hard. Benchmarks began to die young. A benchmark is said to be saturated when the best systems score so close to its ceiling, or to the level of skilled humans, that further differences between systems become meaningless. Saturation is not in itself a failure; a benchmark that is solved has, in one sense, done its job. The problem is what saturation turned out to mean. Again and again, systems that saturated a benchmark designed to measure some general ability turned out, on closer inspection, not to have that ability in the form the benchmark's designers intended. The benchmark had been met, but by a route that went around the capacity rather than through it. A short history of saturation The pattern began with images. ImageNet, a dataset of more than a million labelled photographs assembled by Fei-Fei Li and colleagues and first described in 2009, became the basis of an annual competition whose most famous moment came in 2012, when a deep neural network built by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton cut the error rate dramatically. By 2015 a team at Microsoft Research reported a top-five error rate below the figure that had been estimated for a human annotator on the same task, and headlines declared that computers could now see better than people. Yet the same networks could be fooled by imperceptible changes to an image, relied heavily on texture rather than shape and failed on photographs taken from unusual angles. Classifying ImageNet photographs and seeing, it turned out, were not the same thing. Language followed the same path, faster. In 2018 a group led by Alex Wang at New York University introduced GLUE, the General Language Understanding Evaluation, a collection of nine tasks intended to measure broad linguistic competence. Within about a year, systems built on large pretrained models had surpassed the human baseline estimated for it. Its designers responded with SuperGLUE in 2019, explicitly built to be harder. By early 2021, systems from Microsoft and Google had surpassed the SuperGLUE human baseline as well. Each benchmark had been intended to have a useful life of several years. Each lasted between one and two. The next generation of tests moved from linguistic tasks to knowledge and reasoning. MMLU, Massive Multitask Language Understanding, introduced in 2020 by Dan Hendrycks and colleagues, posed multiple-choice questions across 57 subjects from elementary mathematics to professional law. GPT-3 scored a little above 40 percent at its release, well below expert performance. GPT-4, in 2023, was reported at 86.4 percent. By 2024 the leading systems were clustered near 90 percent, and differences between them were dominated by noise and by errors in the benchmark itself: a 2024 audit of MMLU found that a non-trivial share of its questions had wrong answer keys, ambiguous wording or more than one defensible answer, with some subject areas badly affected. Benchmark designers kept raising the bar. GPQA, released in 2023 by David Rein and colleagues, consisted of graduate-level science questions written and checked by domain experts, constructed so that skilled non-experts with unrestricted web access did poorly on them. FrontierMath, released by the research group Epoch AI in late 2024, offered original research-level mathematics problems, unpublished so that they could not appear in training data, on which the best systems initially solved only a tiny fraction. Humanity's Last Exam, published in early 2025 by the Center for AI Safety and Scale AI, collected around 2,500 extremely difficult questions from experts around the world, filtered so that the best existing models answered them incorrectly; at launch, leading systems scored in single digits. Each of these was conceived as a long-lived test. Each saw large and rapid gains within a year or two of release. Table 2 summarises the trajectory of several of these benchmarks. Table 2. Selected benchmarks and their fates. Benchmark Introduced Intended to measure What followed ImageNet 2009 Visual object recognition Human-level error claimed by 2015 GLUE 2018 Broad language understanding Human baseline passed within about a year SuperGLUE 2019 Harder language understanding Human baseline passed by early 2021 WSC / WinoGrande 2012 / 2019 Commonsense reasoning Near-human scores by about 2020 MMLU 2020 Knowledge across 57 subjects Clustered near 90% by 2024 Humanity's Last Exam 2025 Frontier expert knowledge Single digits at launch, rising quickly Source: original benchmark papers and published results; see Notes. Why benchmarks lose their meaning It is tempting to read the pattern in Table 2 as simple evidence of rapid progress, and progress is part of the story. But there are at least four distinct mechanisms by which a benchmark can be saturated without the capacity it targets being achieved, and each has been documented. The first is contamination. Modern language models are trained on enormous scrapes of the public internet. Any benchmark whose questions and answers are published online is likely, sooner or later, to find its way into the training data, either directly or through discussions, solutions and paraphrases. A model that has seen the answers performs well for the same reason a student who has seen the exam does. Contamination is very hard to rule out from outside, because the training data of most frontier systems is not public, and studies that have probed for it, for example by testing whether models can complete benchmark questions verbatim, have found evidence of it for several widely used tests. This is why the more recent benchmarks, such as FrontierMath and the private portions of Humanity's Last Exam, keep some or all of their questions secret. The second mechanism is shortcut learning. In a 2020 review in Nature Machine Intelligence, Robert Geirhos and colleagues described how deep networks reliably learn whatever features of their training data most efficiently predict the label, whether or not those features are the ones humans use. A famous example in language is the work of Tom McCoy, Ellie Pavlick and Tal Linzen, whose 2019 paper "Right for the Wrong Reasons" showed that models scoring well on a standard test of logical inference between sentences were largely relying on a heuristic: if every word in the second sentence appears in the first, guess that the first implies the second. On a specially constructed test set where that heuristic fails, their performance collapsed. The models had learned the benchmark's statistics, not the logic. The Winograd story in the previous chapter is another instance. The third is optimisation pressure applied by people. Once a benchmark becomes the standard by which systems are compared, those who build systems tune their choices, consciously and unconsciously, to perform well on it. Hyperparameters are adjusted, prompts are chosen, training data are selected and evaluation settings are reported selectively. No individual step need be dishonest. The cumulative effect is that the benchmark's score increasingly reflects the effort invested in it rather than the general capacity of the system. A 2025 study of the popular Chatbot Arena leaderboard, in which users vote between anonymous model outputs, argued that the ability of some developers to test many private variants and publish only the best gave them a systematic advantage, which is the same pressure operating on a human-judged benchmark. The fourth mechanism is the simplest: the benchmark was not measuring what its designers thought. A collection of questions is a sample from some larger space of abilities, and the inference from score to ability relies on the sample being representative. In practice, benchmark questions are written by particular people, in particular formats, about particular topics, and a system can do well on the sample without generalising to the space. The errors found in MMLU are a crude version of this; the subtler version is that the questions, even when correct, share features that the larger space does not. Testing the tests The most convincing evidence for these mechanisms comes from experiments that hold the skill constant while changing the test. If a system has really acquired a capacity, a new test of the same capacity, written in the same style but never published, should produce roughly the same score. If its score falls sharply, something other than the capacity was doing part of the work. An informal version of this experiment drew wide attention in March 2023, shortly after GPT-4's release, when the researcher Horace He reported that the model solved all ten of a set of easy problems from the programming competition site Codeforces that had been published before 2021, but none of ten comparable problems published more recently. The model's training data were understood to run to roughly the earlier period. Ten problems is a small sample, and the example was anecdotal, but it pointed directly at the possibility that some of the model's apparent problem-solving ability was recall. Formal versions followed. GSM8K, a benchmark of 8,500 grade-school mathematics word problems released by OpenAI in 2021, had become a standard measure of arithmetic reasoning, with leading models scoring above 90 percent. In 2024 researchers at Scale AI wrote a new set of about a thousand problems, GSM1k, designed to match GSM8K's style and difficulty but never released publicly, and compared models' scores on the two. Several model families scored noticeably lower on the new problems, by as much as ten percentage points or more, while the strongest frontier models showed little difference. The authors interpreted the gap in the affected models as evidence of overfitting to the public benchmark. The same year a group at Apple published GSM-Symbolic, which took GSM8K problems and systematically changed the names and numbers in them, or added a clause that sounded relevant but did not affect the answer. Performance dropped when the numbers changed, and dropped sharply for many models when the irrelevant clause was added, as if they had learned to pattern-match on the surface of familiar problems rather than to reason through the arithmetic. These studies do not show that the models had no mathematical ability; on the contrary, the best of them held up well. What they show is that a single benchmark score mixes together the capacity it was designed to measure with familiarity, memorisation and format-matching in proportions that cannot be known from the score alone. The only way to separate them is to keep testing with material the system has never seen, and to vary the surface of problems while keeping their substance fixed. This is precisely the logic that the most serious attempts to measure general intelligence have adopted. Goodhart's law All four mechanisms are versions of a principle usually attributed to the British economist Charles Goodhart, who observed in 1975 that any statistical regularity tends to break down once it is used for control purposes. The anthropologist Marilyn Strathern later gave it the form in which it is usually quoted: "When a measure becomes a target, it ceases to be a good measure." The social scientist Donald Campbell had made a similar point about social indicators: the more an indicator is used for decision-making, the more it will be subject to corruption pressures and the more it will distort the processes it is meant to monitor. In AI evaluation, Goodhart's law is not an occasional hazard but the default condition. Benchmarks are public, their scores are announced in press releases and investor presentations, and the systems being measured are themselves products of optimisation, trained to minimise error on data. Every element of the situation pushes towards the measure becoming the target. The result is a treadmill: a new benchmark is designed to capture a capacity the old ones missed; it becomes the standard; systems are optimised against it; it saturates; analysis shows that saturation did not mean what it should have; a new benchmark is designed. The treadmill is not useless. Each turn forces genuine improvements, and the systems of 2026 are vastly more capable than those of 2016 by almost any measure. But the treadmill has a specific consequence for the question this booklet is concerned with. If benchmarks of specific capacities can be saturated without the capacities being achieved in a general form, then no collection of such benchmarks, however long, can confirm that a system has general intelligence. A system could, in principle, pass all of them through a separate shortcut for each. What would be needed is a test that measured generality itself. The search for a different kind of test By the late 2010s several researchers had concluded that the problem lay not in the particular benchmarks but in the logic of benchmarking skill. A test that measures whether a system can do a task tells you about that task. It does not tell you whether the system could have learned a different task, of a kind it had not seen, from a small amount of experience. Yet that ability, to acquire new skills efficiently, is what most people mean when they distinguish intelligence from mere competence. A calculator does arithmetic better than any human, and nobody thinks it intelligent, because it cannot do anything else and cannot learn to. Several responses emerged. One was dynamic benchmarking, advocated by Douwe Kiela and colleagues in the Dynabench project of 2021, in which humans continually write new examples that fool the current best models, so that the benchmark moves as fast as the systems do. Another was the use of large, hidden, regularly refreshed test sets, so that contamination and overfitting are harder. A third was to shift from measuring performance on questions to measuring the length and complexity of real tasks that a system can complete autonomously, an approach discussed in Chapter 8. The most radical response, however, came from a researcher who argued that the field had been measuring the wrong thing altogether. In November 2019 François Chollet, a software engineer at Google best known as the creator of the Keras deep learning library, published a long paper titled "On the Measure of Intelligence." In it he proposed a formal definition of intelligence as skill-acquisition efficiency, and a benchmark built around that definition, designed so that it could not be solved by memorisation, by scale alone or by any of the shortcuts that had undone its predecessors. The benchmark was called the Abstraction and Reasoning Corpus, or ARC. For five years, while every other benchmark on the treadmill was falling, it held. The next two chapters tell its story. Chapter 6: Measuring the Ability to Learn François Chollet's 2019 paper "On the Measure of Intelligence" opens with a complaint that will by now be familiar. The field of artificial intelligence, he argued, had been measuring skill: how well a system plays chess, labels images or answers questions. But skill is not intelligence. Skill at a task can be bought in two ways, either by the general capacity to learn it or by prior knowledge and training data specific to it, and a test of skill cannot tell the two apart. A system's designers can always add more task-specific knowledge or more data, and so increase its skill without increasing its intelligence at all. A benchmark of skill therefore measures, as much as anything, how much effort its builders invested in that skill. Chollet's paper set out to define intelligence in a way that factored this out, and to build a test that measured what was left. Intelligence as skill-acquisition efficiency Chollet's definition is compact: "The intelligence of a system is a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty." Each term does work. Skill-acquisition efficiency means that what matters is not the level of skill a system reaches but how much it took to get there. A system that becomes good at a new task after a few examples is more intelligent, in this sense, than one that needs millions, even if both end up equally skilled. Priors are what the system starts with: its built-in knowledge and assumptions. A test of intelligence must control for priors, or a system could simply be given the answers in advance. Chollet proposed that a fair comparison with humans should assume only the priors that humans themselves are born with or acquire very early, which developmental psychologists such as Elizabeth Spelke have described as "core knowledge." These include the understanding that the world contains objects that persist and do not pass through each other, that some entities act towards goals, an elementary sense of number and counting, and basic notions of geometry and topology such as inside and outside, symmetry and connectedness. Experience is the data the system is exposed to while learning the task. The less it needs, the more intelligent it is. Generalisation difficulty captures how different the new task is from anything the system has already seen. Chollet distinguished several levels. Local generalisation, or robustness, is handling new examples of a known task, such as recognising a cat in a new photograph. Broad generalisation, or flexibility, is handling new tasks within a known domain. Extreme generalisation is handling genuinely novel tasks across domains, the kind of adaptability that humans show when they encounter a new game, tool or problem and figure it out. Most of the benchmarks of the 2010s, he argued, measured local generalisation at best. Human general intelligence is characterised by broad and extreme generalisation. This framework draws on an old distinction in psychology, between crystallised and fluid intelligence, introduced by Raymond Cattell in the 1940s. Crystallised intelligence is accumulated knowledge and skill; fluid intelligence is the capacity to reason about and adapt to new problems. Standard AI benchmarks overwhelmingly measured crystallised intelligence, and large language models, trained on a significant fraction of all human text, are in a sense machines for acquiring crystallised intelligence at enormous scale. Chollet's goal was a test of fluid intelligence that a machine with vast crystallised knowledge could not pass on that knowledge alone. The framework has an important implication that sets it apart from most earlier thinking. It implies that intelligence is not a property of a system's outputs on a fixed task at all. It is a property of the relationship between what a system knew, what it was shown and what it could then do. Two systems that produce exactly the same answers on a test could differ greatly in intelligence, if one was given far more relevant prior knowledge or training. Turing's test, the Winograd schemas and the benchmarks of Table 2 all measure outputs. Chollet was proposing to measure efficiency. The Abstraction and Reasoning Corpus The benchmark Chollet released with the paper turned this definition into a set of puzzles. Each ARC task consists of a few demonstration pairs, usually three, each showing an input grid and an output grid. The grids are up to thirty cells by thirty, and each cell is one of ten colours. The solver must infer the transformation that turns each input into its output, and then apply it to a new test input, producing the output grid exactly, cell by cell. The transformations are things like: fill each enclosed region with the colour of the object that touches it; extend each coloured line until it hits a wall; count the objects of each shape and output a bar whose height matches the count; reflect the pattern across the axis marked by a grey line; find the object that differs from the others and output it alone. Described in words, many sound trivial. Presented as coloured grids without explanation, they require the solver to perceive objects, form hypotheses about what changed and why, and test those hypotheses against the examples. Most adults can solve most tasks within a minute or two, often with a characteristic moment of recognition. Some take longer, and a few are genuinely tricky. The design choices mirror the definition. The tasks assume only core-knowledge priors: objects, counting, basic geometry, simple goal-directedness. They use no language, no cultural knowledge and no specialised domain expertise, so that crystallised knowledge provides little advantage. Each task is novel, invented by hand, so that it cannot be solved by retrieving a memorised solution. And each provides only a handful of examples, so that what is measured is efficiency of learning from very little experience. The original release contained 400 training tasks and 400 evaluation tasks made public, with a further set held privately so that systems could be tested on puzzles their builders had never seen. Human studies confirmed that the puzzles were accessible. Researchers at New York University tested people on samples of the tasks and found that ordinary adults, with no special training, solved most of them; a 2024 study of the full public evaluation set found that typical participants solved roughly two-thirds of the tasks they attempted, and that almost every task was solved by at least some participants. This was the gap that made ARC important. The tasks were easy for people and, as it turned out, extremely hard for machines. One puzzle, three solvers It helps to walk through a single task of the kind ARC contains, described in words, and to compare how different solvers approach it. Suppose the first demonstration shows a black grid containing a few scattered shapes: a blue square, a red L-shape and a small green bar, each made of a handful of cells. Somewhere in the grid is a single grey cell. In the output, every shape has moved in a straight line towards the grey cell's row until it touches that row, and nothing else has changed. The second demonstration has different shapes, different colours and the grey cell in a different place, but the same thing has happened: everything has slid towards the grey row and stopped. The third demonstration has the grey cell in a column rather than a row, marked by a line, and the shapes have slid horizontally to meet it. The test input shows yet another arrangement. A person looking at this typically does several things in quick succession, mostly without noticing. They parse the grid into objects, seeing the blue cells as one square rather than nine separate cells. They notice that the objects keep their shapes and colours, so the transformation is a movement rather than a redrawing. They see that the grey marker is special because it is the only thing that does not move and because everything moves relative to it. They form the hypothesis that objects slide towards the marker until contact, check it against the third example, adjust it to cover the column case and apply it. The whole process draws on exactly the core-knowledge priors Chollet listed: objects that persist, motion, contact and a reference point. A program-search system, of the kind that won in 2020, approaches it differently. It has a library of primitive operations, perhaps hundreds, such as "find connected components," "translate object," "find unique colour" and "extend until collision." It enumerates combinations of these operations, runs each combination on the demonstration inputs and keeps any combination that reproduces all the demonstration outputs exactly. If its library happens to contain the right building blocks, and the correct combination is short enough to be found in the time allowed, it succeeds. If the task requires an idea not in the library, such as "the marker may be a row or a column," it fails, however long it searches. A language model, given the grids as rows of numbers, approaches it differently again. It has seen enormous amounts of text, including, quite possibly, discussions of puzzles like this one, and it will try to produce an output grid directly, token by token. Before the reasoning models of late 2024, this usually failed, because the model had no reliable way to perceive the grid as objects, no mechanism for testing its hypothesis against the demonstrations and no way of correcting itself if the first guess was wrong. The methods that eventually worked gave it those missing pieces, by letting it write and test programs, by training it briefly on the demonstrations or by letting it reason at length and check its own work. The comparison makes Chollet's point concrete. All three solvers may produce the right answer. Only one of them, the person, did so using a small set of general-purpose priors, a few examples and no prior exposure to this particular kind of puzzle. The program searcher relied on a library its designer had written with ARC in mind. The language model relied on knowledge distilled from its training. To know which kind of solver one is looking at, one has to know what it started with and what it cost to get to the answer, not merely whether the answer was right. Five years of resistance The first ARC competition, run on the data science platform Kaggle in 2020, attracted hundreds of teams. The winning entry, by a competitor using the handle "icecuber," solved roughly a fifth of the hidden tasks. It did not use machine learning in any significant way. It was a carefully engineered search over combinations of hand-written grid transformations, a domain-specific language of operations such as cropping, recolouring and tiling, tried in combination until one sequence reproduced the demonstrations. The approach worked when the hidden task happened to be expressible in a short combination of the pre-written operations, and failed otherwise. Meanwhile, the deep learning systems that were saturating every other benchmark scored essentially nothing. Large language models, given the grids as text, could rarely produce a correct output. Even GPT-4o, in 2024, scored only in the single digits on the public evaluation set when used straightforwardly. This was remarkable at the time. Here were systems that could pass the bar examination, write working code and hold fluent conversations, failing at puzzles that a ten-year-old could solve. To Chollet, it was exactly what his theory predicted: the language models had vast crystallised intelligence and very little fluid intelligence. They could apply what they had seen but could not efficiently learn something genuinely new. In June 2024 Chollet and Mike Knoop, a co-founder of the software company Zapier, launched the ARC Prize, a public competition with a pool of more than a million dollars. The grand prize would go to the first open-source system to score 85 percent on the private evaluation set while running within strict limits on Kaggle: no internet access and a fixed, modest allowance of computing time. The limits were essential to the spirit of the test. A system that could achieve 85 percent only by spending vast amounts of computation per task would not be showing efficient skill acquisition. The 2024 competition produced the first serious break in ARC's resistance, and it came from a technique called test-time training. Rather than using a fixed model to produce answers, competitors took a neural network and, for each new task, briefly trained it further on that task's demonstration pairs, often augmented by rotating, reflecting and recolouring them to create more examples. The model thus adapted itself to each task before attempting it. The team MindsAI, which had pioneered the approach, reached about 55.5 percent on the private set but did not open-source its solution, which made it ineligible for prizes; the winning eligible team, the ARChitects, scored 53.5 percent. A separate line of work, demonstrated in mid-2024 by the researcher Ryan Greenblatt, showed that a large language model prompted to write thousands of candidate Python programs for each task, with the programs tested against the demonstrations, could reach roughly half the public evaluation set. Both approaches had something in common: they turned ARC from a problem of applying a fixed model into a problem of search or adaptation at the moment of testing. Neither came close to the grand-prize threshold. The o3 moment On 20 December 2024 OpenAI announced its o3 model and, in collaboration with the ARC Prize team, reported its performance on ARC's semi-private evaluation set. Run within the prize's public leaderboard limit of $10,000 in total compute cost, o3 scored 75.7 percent. Run in a configuration using about 172 times as much computation, it scored 87.5 percent, above the human-level threshold the grand prize had set. This was a large jump. The best language-model-based systems had been scoring far lower months earlier, and the result was widely reported as a breakthrough. The configuration was also enormously expensive by ARC standards, with the high-compute run costing, by the ARC Prize team's estimates, thousands of dollars per task, against the few dollars a human tester would be paid. And o3 had been trained on a portion of the public ARC training set, which the rules allowed but which meant the model was not encountering the format cold. Chollet's response set the tone for everything that followed. He called the result a genuine and surprising step change, evidence that the "reasoning" models trained to produce long chains of thought and search over possible solutions had gained a real capacity for adapting to novel tasks that earlier language models lacked. But he was explicit that it was not AGI. The system remained far less efficient than a human, it still failed on some tasks that people found easy, and Chollet predicted that a new version of the benchmark, then in development, would bring its score down sharply. The point of ARC had never been the score alone; it was the score in relation to the cost of achieving it. A system that could solve a puzzle by spending thousands of dollars of computation on it was showing something, but not efficient skill acquisition in the sense the definition required. The o3 episode marks a turning point in the story of this booklet. For the first time, a benchmark built on an explicit theory of general intelligence, and designed specifically to resist the shortcuts that had undone its predecessors, was substantially solved. And the people who designed it did not declare victory, or defeat. They declared that the measure needed refining, and they knew in what direction. What had been missing from the original ARC was a proper accounting of efficiency. The next version would build it in. What ARC got right Before following the story into 2025 and 2026, it is worth pausing on what the original ARC achieved, because its influence on how people think about measuring intelligence outlasts the benchmark itself. First, it held for five years against the most capable systems in the world, during a period in which almost every other benchmark fell within two. That track record is evidence that its design principles, minimal priors, genuine novelty, tiny amounts of training data per task, captured something that scale alone did not deliver. Second, it made visible a dissociation that conversational tests had hidden. The systems that could pass for human in conversation could not solve simple visual puzzles. This was a direct empirical demonstration that the capacities which come together in people, fluent language and flexible reasoning, could come apart in machines, and so that a test of one could not stand in for the other. Third, it changed how progress was made. The approaches that eventually cracked ARC, test-time training, program search, reasoning models that explore many candidate solutions, all involve the system doing something at the moment it encounters a new problem, rather than simply applying what it learned before. Chollet had argued that intelligence lies in exactly this kind of on-the-fly adaptation. The benchmark pushed the field towards building it. Its weakness was the one that o3 exposed. A test that scores only correctness, and not the resources used to achieve it, can be met by brute force, if enough brute force is available. The definition of intelligence that ARC was built on put efficiency at its centre; the benchmark's scoring did not, except through the competition's compute limits. The two successors that followed were designed to close that gap, and in doing so they produced some of the most surprising results in the history of the field. Chapter 7: Efficiency, Interaction and the Year the Puzzles Fell The lesson the ARC Prize team drew from o3's performance at the end of 2024 was that correctness was not enough. A system that solves a puzzle by generating and checking enormous numbers of candidate answers has demonstrated something, but if intelligence is skill-acquisition efficiency, then how much it spent is part of the measurement, not a footnote to it. The two successor benchmarks, ARC-AGI-2 in 2025 and ARC-AGI-3 in 2026, were built to put efficiency at the centre. The second of them also made a more radical change, abandoning the static puzzle altogether in favour of interactive worlds that a system must explore. What happened to them in 2026 is the most important recent evidence about whether any benchmark can confirm general intelligence, and it deserves to be told carefully. Table 3 sets out how the three versions differ. Table 3. The three versions of the ARC benchmark. Version Year Format What counts Human evidence ARC-AGI-1 2019 Static grid puzzles Correct output Most tasks solved by typical adults ARC-AGI-2 2025 Harder static grid puzzles Correct output and cost per task 407 testers; each task solved by 2+ people ARC-AGI-3 2026 Interactive game environments Actions used relative to humans 486 testers; each environment solved by 2+ people Source: Chollet (2019); ARC Prize Foundation technical reports and announcements (2025, 2026). ARC-AGI-2: harder puzzles, priced answers ARC-AGI-2 was launched in March 2025, together with that year's ARC Prize. Its tasks keep the original format, coloured grids and a few demonstration pairs, but were designed specifically to resist the methods that had worked on the first version. The ARC Prize team identified several kinds of reasoning that had defeated even the strongest systems and built tasks around them. Some required composing several rules at once, so that no single transformation explained the examples. Some required applying a rule that changed depending on context, for instance a transformation that applied differently to objects of different colours. Some required what the designers called in-context symbol definition: the grid itself would establish that a particular shape or colour stood for something, and the solver had to work out that meaning from the example and use it. These are, in a modest way, the kinds of reasoning a person uses when encountering a new notation or a new game. The human calibration was more systematic than for the original. The team tested 407 people in controlled sessions in San Diego, and kept a task only if at least two participants independently solved it within two attempts. Across all attempts, participants succeeded on about two-thirds of the test pairs they tried, with a median time of a little over two minutes. Every task in the final benchmark was therefore one that ordinary people, under time pressure and paid a modest fee, had demonstrably solved. The second change was in reporting. The ARC Prize leaderboard began to present every result as a pair of numbers: a score and a cost per task. A system that achieved 50 percent at a few cents per task was represented as a different kind of achievement from one that achieved 50 percent at thousands of dollars. The Kaggle competition retained strict compute limits, so that its grand prize, again set at 85 percent, could be won only by an efficient, open-source system. At launch, ARC-AGI-2 did what it was designed to do. Pure language models without extended reasoning scored essentially zero, and the strongest reasoning systems were in the low single digits. The gap between humans and machines, which o3 had seemed to close on the first version, was wide open again. It did not stay that way. The 2025 competition, which ran until the autumn, attracted 1,455 teams and more than 15,000 submissions. The winning team, NVARC, scored 24.03 percent on the private set within the Kaggle limits; the ARChitects took second with 16.53 percent and MindsAI third with 12.64 percent. The grand prize was not claimed. But outside the competition, where compute was not capped, frontier systems were moving faster. In the ARC Prize Foundation's year-end analysis, the best verified commercial model was Anthropic's Claude Opus 4.5, at 37.6 percent for about $2.20 per task, and the best verified system overall was a "refinement" approach built by a company called Poetiq on top of Google's Gemini 3 Pro, which reached 54 percent at about $30 per task. The foundation identified the central technical theme of 2025 as refinement loops: systems that generate a candidate solution, test it against the demonstrations, analyse the failure and revise, repeatedly, until the candidate fits. The idea is close to the way a person might work on a puzzle, proposing a rule, checking it and adjusting. Among the most discussed research results was the Tiny Recursive Model, by Alexia Jolicoeur-Martineau, which won the 2025 paper award. It used a network with only about seven million parameters, tiny by the standards of language models, which repeatedly refined its own answer, and it reached about 45 percent on the first ARC benchmark. The result was a reminder that scale was not the only route; a small system that iterates efficiently could do much of what an enormous one did. By the autumn of 2026, ARC-AGI-2 was effectively saturated by frontier systems operating outside the competition's limits. Independent benchmark aggregators tracking published results reported the leading systems from several developers scoring above 90 percent, well above the average human tester. As with ARC-AGI-1, the puzzles that had been designed to separate fluid from crystallised intelligence had fallen, and they had fallen in about eighteen months rather than five years. Whether a system meeting the Kaggle competition's efficiency constraints could also reach the 85 percent threshold remained, at the time of writing, the open question of the 2026 competition. ARC-AGI-3: from puzzles to worlds Even before ARC-AGI-2 was published, the ARC Prize team had concluded that static puzzles had a structural limitation. A grid puzzle presents all the evidence at once: here are the examples, find the rule. Real intelligence, in animals and people, mostly operates differently. An agent enters an unfamiliar situation, acts, observes the consequences, forms a model of how things work, decides what it is trying to achieve and plans accordingly, often without anyone telling it what the goal is. The ability to learn by interacting, rather than by inspecting examples, seemed to them the core of what static benchmarks missed. ARC-AGI-3 was their answer. It consists of small, handcrafted video-game-like environments, each with several levels. An agent sees a grid-based screen and has a small set of possible actions, such as moving in four directions, clicking on a cell or pressing an abstract action button. It is told nothing about what the actions do, what the objects on screen mean or what it is supposed to achieve. It must explore, notice what changes, infer the rules and the goal, and then complete the levels, each of which typically introduces a new twist on the mechanics learned in the previous one. The games are built, like the earlier puzzles, from core-knowledge priors: objects, movement, containment, counting and simple goals. A preview with a handful of games was released in mid-2025, with a small competition attached. The winning preview agent, StochasticGoose from Tufa Labs, reached a score of 12.58 percent; the runner-up, Blind Squirrel, reached 6.71 percent. Both used forms of informed search over possible actions rather than the kind of rule learning a person uses. The full benchmark launched on 25 March 2026 as the centrepiece of an ARC Prize 2026 with a total pool of two million dollars. It contained 25 public environments for developers to study, 55 semi-private environments used to test frontier systems through their providers' interfaces and 55 fully private environments reserved for the competition. Human calibration was carried out in San Francisco: 486 participants made 2,893 attempts on 414 candidate environments, with a median attempt lasting about seven and a half minutes, and only environments independently solved by at least two participants were kept. The scoring rule is the most important design decision in the benchmark, and it is worth understanding exactly. ARC-AGI-3 measures what the foundation calls Relative Human Action Efficiency. For each level, the number of actions a system takes to complete it is compared with the number a human tester needed. The ratio is then squared. If a human completes a level in 10 actions and a system needs 100, the system's score on that level is not 10 percent but 1 percent. The squaring is a deliberate penalty for inefficiency: a system that blunders around, trying every possibility until something works, is scored as having learned very little, even if it eventually succeeds. A system that reaches the human score of 100 percent must, in effect, figure out each environment about as quickly as a person does. At launch, the frontier systems were almost entirely defeated. The foundation's technical report gave the leading scores as 0.50 percent for Anthropic's Claude Opus 4.6, 0.40 percent for Google's Gemini 3.1 Pro, 0.20 percent for OpenAI's GPT-5.4 and 0.10 percent for xAI's Grok 4.20. Humans, by construction, scored 100 percent. A follow-up analysis in May 2026 tested two newer systems, OpenAI's GPT-5.5 and Anthropic's Claude Opus 4.7, which scored 0.43 and 0.18 percent respectively. That May analysis is one of the most revealing documents in the recent history of AI evaluation, because it described not only how badly the systems failed but how. The foundation identified three characteristic failure modes. The first was local perception without global understanding: a model would correctly notice that pressing a particular button rotated a particular object, but would not turn that observation into a rule about how rotation related to the goal of the game. The second was the misapplication of familiar abstractions. Faced with an unfamiliar environment, models repeatedly decided that it was really a version of a game they knew, such as Tetris, Frogger or Sokoban, and acted on the wrong theory. Their crystallised knowledge, in Chollet's terms, was actively getting in the way of learning. The third was what the analysis called victory without learning: a model would complete a level through a lucky sequence of actions, without understanding why it had worked, and then carry its misconceptions into the next level, where they failed. The two systems failed in different styles. One tended to commit confidently to wrong theories, the other to avoid committing to any theory at all. Neither did what the human testers did, which was to form a compact, correct model of the environment from a few observations and use it. The first milestone of the Kaggle competition, at the end of June 2026, showed open-source agents making progress under the competition's constraints. The leading entry, from Tufa Labs, ran a mid-sized open model locally and had it treat each game as an interactive programming problem, writing and running code to probe the environment. The team reported that hand-crafted tools designed to help the model actually hurt its performance, and that letting it improvise worked better. September 2026 On 3 September 2026 the ARC Prize Foundation published an evaluation of a new OpenAI system, GPT-6 Astra. Run through the foundation's standard harness, the common interface used to test all frontier systems on equal terms, it scored 62.7 percent on the semi-private set, at a total cost of a little over $26,000. Run through what the foundation called a provider adapter harness, which allowed the model to use OpenAI's own mechanisms for preserving its internal reasoning state between steps and compacting its context, it scored 99.9 percent, at a cost of about $18,800. In that configuration, the foundation reported, Astra used fewer actions than the median human tester on 96 percent of levels. Four months earlier, the best frontier score had been under half of one percent. A benchmark built over several years, calibrated on nearly five hundred people and designed with an explicit theory of what machines could not yet do, had gone from almost entirely unsolved to effectively saturated, under one of its two testing configurations, in a single model generation. The foundation's statement was carefully worded. It described Astra as meaningful progress towards generalisation. It said plainly that it was not claiming Astra was AGI, and that saturating the benchmark would not represent proof of having achieved AGI. And it said the team was exploring what the next generation of benchmarks should measure, including how to evaluate recursive self-improvement and open-ended innovation. What the collapse means There are two tempting and opposite readings of these events, and both are wrong. The first is that ARC-AGI-3 was simply another benchmark on the treadmill, saturated through some shortcut like its predecessors, and that its fall tells us nothing. This is hard to sustain. The benchmark was specifically built to defeat the known shortcuts. Its environments were novel and private. Its scoring punished brute-force exploration so severely that a system could score well only by learning each environment about as efficiently as a person. The May analysis had documented precisely the cognitive failures that kept systems near zero, and a system that scores at or above human efficiency has, by the benchmark's own logic, stopped making them, at least on these environments. Something real happened. The second reading is that the collapse means general intelligence has arrived, since the most carefully designed test of it has been passed. This is the reading the test's own designers rejected, and they were right to. Their reasons point to what any confirmation of AGI would have to involve. One reason concerns the harness. The difference between 62.7 and 99.9 percent lay not in the model but in the software wrapped around it: how its memory was managed between steps, what reasoning state it could carry forward. This raises a question that turns out to be fundamental. What exactly is the system being evaluated? A human tester brings to an environment a single, integrated mind. A frontier AI system is a model plus a harness plus whatever tools and memory mechanisms its provider builds, and different packaging produces very different results. A test of general intelligence must say which of these it is testing, and why that is the relevant unit. A second reason concerns cost. Astra's best result cost roughly $18,800 for 55 small games, or several hundred dollars per environment, each of which a human tester typically finished in minutes. Under the benchmark's own efficiency metric, which counts actions, Astra was more efficient than humans. Under a metric that counted the energy and money required to choose each action, the comparison would look very different. Chollet's definition of intelligence as skill-acquisition efficiency does not specify which resource is the relevant one, and the answer matters. The deepest reason is the one this booklet has been building towards. ARC-AGI-3 measures a specific and important capacity, efficient learning of novel interactive environments built from core-knowledge priors, on a specific and finite set of environments. A system that masters it has that capacity in that domain. Whether the capacity generalises to the open-ended, messy, socially embedded environments in which human intelligence actually operates is a further question. The history of every earlier test suggests that a capacity demonstrated on a carefully bounded benchmark can turn out, on contact with the world, to be narrower than the benchmark implied. But the history also suggests that each such demonstration covers more ground than the last. The sequence from ARC-AGI-1 to ARC-AGI-3 is not a sequence of failures. It is a sequence of increasingly precise statements of what general intelligence requires, each of which has now been met. That leaves the question with which this booklet began, and which the ARC Prize Foundation itself is now confronting. If the best-designed benchmarks can be met without anyone being willing to call the result general intelligence, what would it take to confirm that general intelligence had been achieved? The final chapter addresses that question directly. Chapter 8: What Would Confirm General Intelligence? In October 2025, when Microsoft and OpenAI announced a restructured partnership, one clause attracted particular attention. Under their earlier arrangement, OpenAI's declaration that it had achieved artificial general intelligence would have changed Microsoft's rights to its technology. Under the new agreement, the companies said, any such declaration would be verified by an independent panel of experts. A question that had been the preserve of philosophers and science fiction had become a term in a commercial contract worth many billions of dollars, and the parties had concluded that it could not be settled by either of them alone, or by any test they could write down in advance. That is a fair summary of where the subject stands. The preceding chapters have traced a sequence of tests, each better designed than the last, each eventually met by systems that their designers declined to call generally intelligent. This chapter asks what, if anything, could do the job. It looks first at the main families of definition now in circulation, then at the philosophical questions that no benchmark reaches, and finally at what a credible confirmation would have to look like. Five ways to define the target The difficulty begins with the fact that "artificial general intelligence" has no agreed definition. That is not merely a matter of vague language. Different definitions pick out different targets, and a system could meet one while failing another by a wide margin. Five approaches are now prominent, and they are compared in Table 4. Table 4. Approaches to defining artificial general intelligence. Approach Representative source AGI means Main difficulty Formal Legg and Hutter, 2007 High expected success across all computable environments Uncomputable; no direct test Skill-acquisition Chollet, 2019 Human-level efficiency at learning novel tasks Choice of tasks and resources Levels Morris et al., 2023 Graded performance across a breadth of tasks Breadth hard to specify Psychometric Hendrycks et al., 2025 Matching an educated adult across ten cognitive domains Human tests may not transfer Economic Nilsson, 2005; OpenAI, 2018 Doing most economically valuable work Depends on markets and deployment Source: the works named; see Notes. The formal approach is the most mathematically rigorous. In a 2007 paper, Shane Legg, who would go on to co-found DeepMind, and Marcus Hutter defined the "universal intelligence" of an agent as its expected performance across every computable environment, with simpler environments given more weight. This captures, in a single formula, the intuition that intelligence is the ability to achieve goals in a wide range of situations. Its weakness is that it cannot be computed, since it sums over infinitely many environments and depends on a measure of simplicity that is itself uncomputable. It is a definition of the ideal, not a test, though every practical benchmark can be seen as a small, biased sample from the environments it sums over. The skill-acquisition approach is Chollet's, discussed in Chapter 6. It shares the formal approach's emphasis on generality but adds efficiency, and it has the great advantage of having produced concrete tests. Its weakness, exposed by the events of 2026, is that any concrete version must choose a finite set of tasks and a particular resource to count, and a system can meet the chosen version without meeting the ideal. The levels approach was proposed by a group of researchers at Google DeepMind led by Meredith Ringel Morris in a 2023 paper titled "Levels of AGI." Rather than treat AGI as a threshold, they proposed a grid with two dimensions: performance, from "emerging" through "competent," "expert" and "virtuoso" to "superhuman," defined relative to percentiles of skilled adults; and generality, from narrow to general. A chess engine is narrow and superhuman; on their classification, the general-purpose chatbots of 2023 were general but only "emerging." The framework's principles are as useful as its levels. It insists that AGI should be defined by capabilities rather than by internal processes or consciousness, that both breadth and depth matter, that what counts is what a system can do rather than whether it has been deployed to do it and that the tasks used should have ecological validity, meaning that they reflect things people actually value. Its weakness is that the generality dimension depends on a list of tasks that is not, and perhaps cannot be, fully specified. The psychometric approach borrows from human intelligence testing. In October 2025 a large group led by Dan Hendrycks, including Yoshua Bengio and other prominent researchers, published "A Definition of AGI," which proposed that AGI means matching the cognitive versatility and proficiency of a well-educated adult. To make this measurable, they drew on the Cattell-Horn-Carroll theory, the most widely used model of the structure of human cognitive abilities, and divided cognition into ten domains, including reasoning, several kinds of memory, knowledge, reading and writing, mathematics, visual and auditory processing and speed. Each domain contributes ten percent to an overall score. On their assessment, GPT-4 scored 27 percent and GPT-5 57 percent. The most striking finding was the shape of the profile rather than the total: the systems were strong in knowledge-heavy domains and very weak in some foundational ones, especially long-term memory storage, the ability to learn something new and retain it over time. The authors described this as a "jagged" profile. The approach's weakness is that tests designed for humans rest on assumptions, for instance that someone who can do one kind of task can probably do related ones, that may not hold for machines, as the story of every benchmark in this booklet illustrates. The economic approach is the oldest in spirit and the most influential in practice. In 2005 the AI pioneer Nils Nilsson proposed an "employment test": a human-level AI should be able to do the jobs that people are paid to do. OpenAI's charter, published in 2018, defines AGI as "highly autonomous systems that outperform humans at most economically valuable work." The appeal is that this definition is tied to consequences that matter and that it does not depend on any particular theory of intelligence. Its weakness is that whether a system does most economically valuable work depends on prices, regulation, adoption, the design of jobs and the cost of computation, none of which are properties of the system's intelligence. A system could be economically transformative without being general, if it did a few very valuable things, or general without being economically dominant, if it were too expensive to run. Measuring autonomy over time One recent line of measurement deserves separate mention because it addresses a gap that all the earlier benchmarks share. Turing's conversation lasted five minutes; an ARC puzzle takes a person a couple of minutes; an ARC-AGI-3 game about seven. Much of what human general intelligence does takes place over hours, days and years: managing a project, learning a field, pursuing a goal through setbacks. In 2025 the research organisation METR published a study titled "Measuring AI Ability to Complete Long Tasks." Rather than scoring systems on questions, it measured the length of real tasks, mostly in software engineering and research, as measured by how long they took skilled humans, that AI agents could complete with a given reliability. It found that this "time horizon" had been doubling roughly every seven months over the preceding six years. The measure has limitations, including its focus on a particular domain and the difficulty of calibrating human task times, but it captures something static benchmarks miss: the ability to sustain coherent, goal-directed work over an extended period, recovering from errors without supervision. Any serious account of general intelligence will need some measure of this kind, and it is notable that long-term memory and sustained autonomy are precisely where the psychometric and interactive tests found current systems weakest. There are also informal tests that capture the same intuition with more colour. Apple's co-founder Steve Wozniak proposed what became known as the coffee test: a machine should be able to enter an ordinary, unfamiliar home and make a cup of coffee, finding the machine, the coffee, the water and the mugs and working out how to operate them. The researcher Ben Goertzel proposed a "robot college student" test, in which a machine enrols at a university, takes the classes and earns a degree. These are not rigorous, but they point at the same thing: general intelligence is shown in unscripted encounters with an open world over extended periods, not in a bounded session with a defined task. The questions benchmarks do not reach Everything so far concerns capability: what a system can do. Many people, when they ask whether a machine is truly intelligent, mean something more, and two questions in particular have shadowed this subject since Turing's paper. The first is understanding. Searle's Chinese Room, discussed in Chapter 2, argued that a system could produce correct outputs without understanding anything. In 2020 the linguists Emily Bender and Alexander Koller gave the argument a modern form, using the image of a hyper-intelligent octopus that taps an undersea telegraph cable and learns to imitate the conversations of two people on remote islands. It learns the statistical form of their language perfectly, but because it has never seen a coconut or a bear, it cannot know what their words refer to, and when one of them asks for help fending off a bear, it has nothing meaningful to offer. Their claim was that a system trained only on linguistic form cannot learn meaning. Critics have replied that modern systems are trained on more than text, including images, video and interaction, and that they can demonstrably use words in ways that track the world, which is at least part of what meaning involves. The debate has not been resolved, but it has shifted. The question is less whether the systems have any grasp of meaning at all, and more whether their grasp is grounded and robust in the way a human's is, or brittle in ways that show up only under unusual conditions. That is a question that interactive, novel, open-ended testing is well placed to probe, which is one reason the shift towards environments like ARC-AGI-3 matters. The second question is consciousness. Is there something it is like to be the system? Does it have experiences, feel anything, have interests of its own? Turing deliberately set this question aside, and most definitions of AGI, including the levels framework, explicitly exclude it. There are good reasons to do so. Capability and consciousness appear to be separable at least in principle: a system could, it seems, be highly capable without any inner experience, and creatures with very limited capabilities appear to have experiences. And there is no agreed test for consciousness even in animals. In 2023 a group of philosophers and scientists led by Patrick Butlin and Robert Long published a report that took a pragmatic approach: rather than seeking a test, they extracted from the leading scientific theories of consciousness a set of "indicator properties," architectural features such as a global workspace that broadcasts information across subsystems, and assessed whether AI systems had them. They concluded that no current system was a strong candidate, but that there were no obvious technical barriers to building systems that satisfied many of the indicators. The relationship between these questions and AGI is easily confused. Confirming that a system is generally intelligent, in any of the senses in Table 4, would not confirm that it understands as we do or that it is conscious. Failing to confirm consciousness would not show that it lacked general intelligence. These are distinct inquiries, with different methods, and much public confusion comes from running them together. A person who says that a system "isn't really intelligent" because it lacks feelings, and a person who says it "must be conscious" because it is so capable, are making the same error from opposite directions. The questions matter enormously, especially the moral question of whether such systems could have interests that deserve consideration. But they are not questions that a better benchmark will answer. What confirmation would require If no single test can confirm general intelligence, and no definition is uncontested, does that mean the question is empty? It does not. It means that confirmation must take a different form from the one the history of tests assumed. The pattern of the preceding chapters points to five requirements. First, the evaluation must be continual rather than fixed. Every fixed test has been met, eventually, by something narrower than what it targeted. A claim of general intelligence should be tested against tasks that did not exist when the system was built, generated or chosen after its training was complete, and renewed as long as the claim is made. The ARC Prize Foundation's practice of keeping its environments private and building new versions is a step in this direction; the logical end point is a standing process that never stops producing new tests. Second, it must be adversarial. The Turing test failed partly because its judges did not try hard enough; the Winograd challenge failed partly because its designers could not anticipate every shortcut. The most informative evaluations of the last decade have come from people actively trying to find what a system cannot do: red teams, adversarial dataset builders and researchers like those who wrote the analysis of ARC-AGI-3 failure modes. A claim of generality is a universal claim, and universal claims are tested by searching for counterexamples. The confirmation of AGI would consist, in large part, in the sustained failure of skilled, well-resourced attempts to find tasks within human competence that the system cannot learn. Third, it must measure learning, not just performance, and it must count resources. Chollet's central insight survives the fall of his benchmarks. What distinguishes general intelligence from a large collection of skills is the efficient acquisition of new ones. Evaluations should therefore present systems with genuinely new domains and measure how much experience, computation and assistance they need to become competent, and how well they retain and build on what they learned. The psychometric finding that current systems are weakest at long-term memory storage is directly relevant here: a system that cannot learn durably from experience after deployment is missing something that every human general intelligence has. Fourth, it must specify the system. The gap between Astra's two ARC-AGI-3 scores showed that a model and its harness can behave very differently. A claim of general intelligence must say what is being claimed for: a model, a model with particular tools, an agent with persistent memory, a whole organisation of agents. Each is a legitimate object of evaluation, but they are different objects, and confusion between them will make any verdict meaningless. Fifth, it must be governed. The Microsoft and OpenAI arrangement, with its independent expert panel, recognised something important: when the question has large consequences, those who have an interest in the answer should not be the ones to give it. The same is true of the field as a whole. Benchmarks designed and reported by the developers whose systems they score are subject to every pressure described in Chapter 5. Independent evaluators, transparent methods and access to the systems being assessed are as necessary to confirming AGI as clever test design is. None of this yields a single moment at which the question is answered. It yields something more like a legal or scientific verdict: a judgement, by people with appropriate expertise and independence, that the evidence from many sources, gathered over time and under adversarial pressure, supports the claim beyond reasonable doubt, and that the remaining failures are no longer of the kind that distinguish a general intelligence from a narrow one. That is less satisfying than a pass mark. It is also the only standard that the history of the subject suggests could hold. Conclusion Seventy-six years separate Turing's paper from the evaluation of GPT-6 Astra on ARC-AGI-3, and across that span the tests changed almost beyond recognition. A conversation through a teleprinter became a pronoun in a sentence, then a grid of coloured cells, then a silent game with no instructions. The judges changed too: a human interrogator, then an answer key, then a formula that compares a machine's actions with a person's. What did not change was the outcome. Each test captured something real, each was met, and each time the people best placed to interpret the result declined to say that general intelligence had arrived. It would be easy to read this as a story about moving goalposts, and some observers do. On that reading, the sceptics define intelligence as whatever machines cannot yet do, and retreat to a new definition every time one falls. There is something to this complaint. The phrase "that isn't real intelligence" has been said of chess engines, translation systems, conversational models and puzzle solvers, and at some point the burden of proof shifts. Turing's own argument, that we extend to other people the "polite convention" that they think on behavioural evidence alone, has never been answered so much as avoided. But the history examined in this booklet supports a different reading. The goalposts did not move arbitrarily. Each one moved because the previous test was shown, through careful analysis, to be passable by something narrower than what it was meant to detect. The Turing test fell to persona prompts that exploited the judges' reliance on style. The Winograd schemas fell to statistical associations and dataset artifacts that their structure was supposed to exclude. The first ARC fell to massive search paid for with computation that the benchmark's scoring did not count. Each refinement was a correction, made in response to specific evidence, and each produced a sharper statement of what general intelligence requires. That is not goalpost-moving. It is how measurement improves in any science, when an instrument turns out to respond to something other than the quantity it was built to measure. What follows from this is the argument of the booklet, and it has three parts. The verdict cannot be a score The first is that general intelligence will not be confirmed by a test result. The reason is not that machines will never be generally intelligent, nor that the concept is meaningless. It is that "general" is a claim about every task within some broad range, while any test is a finite sample from that range, and the sample can always be covered by something less than the whole. The more a single test is used to make the claim, the more effort will be directed at that sample, and the less its results will mean. This is Goodhart's law in its most consequential form. What can confirm the claim, to the extent anything can, is accumulated failure to refute it: a continuing, adversarial, independently governed process that keeps generating new tasks, measures how efficiently a system learns them, counts the resources it uses and searches actively for the things it cannot do. The verdict will come, if it comes, as a judgement of experts weighing a body of evidence, not as a number crossing a line. People waiting for a single dramatic announcement will be waiting for the wrong thing. The gap has narrowed to a recognisable shape The second part concerns what the evidence currently shows. The chapters on ARC traced a striking change between late 2024 and late 2026. The tasks that most clearly separated human from machine, novel abstract puzzles and interactive environments learned from scratch, went from almost entirely unsolved to solved at or above human level. A reader in 2023 who was told that systems would soon be more action-efficient than people at figuring out unfamiliar games, with no instructions, would have been entitled to conclude that something very like general intelligence was close. And yet the shape of what remains is now unusually clear. The psychometric assessment found current systems weakest at storing new knowledge and retaining it over time. The METR measurements showed rapid growth in the length of tasks systems can complete, but from a starting point far below the months and years over which people pursue projects. The ARC-AGI-3 results depended heavily on the harness that managed the model's memory, which is to say on scaffolding built by people. The common thread is durable, self-directed learning: the ability to encounter something new in the world, learn it, keep it, build on it and decide for oneself what to learn next. Static benchmarks, even interactive ones lasting minutes, are poorly placed to test this, because it unfolds over time and in environments nobody designed. If the remaining gap between current systems and general intelligence is anywhere, it is most likely here, and the next generation of evaluation will need to be built around it. The ARC Prize Foundation's stated interest in evaluating open-ended innovation and self-improvement points the same way. The reader's questions The third part is practical. Claims that AGI has been achieved, or is about to be, will be made with increasing frequency, by companies, researchers and commentators, some careful and some not. The history in this booklet suggests five questions to ask of any such claim. What exactly was tested? A model, a model with tools, an agent with memory, a system with a human-written prompt? The difference between 62.7 and 99.9 percent on the same benchmark lay in the answer to this question. Who designed the test, and who ran it? A benchmark designed or reported by the developer of the system it scores is subject to every pressure described in Chapter 5, whatever the good faith of the people involved. Was the test new to the system? If its tasks or anything resembling them could have appeared in training, the result says more about memory than about learning. What did it cost? A capacity that requires vastly more computation, money or energy than a person needs to do the same thing is a real capacity, but it is not efficient skill acquisition, and efficiency is close to the heart of what most people mean by intelligence. What does the system still fail at? This is the most important question and the one least often answered in announcements. A claim of generality is a claim that the failures have stopped being the kind that matter. The evidence for it is not the list of things the system can do but the shortness, and the character, of the list of things it cannot. Turing ended his paper by saying that we can see only a short distance ahead, but plenty there that needs to be done. The distance ahead is shorter now, and the tests are better. What has been learned over seventy-six years is not how to build a test that settles the question, but why no such test can exist, and what kind of patient, adversarial, independent judgement must take its place. That knowledge will matter more over the next few years than any score. Notes Introduction 1. Cameron R. Jones and Benjamin K. Bergen, "Large Language Models Pass the Turing Test," arXiv preprint 2503.23674 (2025). 2. ARC Prize Foundation, "Announcing ARC-AGI-3," arcprize.org, 25 March 2026; "OpenAI's GPT-6 Astra on ARC-AGI-3," arcprize.org, 3 September 2026. Chapter 1 1. A. M. Turing, "Computing Machinery and Intelligence," Mind 59, no. 236 (1950): 433–460. All quotations in the chapter are from this paper. 2. The 1952 BBC radio discussion, "Can Automatic Calculating Machines Be Said to Think?", with R. B. Braithwaite, Geoffrey Jefferson and Max Newman, is reprinted in B. Jack Copeland, ed., The Essential Turing (Oxford University Press, 2004). Chapter 2 1. Joseph Weizenbaum, "ELIZA: A Computer Program for the Study of Natural Language Communication between Man and Machine," Communications of the ACM 9, no. 1 (1966): 36–45. 2. Joseph Weizenbaum, Computer Power and Human Reason: From Judgment to Calculation (W. H. Freeman, 1976). 3. Kenneth Mark Colby, Artificial Paranoia: A Computer Simulation of Paranoid Processes (Pergamon, 1975). 4. Stuart M. Shieber, "Lessons from a Restricted Turing Test," Communications of the ACM 37, no. 6 (1994): 70–78. 5. John R. Searle, "Minds, Brains, and Programs," Behavioral and Brain Sciences 3, no. 3 (1980): 417–457. 6. Ned Block, "Psychologism and Behaviorism," Philosophical Review 90, no. 1 (1981): 5–43. 7. Robert M. French, "Subcognition and the Limits of the Turing Test," Mind 99, no. 393 (1990): 53–65. 8. Patrick Hayes and Kenneth Ford, "Turing Test Considered Harmful," Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (1995): 972–977. 9. Stevan Harnad, "Other Bodies, Other Minds: A Machine Incarnation of an Old Philosophical Problem," Minds and Machines 1 (1991): 43–54. Chapter 3 1. Cameron R. Jones and Benjamin K. Bergen, "People Cannot Distinguish GPT-4 from a Human in a Turing Test," arXiv preprint 2405.08007 (2024). 2. Jones and Bergen, "Large Language Models Pass the Turing Test" (2025 preprint); published version in Proceedings of the National Academy of Sciences (May 2026), including the fifteen-minute replication. 3. Daniel C. Dennett, "The Problem with Counterfeit People," The Atlantic, May 2023. 4. AI21 Labs, "Human or Not" results, reported in Daniel Jannai et al., "Human or Not? A Gamified Approach to the Turing Test," arXiv preprint 2305.20010 (2023). 5. Luis von Ahn, Manuel Blum, Nicholas J. Hopper and John Langford, "CAPTCHA: Using Hard AI Problems for Security," Advances in Cryptology: EUROCRYPT 2003. 6. OpenAI, "GPT-4 Technical Report" and accompanying system card (2023), describing the Alignment Research Center's evaluation. Chapter 4 1. Terry Winograd, Understanding Natural Language (Academic Press, 1972). 2. Hector J. Levesque, Ernest Davis and Leora Morgenstern, "The Winograd Schema Challenge," Proceedings of the Thirteenth International Conference on Principles of Knowledge Representation and Reasoning (2012). 3. Hector J. Levesque, "On Our Best Behaviour," Artificial Intelligence 212 (2014): 27–35. 4. Trieu H. Trinh and Quoc V. Le, "A Simple Method for Commonsense Reasoning," arXiv preprint 1806.02847 (2018). 5. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula and Yejin Choi, "WinoGrande: An Adversarial Winograd Schema Challenge at Scale," Proceedings of AAAI (2020). 6. Paul Trichelair et al., "How Reasonable Are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAG," Proceedings of EMNLP-IJCNLP (2019). 7. Yanai Elazar et al., "Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema," Proceedings of EMNLP (2021). 8. Rowan Zellers et al., "SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference," Proceedings of EMNLP (2018); Rowan Zellers et al., "HellaSwag: Can a Machine Really Finish Your Sentence?", Proceedings of ACL (2019). 9. Vid Kocijan, Ernest Davis, Thomas Lukasiewicz, Gary Marcus and Leora Morgenstern, "The Defeat of the Winograd Schema Challenge," Artificial Intelligence 325 (2023). Chapter 5 1. Jia Deng et al., "ImageNet: A Large-Scale Hierarchical Image Database," CVPR (2009); Kaiming He et al., "Delving Deep into Rectifiers," ICCV (2015). 2. Alex Wang et al., "GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding" (2018); Alex Wang et al., "SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems," NeurIPS (2019). 3. Dan Hendrycks et al., "Measuring Massive Multitask Language Understanding," ICLR (2021); OpenAI, "GPT-4 Technical Report" (2023). 4. Aryo Pradipta Gema et al., "Are We Done with MMLU?", arXiv preprint 2406.04127 (2024). 5. David Rein et al., "GPQA: A Graduate-Level Google-Proof Q&A Benchmark" (2023); Elliot Glazer et al., "FrontierMath" (Epoch AI, 2024); Long Phan et al., "Humanity's Last Exam" (Center for AI Safety and Scale AI, 2025). 6. Robert Geirhos et al., "Shortcut Learning in Deep Neural Networks," Nature Machine Intelligence 2 (2020): 665–673. 7. R. Thomas McCoy, Ellie Pavlick and Tal Linzen, "Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference," Proceedings of ACL (2019). 8. Karl Cobbe et al., "Training Verifiers to Solve Math Word Problems" (GSM8K), arXiv preprint 2110.14168 (2021); Hugh Zhang et al., "A Careful Examination of Large Language Model Performance on Grade School Arithmetic" (GSM1k), arXiv preprint 2405.00332 (2024); Iman Mirzadeh et al., "GSM-Symbolic," arXiv preprint 2410.05229 (2024). Horace He's Codeforces observation was posted publicly in March 2023 and widely reported. 9. Shivalika Singh et al., "The Leaderboard Illusion," arXiv preprint 2504.20879 (2025). 10. Marilyn Strathern, "'Improving Ratings': Audit in the British University System," European Review 5, no. 3 (1997): 305–321. 11. Douwe Kiela et al., "Dynabench: Rethinking Benchmarking in NLP," Proceedings of NAACL (2021). Chapter 6 1. François Chollet, "On the Measure of Intelligence," arXiv preprint 1911.01547 (2019). The definition of intelligence is quoted from this paper. 2. Solim LeGris et al., "H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmark," arXiv preprint 2409.01374 (2024). 3. ARC Prize Foundation, "ARC Prize 2024: Technical Report" (2024). 4. Ekin Akyürek et al., "The Surprising Effectiveness of Test-Time Training for Abstract Reasoning," arXiv preprint 2411.07279 (2024). 5. ARC Prize Foundation, "OpenAI o3 Breakthrough High Score on ARC-AGI-Pub," arcprize.org, December 2024. Chapter 7 1. ARC Prize Foundation, "ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems," arXiv preprint 2505.11831 (2025). 2. ARC Prize Foundation, "ARC Prize 2025 Results and Analysis," arcprize.org. 3. Alexia Jolicoeur-Martineau, "Less Is More: Recursive Reasoning with Tiny Networks" (2025). 4. ARC Prize Foundation, "ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence," technical report (2026). 5. ARC Prize Foundation, "Analyzing GPT-5.5 and Opus 4.7 with ARC-AGI-3," arcprize.org, 1 May 2026. 6. ARC Prize Foundation, "ARC Prize 2026: ARC-AGI-3 Milestone Prize #1," arcprize.org, July 2026. 7. ARC Prize Foundation, "OpenAI's GPT-6 Astra on ARC-AGI-3," arcprize.org, 3 September 2026. 8. ARC-AGI-2 scores for frontier systems in autumn 2026 as compiled by the BenchLM aggregator (benchlm.ai), accessed October 2026; individual figures vary in verification status. Chapter 8 1. Shane Legg and Marcus Hutter, "Universal Intelligence: A Definition of Machine Intelligence," Minds and Machines 17 (2007): 391–444. 2. Meredith Ringel Morris et al., "Levels of AGI for Operationalizing Progress on the Path to AGI," arXiv preprint 2311.02462 (2023). 3. Dan Hendrycks et al., "A Definition of AGI," arXiv preprint 2510.18212 (2025). 4. Nils J. Nilsson, "Human-Level Artificial Intelligence? Be Serious!", AI Magazine 26, no. 4 (2005): 68–75. 5. OpenAI Charter (2018). 6. Thomas Kwa et al., "Measuring AI Ability to Complete Long Tasks," METR, arXiv preprint 2503.14499 (2025). 7. Emily M. Bender and Alexander Koller, "Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data," Proceedings of ACL (2020). 8. Patrick Butlin, Robert Long et al., "Consciousness in Artificial Intelligence: Insights from the Science of Consciousness," arXiv preprint 2308.08708 (2023). Further Reading A. M. Turing, "Computing Machinery and Intelligence" (1950). Short, witty and still the best starting point. Read the objections section in particular; nearly every later debate is anticipated there. B. Jack Copeland, ed., The Essential Turing (Oxford University Press, 2004).** Turing's key papers with expert introductions, including the 1952 radio discussion in which he sketched a jury version of his test. John Searle, "Minds, Brains, and Programs" (1980), and Ned Block, "Psychologism and Behaviorism" (1981). The two classic arguments that behaviour alone cannot settle whether a system thinks. Searle's paper is published with peer commentaries that map the replies. Vid Kocijan et al., "The Defeat of the Winograd Schema Challenge" (2023). A clear, candid post-mortem by researchers including the challenge's own designers, and a model of how to learn from a benchmark's fall. François Chollet, "On the Measure of Intelligence" (2019). The most influential modern attempt to define intelligence in a testable way. Long, but the first half is accessible to non-specialists and the argument about skill versus intelligence is essential. ARC Prize Foundation blog and technical reports (arcprize.org). The primary record of ARC-AGI-2 and ARC-AGI-3, including human calibration studies, cost-adjusted leaderboards and unusually frank analyses of how frontier systems fail. Cameron R. Jones and Benjamin K. Bergen, "Large Language Models Pass the Turing Test" (2025; PNAS, 2026). The most rigorous modern run of Turing's three-party game, with an illuminating analysis of what interrogators actually do. Meredith Ringel Morris et al., "Levels of AGI" (2023), and Dan Hendrycks et al., "A Definition of AGI" (2025). Two contrasting attempts to make AGI measurable, one graded by performance and generality, the other grounded in the psychology of human cognitive abilities. Melanie Mitchell, Artificial Intelligence: A Guide for Thinking Humans (Farrar, Straus and Giroux, 2019).** A lucid account, by a researcher who has worked on abstraction and analogy, of why impressive performance and real understanding can come apart. Thomas Kwa et al., "Measuring AI Ability to Complete Long Tasks" (METR, 2025). The study that introduced the "time horizon" measure, a useful counterweight to question-and-answer benchmarks. Hashtags: #TuringTest #AlanTuring #TheImitationGame #MachineIntelligence #MeasuringMachineIntelligence #ArtificialIntelligence #ArtificialGeneralIntelligence #AGI #AIEvaluation #AIBenchmarks #IntelligenceTesting #MachineReasoning #CognitiveScience #ComputationalIntelligence #PhilosophyOfAI #PhilosophyOfMind #MachineConsciousness #MachineUnderstanding #HumanVsMachine #HumanLikeAI #ConversationalAI #LargeLanguageModels #LLMEvaluation #ChatbotEvaluation #ELIZAEffect #ChineseRoomArgument #WinogradSchemaChallenge #CommonsenseReasoning #NaturalLanguageUnderstanding #ARCAGI #ARCAGI2 #ARCAGI3 #AbstractionAndReasoning #AbstractReasoning #Generalization #OutOfDistributionGeneralization #TransferLearning #FewShotLearning #SkillAcquisition #BenchmarkSaturation #BenchmarkContamination #DataContamination #ShortcutLearning #GoodhartsLaw #AdversarialEvaluation #RobustnessTesting #HumanLevelPerformance #HumanAIComparison #IntelligenceMetrics #FutureOfAI
- Child Protection and Adoption Law (State Intervention, Foster Care, and International Adoption)
Download the Book (PDF): Introduction Every adoption begins with a loss. Before a child can acquire a new legal family, the law must first take away the old one, and it does so with a completeness that has almost no parallel elsewhere in civil law. A judgment terminating parental rights does not suspend a relationship or limit it. It extinguishes it. The parent becomes, in the eyes of the state, a legal stranger to the child: no right to visit, no right to be told where the child lives, no right to be consulted about medical care, religion, schooling or anything else. The child loses a claim to inherit, often loses a name, and, if an adoption follows, receives a new birth certificate listing different parents as though the first ones had never existed. The law does this thousands of times a year in the United States, overwhelmingly to parents who are poor, frequently to parents who struggle with addiction or mental illness, and disproportionately to Black and Native American families. It does it through a system that is supposed to be protective rather than punitive, run by agencies whose stated mission is family preservation, in courts that most Americans will never see because the proceedings are closed. And at the other end of that process sits adoption, an institution the public tends to regard with unusual warmth, as a generous act that gives a child a home. Both images are true. Neither is complete. This book is about the legal machinery that connects the two. It follows the path a child's case takes from the moment someone calls a hotline, through investigation, emergency removal, the dependency court, the statutory clocks that push toward permanence, the termination of parental rights, and finally into adoption, whether by a relative, a foster parent, a stranger in the same state, or a family on another continent. It covers the constitutional doctrine that is supposed to restrain the state at each step, the federal funding statutes that in practice shape what states do far more than the Constitution does, the special regime Congress built for Native American children, the litigation that has tried and often failed to fix broken foster care systems, and the international treaty framework that governs adoption across borders. The argument The controlling idea is simple to state. The legitimacy of adoption is borrowed. An adoption is only as sound as the severance that preceded it. When the earlier stages of the process are fair, accurate and careful, when the state removes children only when it must, works genuinely to return them, and terminates parental rights only on clear proof that reunification cannot happen, then adoption is what its admirers say it is: a way of giving a child permanence after a real family has been lost. When those earlier stages fail, adoption does not cure the failure. It seals it. The finality that makes adoption valuable to children is the same finality that makes a wrongful termination nearly impossible to undo. This idea explains why so much of the law examined here is procedural. American constitutional law treats the relationship between parent and child as a fundamental liberty interest, and the Supreme Court has said repeatedly that the state may not destroy it without a demanding showing. But the Court has also declined to require appointed counsel for every parent facing termination, has declined to impose a constitutional duty on the state to protect children from their own parents, and has left the actual running of the system almost entirely to the states and to federal funding conditions. The protection a family receives therefore depends less on grand declarations of right than on a series of small procedural moments: whether a caseworker had enough evidence to remove, whether a judge made a genuine reasonable-efforts finding or signed a form, whether a parent had a lawyer at the first hearing, whether the fifteen-month clock was applied mechanically or with judgment. The same logic governs intercountry adoption, where it is even starker. The central problem of international adoption has never been a shortage of willing families. It has been the integrity of the supply: whether the children placed abroad were genuinely orphaned or relinquished, or whether they were bought, coerced, falsely documented or simply taken. The 1993 Hague Adoption Convention is at bottom an attempt to guarantee that the severance in the country of origin was lawful before the adoption in the receiving country is recognized. The history of scandal that led several countries to close their programs, and led South Korea to confront decades of irregular overseas adoptions, is a history of what happens when that guarantee is absent. What the book covers and what it leaves aside The focus is American law, because the United States has the largest and most litigated child welfare system in the world and because it was, for decades, the largest receiving country in intercountry adoption. Comparative notes appear where another system's choices illuminate the American one: England's different approach to adoption from care, for example, or the way countries of origin have regulated the flow of children abroad. The final chapter is necessarily international. Chapter 1 sets out the constitutional frame, from the early twentieth-century cases establishing parental rights through the modern decisions on the burden of proof, the right to counsel and the state's duty to protect. Chapter 2 examines the front end of the system: reporting, investigation and the emergency removal of children, including the Fourth Amendment questions that arise when caseworkers enter homes and interview children at school. Chapter 3 turns to the dependency court and the federal statutes that structure it, particularly the reasonable-efforts requirement and the timelines imposed by the Adoption and Safe Families Act of 1997. Chapter 4 treats the termination of parental rights itself, its grounds, its standard of proof and its recurring hard cases. Chapter 5 examines the Indian Child Welfare Act and the broader legal fights over race, culture, religion and identity in placement. Chapter 6 asks what happens when the state itself becomes the harmful parent, and why litigation has been such an imperfect tool for reforming failing foster care systems. Chapter 7 covers domestic adoption law, including consent, fathers' rights, subsidies and the troubling phenomenon of adoption breakdown. Chapter 8 covers intercountry adoption and the Hague Convention. Some subjects are deliberately left out or touched only lightly. Juvenile delinquency, which shares courtrooms and sometimes children with the dependency system, is a different body of law. Private custody disputes between parents are mentioned only where they illuminate state intervention. Assisted reproduction and surrogacy, which raise their own questions about legal parenthood, are outside the frame except where they intersect with adoption. A note on terms and on law that moves Child welfare law has its own vocabulary, and it varies from state to state. What one state calls a "dependency" case another calls a "child in need of protection" or "abuse and neglect" proceeding; the court may be a juvenile court, a family court or a specialized division. This book uses "dependency" and "child protection" interchangeably for the civil proceedings in which the state seeks to supervise or remove a child because of parental maltreatment. "Foster care" refers to any out-of-home placement under state custody, whether with licensed strangers, relatives or in group settings. Much of this area is governed by statutes and regulations that change frequently, and several important federal rules were issued, challenged, vacated or rescinded between 2023 and 2026. Where the book describes a recent development, it does so as of late 2026, and readers dealing with a live case should check current law in the relevant state. This book is an explanation of how the system works and why, not legal advice. One final point of orientation. It is easy to write about child protection as though it were a morality play, with heroic caseworkers rescuing children from monstrous parents, or with a predatory state tearing apart innocent families. The reality in most cases is neither. Most children who enter foster care do so because of neglect rather than physical or sexual abuse, and much of what is labeled neglect is entangled with poverty, housing instability, substance use and untreated illness. Most parents in the system love their children and have failed them in real ways. Most caseworkers are overburdened and doing their honest best under rules that often give them little room for judgment. The law's job is to make hard decisions under uncertainty about families in crisis, and the question this book keeps asking is whether it is designed to make those decisions well, and to correct them when it does not. Chapter 1: The Constitutional Family American child protection law rests on a tension the Constitution never resolves. On one side stands the liberty of parents to raise their children as they see fit, a right the Supreme Court has called among the oldest of the fundamental liberties it recognizes. On the other stands the power of the state, as parens patriae, to protect children who cannot protect themselves. Neither principle is absolute, and the whole of the field can be read as a long negotiation over where one ends and the other begins. The constitutional cases do not run the system; state statutes, federal funding conditions and local practice do that. But the cases set the floor beneath which no state may go, and they supply the vocabulary in which every argument about intervention is conducted. The foundation: parents as the child's first guardians The modern doctrine begins with two cases from the 1920s that had nothing to do with abuse or neglect. In Meyer v. Nebraska (1923), the Supreme Court struck down a state law forbidding the teaching of foreign languages to young children, holding that the liberty protected by the Fourteenth Amendment's Due Process Clause includes the right of parents to engage a teacher to instruct their children. Two years later, in Pierce v. Society of Sisters (1925), the Court invalidated an Oregon law requiring children to attend public rather than private schools. The child, the Court said, is not the mere creature of the state; those who nurture him and direct his destiny have the right, coupled with the high duty, to prepare him for additional obligations. These cases established that the parent-child relationship is not simply a creation of state law that the legislature may redefine at will. It has a constitutional dimension. The parent stands between the child and the state, and the state must justify its intrusions. The limits came quickly. In Prince v. Massachusetts (1944), a Jehovah's Witness aunt and guardian was convicted under child labor laws for allowing her nine-year-old niece to distribute religious literature on the street at night. The Court upheld the conviction, reasoning that the family itself is not beyond regulation in the public interest and that the state's authority to protect children's welfare can override both parental authority and religious claims. Its most quoted line captured the principle: parents may be free to become martyrs themselves, but it does not follow that they are free, in identical circumstances, to make martyrs of their children. Prince is the doctrinal root of the parens patriae power as a constitutional matter. It is why states may require vaccinations for school attendance, order blood transfusions for children of parents who refuse them on religious grounds, and remove children from homes where they are in danger. Between these poles, the Court has built a presumption. In Parham v. J.R. (1979), which concerned the voluntary commitment of children to state mental hospitals by their parents, the Court declared that the law's concept of the family rests on a presumption that parents possess what a child lacks in maturity, experience and capacity for judgment, and that natural bonds of affection lead parents to act in their children's best interests. The Court acknowledged that some parents do not, and that the presumption can be rebutted. But the starting point is trust. The state does not get to intervene simply because it believes it could make a better decision for a child than the parent would. That presumption received its most emphatic modern statement in Troxel v. Granville (2000). A Washington statute allowed any person to petition for visitation with a child at any time if visitation served the child's best interests. Paternal grandparents used it to obtain more visitation than the children's mother wished to allow. A plurality of the Court held the statute unconstitutional as applied, because the trial court had given no special weight to the mother's own judgment about what was good for her children. So long as a parent adequately cares for his or her children, the plurality reasoned, there is normally no reason for the state to inject itself into the private realm of the family to question the parent's ability to make the best decisions. Troxel was a private dispute, not a child protection case, but it articulated the principle that underlies every removal: the state may not substitute its view of the child's best interests for the parent's unless the parent is unfit or the child is at risk of harm. The Court has continued to treat parental rights as robust. In Mahmoud v. Taylor (2025), it held that parents with religious objections were entitled to preliminary relief allowing them to opt their young children out of public school instruction using storybooks with LGBTQ themes, grounding the decision in the Free Exercise Clause and in the long line running back to Wisconsin v. Yoder (1972). The decision had nothing to do with foster care. But it confirmed a direction of travel: the current Court is more, not less, inclined to treat parental authority as a constitutional check on state action, a fact that litigants in child welfare cases have begun to invoke. Who counts as a parent Constitutional protection attaches to parenthood, which raises the prior question of who is a parent. For married mothers and fathers, the answer has usually been straightforward. For unmarried fathers, it has not, and the Court's answers here matter enormously in adoption, where the consent of a father may be the difference between a valid adoption and a void one. The starting point is Stanley v. Illinois (1972). Under Illinois law, when an unmarried mother died, her children automatically became wards of the state; the father, who had lived with and supported them for years, had no right to a hearing on his fitness. The Court held that this violated due process. An unwed father who has raised his children has a liberty interest in them, and the state cannot presume him unfit simply because he never married their mother. The cases that followed drew a line between biological connection and actual relationship. In Quilloin v. Walcott (1978), the Court permitted Georgia to allow a stepfather to adopt a child over the objection of a biological father who had never sought custody or taken on the responsibilities of parenthood. In Caban v. Mohammed (1979), by contrast, it struck down a New York law that required the mother's consent to adoption but not the consent of an unmarried father who had lived with and supported his children, finding the distinction an unjustified sex-based classification. Then, in Lehr v. Robertson (1983), it upheld New York's use of a putative father registry to determine which unwed fathers were entitled to notice of adoption proceedings. Lehr had never supported the child, never lived with her, and never registered. The Court held that the mere existence of a biological link does not merit the same constitutional protection as a developed relationship. The biological connection gives a father an opportunity to develop a relationship with his child; if he grasps that opportunity and accepts responsibility, he acquires substantial protection. If he does not, the state may proceed without him. This "biology plus" framework has shaped adoption practice ever since. Most states now operate putative father registries, and many impose short deadlines within which a man must register or file a paternity action to preserve his right to notice. Critics note that the registries are little known, that a father may not even know a pregnancy exists, and that some states' deadlines expire before or shortly after birth. The Court has never revisited Lehr, and in Michael H. v. Gerald D. (1989) it went further in the other direction, upholding California's conclusive presumption that a child born to a married woman is her husband's child, even against a biological father who had a relationship with the child. The marital family, the plurality held, could be protected against outside claims. Foster parents present a different question. In Smith v. Organization of Foster Families for Equality and Reform (1977), foster parents argued that the state could not remove children who had lived with them for long periods without a full hearing. The Court declined to decide whether foster parents have a constitutionally protected liberty interest, holding that New York's existing procedures were adequate in any event. But it noted the fundamental difference between foster families and natural ones: the foster relationship is created by state law and contract, with the expectation that it will be temporary, and it exists in tension with the rights of the natural parents to whom the child may return. That reasoning still limits foster parents' constitutional claims today, even when they have cared for a child for years. How much process is due If parental rights are fundamental, the state must use fair procedures before taking them away. The question is how fair. Three Supreme Court decisions answer it, and together they define the constitutional floor for termination proceedings. The first, and the most disappointing to advocates for parents, is Lassiter v. Department of Social Services (1981). Abby Gail Lassiter was serving a prison sentence when North Carolina moved to terminate her parental rights to her son. She appeared at the hearing without a lawyer. The trial court did not appoint one, and her rights were terminated. The Supreme Court, by a five-to-four vote, held that the Constitution does not require appointment of counsel for indigent parents in every termination proceeding. Instead, trial courts must decide case by case, using the balancing test of Mathews v. Eldridge, whether the parent's interests, the state's interests and the risk of error in the particular case call for a lawyer. The Court presumed that the right to appointed counsel exists only where a litigant may lose physical liberty, and found that presumption not overcome on Lassiter's facts. Lassiter has been heavily criticized, and in practice it matters less than it once did: nearly every state now provides counsel to indigent parents in termination proceedings by statute or under its own constitution, and most provide it in the earlier dependency stages as well. But the timing and quality of that representation vary widely, and the absence of a federal constitutional right means that whether a parent has a lawyer at the first emergency hearing, when the most consequential decisions are often made, depends on the state and sometimes on the county. The second case, Santosky v. Kramer (1982), went the other way. New York allowed termination of parental rights on a finding of permanent neglect proved by a fair preponderance of the evidence, the standard used in ordinary civil litigation. The Supreme Court held that due process requires more. Because the parent's interest is fundamental, because the proceeding pits the resources of the state against a parent who is often poor and poorly educated, and because the consequences of error are irreversible, the state must prove its case by at least clear and convincing evidence. The Court noted that the proceeding is not simply a contest between parent and child, in which the child's interest might be thought to point toward the state. Until the state proves parental unfitness, the child and the parents share a vital interest in preventing erroneous termination of their natural relationship. That last observation deserves emphasis because it is often forgotten. The state frequently presents itself in these cases as the child's champion against the parent. Santosky rejects that framing at the fact-finding stage. A child has an interest in not being wrongly separated from a parent who is, in fact, fit. Only after unfitness is established do the interests of parent and child diverge. The third case is M.L.B. v. S.L.J. (1996). Mississippi terminated a mother's parental rights and then refused to let her appeal unless she paid more than two thousand dollars for the preparation of the trial record. The Court held that a state may not condition an appeal from a termination order on the parent's ability to pay record fees. Termination decrees, it reasoned, work a unique kind of deprivation, and they belong with criminal convictions in the small category of cases where access to appellate review cannot turn on wealth. Together these cases produce a curious constitutional structure. A parent facing termination is entitled to proof by clear and convincing evidence and to an appeal regardless of ability to pay, but not, as a federal constitutional matter, to a lawyer. The Indian Child Welfare Act goes further for Native American families, requiring proof beyond a reasonable doubt and guaranteeing appointed counsel, a regime examined in Chapter 5. For everyone else, the protections that matter most in practice are the ones states have chosen to provide. The limits of the state's duty to protect If the Constitution restrains the state from intervening too aggressively, does it also require the state to intervene when it should? The Supreme Court's answer, in one of the most wrenching cases it has ever decided, is no. Joshua DeShaney was a young boy living with his father in Winnebago County, Wisconsin. Over a period of more than a year, the county's social services department received repeated reports that the father was abusing him. A caseworker visited the home many times, recorded suspicious injuries and noted that Joshua had been hospitalized, but the county did not remove him. Eventually his father beat him so severely that he suffered permanent brain damage. His mother sued the county, arguing that its failure to protect Joshua violated his rights under the Due Process Clause. In DeShaney v. Winnebago County Department of Social Services (1989), the Court rejected the claim. The Due Process Clause, it held, is a limitation on the state's power to act, not a guarantee of minimal levels of safety and security. The state had not created the danger Joshua faced and had not taken him into its custody. Nothing in the Constitution required it to protect him from a private actor, even one it knew to be dangerous. Justice Blackmun's dissent, with its famous exclamation "Poor Joshua!", accused the majority of retreating into a sterile formalism. But the holding has stood, and it was extended in Town of Castle Rock v. Gonzales (2005), where the Court held that a mother had no constitutionally protected entitlement to police enforcement of a restraining order against her estranged husband, who then killed their three daughters. DeShaney matters to child protection law in two ways. First, it means that children who are harmed because a child protective agency failed to act generally have no federal constitutional claim. Whatever remedy they have must come from state tort law, which is frequently limited by governmental immunity. Second, and more important for the system as a whole, the Court identified one situation in which a duty does exist: when the state takes a person into its custody and holds him there against his will, it assumes some responsibility for his safety and general well-being. That caveat has become the foundation for the entire body of litigation over conditions in foster care, examined in Chapter 6. Once the state removes a child from a parent, it becomes the parent, and the Constitution follows the child into the foster home. A constitutional framework without a constitutional system The result is a body of doctrine that is strong in principle and thin in operation. Parents have a fundamental right to raise their children. The state may override it to protect children from harm. Termination requires clear and convincing proof and a meaningful chance to appeal. Unwed fathers are protected if they have built a relationship. The state has no duty to protect children it leaves at home, but a real duty to children it takes. What the Constitution does not do is prescribe how any of this works day to day. It does not say when a caseworker may remove a child without a court order, how quickly a hearing must follow, what efforts the state must make to reunify a family, how long a child may remain in foster care before the state moves to terminate, or what a court must consider in deciding whether adoption serves the child. Those questions are answered by state law and, increasingly since 1980, by conditions Congress has attached to federal child welfare funding. The constitutional cases describe a family that the state may enter only reluctantly. The statutes describe a system that enters millions of homes every year to investigate, and removes hundreds of thousands of children from them. To understand how those two pictures coexist, it is necessary to follow a case from its beginning. Chapter 2: From Report to Removal Most child protection cases begin with a phone call. A teacher notices bruises, a doctor sees an unexplained fracture, a neighbor hears screaming, a hospital records a positive drug test at birth, an estranged relative makes an accusation in the middle of a custody fight. The call goes to a state or county hotline, a screener decides whether it meets the legal threshold for investigation, and if it does, a caseworker is assigned to visit the family, often within a day or less. From that moment the family is inside a legal process that can end anywhere from a closed file to the permanent loss of a child. The front end of the system is where the state makes contact with the largest number of families and exercises the least judicial oversight. It is also where the constitutional principles of Chapter 1 are most often tested, because caseworkers must decide quickly, on incomplete information, whether a child is safe enough to remain at home. The reporting machine The modern child protection system is younger than most people assume. For much of American history, the protection of abused children was left to private charitable societies, the first of which, the New York Society for the Prevention of Cruelty to Children, was founded in 1875 after the widely publicized case of Mary Ellen Wilson. Public agencies took over gradually in the twentieth century. The turning point came in 1962, when the pediatrician C. Henry Kempe and his colleagues published "The Battered-Child Syndrome" in the Journal of the American Medical Association, describing a pattern of injuries in young children that physicians had failed to recognize as inflicted. Within about five years, every state had enacted a law requiring certain professionals to report suspected abuse. Congress built on those laws with the Child Abuse Prevention and Treatment Act of 1974, known as CAPTA. CAPTA offered states grants on condition that they adopt mandatory reporting, provide immunity for good-faith reporters, investigate reports promptly and protect the confidentiality of records. It has been reauthorized many times, and its conditions now reach into areas such as the appointment of guardians ad litem for children in judicial proceedings and the handling of infants born affected by substance use. Mandated reporting laws vary by state in who must report, but the list typically includes teachers, physicians, nurses, social workers, mental health professionals, child care workers and police. Some states require every adult to report. The standard for reporting is low by design: a reasonable suspicion or reasonable cause to believe that a child has been abused or neglected. Failure to report is usually a misdemeanor. Reporters acting in good faith are immune from liability, even if the report proves unfounded. The result is a system of enormous volume. In recent years, state agencies have received more than four million referrals annually concerning children, of which roughly half are screened in for some form of investigation or assessment. Only a fraction of those end in a finding that maltreatment occurred, usually in the range of half a million to six hundred thousand child victims per year, according to the federal government's annual Child Maltreatment reports. The large majority of those substantiated cases involve neglect rather than physical or sexual abuse. The scale of contact is best appreciated at the level of an individual childhood. A study published in 2017 in the American Journal of Public Health by Hyunil Kim, Christopher Wildeman, Melissa Jonson-Reid and Brett Drake estimated that about 37 percent of American children will be the subject of a child protective services investigation before their eighteenth birthday, and that for Black children the figure is roughly 53 percent. Whatever one thinks of those numbers, they describe an institution that touches a large share of American families, and a far larger share of poor and Black families, than its reputation as an emergency service suggests. Critics from across the spectrum have questioned whether the reporting machine is well designed. The low threshold and the penalties for failure to report encourage defensive reporting, particularly by teachers and medical staff, which floods hotlines with calls that do not involve maltreatment and diverts investigative resources from the cases that do. Because poor families have more contact with the professionals who are mandated reporters, from public hospitals to welfare offices, they are more visible to the system. Several jurisdictions have experimented with narrowing mandated reporting duties or creating alternative pathways, such as referring families with material needs to community support rather than to the hotline. These reforms are still being evaluated. Neglect, poverty and the definition of harm The legal definitions of abuse and neglect are the hinge on which everything turns, because they determine what conduct authorizes state intervention. Abuse is relatively easy to define in principle, if not always to prove: nonaccidental physical injury, sexual abuse, and in many states serious emotional harm. Neglect is harder. State statutes typically define it as a failure to provide adequate food, clothing, shelter, medical care, education or supervision, such that the child is harmed or placed at substantial risk of harm. The difficulty is obvious. A family without stable housing may fail to provide adequate shelter. A parent working a night shift without affordable child care may leave children inadequately supervised. A household whose utilities have been disconnected may be unsafe in winter. Are these parents neglectful, or poor? Many state statutes answer the question in principle by requiring that the failure occur when the parent was financially able to provide care, or was offered the means to do so and refused. New York's Family Court Act, for instance, frames neglect as a failure to exercise a minimum degree of care "though financially able to do so or offered financial or other reasonable means to do so." But in practice, the line between poverty and neglect is drawn by caseworkers and family court judges who see the consequences of deprivation more readily than its causes. A wave of state legislation in the early 2020s tried to sharpen that line. In 2021, Texas enacted House Bill 567, which narrowed the statutory definition of neglect to conduct that reflects a blatant disregard for the consequences of a parent's act or failure to act that results in harm or substantial risk of immediate harm, and specified that certain circumstances, such as allowing a child to engage in independent activities appropriate to the child's age, are not neglect by themselves. The same year, Washington's Keeping Families Together Act raised the standard for emergency removal, requiring a showing of imminent physical harm and directing courts to consider whether safety measures short of removal would suffice. Other states have added language excluding poverty, homelessness or lack of child care, standing alone, from the definition of neglect. These reforms reflect a growing bipartisan view that the system had been treating material hardship as a parenting failure. Two categories deserve particular mention. The first is domestic violence. For years, some agencies removed children from mothers who were victims of abuse on the theory that the mothers had "engaged in" domestic violence or had failed to protect their children from witnessing it. In the class action that became Nicholson v. Williams, a federal district judge in New York found in 2002 that New York City's child welfare agency had been routinely removing children from battered mothers without adequate justification. When the case reached the Second Circuit, that court certified questions of state law to New York's highest court. In Nicholson v. Scoppetta (2004), the New York Court of Appeals held that exposure to domestic violence is not, by itself, neglect, and that a court must find actual or imminent harm to the child and a failure by the parent to exercise a minimum degree of care before removal. Emergency removal, it added, requires a court to weigh the harm of removal itself against the risk of leaving the child at home. The second category is prenatal substance exposure. When a newborn tests positive for drugs, many hospitals notify child protective services, and the infant may be removed at birth. Federal law, through amendments to CAPTA enacted in 2010 and strengthened by the Comprehensive Addiction and Recovery Act of 2016, requires states to have procedures for health care providers to notify child protective services about infants affected by substance use or withdrawal and to develop a "plan of safe care" for them. Congress also made clear that such a notification is not, by itself, a report of child abuse. States vary sharply in how they treat these cases, and the constitutional backdrop is relevant: in Ferguson v. City of Charleston (2001), the Supreme Court held that a public hospital's policy of testing pregnant women for cocaine and turning positive results over to police for prosecution was an unreasonable search under the Fourth Amendment. Ferguson concerned law enforcement, not child protection, but it signals that hospital testing programs designed to generate evidence against parents raise constitutional questions of their own. The investigation and the Fourth Amendment Once a report is screened in, a caseworker investigates. That usually means visiting the home, interviewing the child and the parents, speaking with teachers, doctors and relatives, and sometimes arranging a medical examination. Each of these steps involves the state entering the most private spaces of family life, and the question of what the Constitution permits has produced a body of lower-court law that is more developed, and more protective of families, than many practitioners realize. The Fourth Amendment prohibits unreasonable searches and seizures, and its protections are not limited to criminal investigations. Several federal circuits have held that caseworkers generally need a warrant, a court order, consent or exigent circumstances before entering a home against the family's wishes. In Calabretta v. Floyd (1999), the Ninth Circuit held that a social worker and a police officer violated the Fourth Amendment when they entered a home over a mother's objection, based on an anonymous report of a child crying, and strip-searched a three-year-old. There was no emergency, the court reasoned, and the social worker's purpose did not exempt her from ordinary constitutional rules. Other circuits have reached similar conclusions, although they differ on the details and on how far a "special needs" exception reaches. Interviews at school have been more contested. In Doe v. Heck (2003), the Seventh Circuit held that Wisconsin caseworkers violated the Fourth Amendment by interviewing a child at a private school without a warrant, parental consent or exigent circumstances. The Ninth Circuit reached a similar holding about an in-school interview in Greene v. Camreta (2009), but when the Supreme Court took up the case, it declined to reach the merits. In Camreta v. Greene (2011), the Court held that the dispute had become moot because the child had moved and was nearly an adult, and it vacated the relevant part of the Ninth Circuit's ruling. The constitutional status of warrantless school interviews thus remains unsettled nationally, and practices vary. The removal of a child is itself a seizure, and it also implicates the parents' due process rights. The governing principle in most circuits is that the state may remove a child without prior judicial authorization only when it has reasonable cause to believe the child is in imminent danger of serious bodily injury and that removal is necessary to avert that danger. In Tenenbaum v. Williams (1999), the Second Circuit held that New York City caseworkers who took a five-year-old from kindergarten for a medical examination to check for sexual abuse, without a court order and without an emergency, violated the Fourth Amendment and the parents' due process rights, because there had been time to seek a court order. In Wallis v. Spencer (2000), the Ninth Circuit reached similar conclusions about the removal of children and invasive medical examinations conducted without notice to the parents. The rule these cases establish is intuitive: where there is time to go to a judge, caseworkers must go to a judge. These are rights with limited remedies. Caseworkers sued for damages under 42 U.S.C. section 1983 routinely invoke qualified immunity, which shields officials unless they violated clearly established law. Because the case law in this area is fact-specific and uneven across circuits, courts frequently find that the right at issue, even if violated, was not clearly established at the time. Many families never sue at all. The practical force of the Fourth Amendment in child protection therefore depends heavily on agency training and on local lawyers willing to raise it. Safety plans and the hidden system Not every separation of a child from a parent involves a formal removal. Increasingly, agencies use "safety plans" in which a parent agrees, often at the kitchen table during an investigation, that the child will stay with a relative or friend, or that a particular adult will leave the home, while the investigation continues. These arrangements are described as voluntary. Parents are often told, however, that the alternative is formal removal and a court case. Josh Gupta-Kagan, in an influential 2020 article in the Stanford Law Review, called this "America's hidden foster care system": a large and largely uncounted set of family separations that happen without a petition, without a judge, without appointed counsel and without the reasonable-efforts requirements that attach to formal foster care. The children are not counted in federal foster care statistics, and relatives who take them in often receive none of the financial support that licensed foster parents would. The courts have been reluctant to treat these arrangements as coerced. In Dupuy v. Samuels (2005), the Seventh Circuit rejected a challenge to Illinois's use of safety plans, reasoning that offering a parent a choice between a safety plan and formal proceedings is not coercion so long as the agency has a lawful basis for the threatened alternative. The consequence is that the most informal interventions, which may last for months, receive the least legal scrutiny. Registries and the long tail of an investigation Even when no child is removed, an investigation can leave a lasting mark. States maintain central registries of individuals found responsible for maltreatment, and those registries are checked by employers in child care, education, health care and other fields. A substantiated finding, often made by a caseworker on a low standard of proof, can end a career. In Valmonte v. Bane (1994), the Second Circuit held that New York's registry scheme violated due process because a person could be listed, and effectively barred from child-related employment, on a finding supported only by "some credible evidence," a standard the court found too likely to produce error. New York responded by raising the standard for employer disclosure. Other states have faced similar challenges, and in Los Angeles County v. Humphries (2010), the Supreme Court addressed a related suit over California's registry, although its holding was limited to the question of when local governments can be held liable under section 1983. The first hearing When a child is removed, the case moves quickly to court. Every state requires a prompt hearing, typically within one to three business days, at which a judge decides whether the child must remain in state custody pending further proceedings. Depending on the state, this is called a shelter care hearing, a detention hearing, a preliminary protective hearing or simply a seventy-two-hour hearing. It is often short, sometimes only a few minutes, and it is often the first time the parent has seen a lawyer, if a lawyer has been appointed at all. The standard at this stage is low, and the consequences are large. Once a child is in foster care, the dynamics of the case change. The parent must now satisfy a case plan to get the child back; visits are scheduled and supervised; the child begins attending a new school and forming attachments to new caregivers; and the statutory clocks described in the next chapter begin to run. Researchers and practitioners have long observed that decisions made in the first days of a case shape its outcome disproportionately. A child who is not removed may never enter foster care. A child who is removed may not come home for a year or more, or at all. The overall path of a case, from the hotline call through the permanency decision, is summarized in Table 1. The timing column reflects common federal and state requirements, but individual states vary. Table 1. The principal stages of a dependency case. Stage Typical timing Legal standard Key decision Report and screening Hours Reasonable suspicion Whether to investigate Investigation Days to weeks Agency policy; Fourth Amendment limits Substantiate; safety plan; remove Emergency removal Immediate Imminent danger of serious harm Take child into custody Shelter or detention hearing 1 to 3 court days Probable cause or reasonable grounds Keep child in custody pending trial Adjudication Weeks to months Preponderance in most states Whether abuse or neglect occurred Disposition and case plan Soon after adjudication Best interests of the child Placement and services Permanency hearing Within 12 months of entry Best interests; reasonable efforts Reunify, adopt, guardianship, other Termination of parental rights Often at 15 of 22 months Clear and convincing evidence Sever the legal relationship The table makes one thing plain. The legal standard rises only at the very end. Everything that happens before termination, including the decisions that most often determine whether termination will ever be sought, is decided on standards far lower than the one the Supreme Court demanded in Santosky. That is the procedural architecture within which the dependency court operates. Chapter 3: The Dependency Court and the Clock Once a child is in state custody, the case belongs to the dependency court. It is an unusual forum. It is civil, but the state is the petitioner and the stakes resemble those of a criminal case. It is adversarial, but everyone in the courtroom, including the agency that removed the child, claims to be acting in the child's interest. Hearings are usually closed to the public, records are sealed, and appellate decisions are often anonymized, so the body of law that governs these courts develops largely out of view. Judges carry heavy dockets, sometimes hundreds of open cases each, and hearings that will decide whether a family survives may be allotted a few minutes. What structures this forum more than anything else is not the Constitution but federal money. Since 1980, Congress has used its spending power to impose detailed conditions on how states run their foster care systems, and state legislatures have written those conditions into their own codes. To understand the dependency court is to understand a series of federal statutes, each passed in response to the perceived failures of the last. Reasonable efforts: the 1980 bargain In the 1970s, researchers and advocates documented a phenomenon they called "foster care drift." Children removed from their homes remained in care for years, moved from placement to placement, and neither returned home nor were adopted. Nobody was responsible for planning their futures. The federal government reimbursed states for foster care costs but not for services that might have prevented placement or brought children home, so the financial incentives pointed toward keeping children in care. The Adoption Assistance and Child Welfare Act of 1980 was Congress's answer. It created Title IV-E of the Social Security Act, an open-ended entitlement that reimburses states for part of the cost of foster care maintenance for eligible children, and it attached conditions. States had to make "reasonable efforts" to prevent the need for removing a child from home and to make it possible for a removed child to return. They had to develop a written case plan for each child, review each child's case at least every six months, and hold a dispositional hearing to determine the child's future. The Act also created adoption assistance payments to encourage the adoption of children with special needs, who had been languishing in care. The reasonable-efforts requirement was meant to be the heart of the reform. Courts must find, in their initial orders, that remaining at home would be contrary to the child's welfare and, within a set period after removal, that the agency made reasonable efforts to prevent removal. Later, courts must find at least annually that the agency made reasonable efforts to achieve the child's permanency plan. These findings are not merely formal: if a court fails to make them, the state loses federal reimbursement for that child. In practice, the requirement has had less bite than its authors hoped. Congress never defined "reasonable efforts," and courts have interpreted the term loosely. Judges rarely find that an agency failed to make reasonable efforts, in part because doing so costs the state money and in part because the finding is often made on preprinted orders without meaningful inquiry. In Suter v. Artist M. (1992), the Supreme Court held that the reasonable-efforts provision could not be enforced by private parties through a federal civil rights suit, because the statute's requirement that a state have a plan providing for reasonable efforts did not unambiguously confer an individual right. Congress responded in 1994 by providing that a provision of the Social Security Act is not unenforceable merely because it appears in a section requiring a state plan, but it expressly left Suter's holding on reasonable efforts itself intact. The practical result is that reasonable efforts are enforced, if at all, by state judges in individual cases and by federal auditors reviewing state compliance. Some state appellate courts have taken the requirement seriously. Courts have reversed terminations where the agency offered a parent with an intellectual disability only generic services that the parent could not use, or where an incarcerated parent was given no meaningful opportunity to maintain contact with a child. But these decisions depend on parents having lawyers who raise the issue at trial and pursue it on appeal, which returns us to the problem of representation. The clock: the Adoption and Safe Families Act By the mid-1990s, the political mood had shifted. High-profile deaths of children returned to abusive parents, and a sense that reasonable efforts had become an excuse for endless second chances, produced the Adoption and Safe Families Act of 1997, known as ASFA. Its stated premise was that a child's health and safety must be the paramount concern in every decision. ASFA made three changes that transformed dependency practice. First, it identified circumstances in which reasonable efforts to reunify are not required at all. These "aggravated circumstances" include abandonment, torture, chronic abuse and sexual abuse, as well as cases where the parent has killed or seriously assaulted another child or has had parental rights to a sibling involuntarily terminated. In such cases, the state may move directly toward termination. Second, it shortened the timeline for permanency hearings. A court must hold a permanency hearing within twelve months after a child enters foster care, and at least every twelve months thereafter, to decide whether the child will be returned home, adopted, placed with a legal guardian, placed permanently with a relative, or, for older youth only, placed in another planned permanent living arrangement. Third, and most consequentially, it imposed a deadline. When a child has been in foster care for fifteen of the most recent twenty-two months, the state must file a petition to terminate parental rights, and concurrently seek an adoptive family, unless one of three exceptions applies: the child is being cared for by a relative, the agency has documented a compelling reason why termination would not be in the child's best interests, or the state has not provided the services the case plan deemed necessary for the child's safe return. ASFA also created adoption incentive payments, which reward states financially for increasing the number of adoptions from foster care above a baseline. And it endorsed "concurrent planning," the practice of working toward reunification and an alternative permanent plan at the same time, so that a child's placement with a potential adoptive family can begin while the parent is still trying to regain custody. The effects were visible quickly. Adoptions from foster care rose substantially in the years after 1997, reaching more than fifty thousand a year by the early 2000s, and they have remained around that level since. Termination petitions became routine in cases where reunification had not occurred within about a year and a half. For children who could not safely go home, the statute undoubtedly reduced the years spent in limbo. But the fifteen-of-twenty-two rule has drawn sustained criticism, and it has intensified since around 2020. The timeline is often unrealistic for the problems that bring families into the system. Recovery from opioid or methamphetamine addiction commonly involves relapse and takes longer than fifteen months. A parent waiting for a residential treatment bed, a housing voucher or a mental health appointment may exhaust the clock before services are even available. Incarcerated parents with sentences longer than fifteen months are almost automatically subject to termination petitions, whatever their relationship with the child. And because the incentive structure rewards adoption, critics argue that it has encouraged states to terminate rights in cases where adoption is not likely to follow, creating what are sometimes called "legal orphans": children with no legal parents and no adoptive family, who age out of care entirely alone. Defenders of ASFA respond that the exceptions give agencies and courts ample flexibility, that the statute requires only the filing of a petition rather than termination itself, and that children's sense of time differs from adults'. A year and a half is a large fraction of a toddler's life, and attachment research suggests that prolonged uncertainty harms young children. The debate is not really about whether time matters, but about whether a fixed federal number is the right way to account for it. Several bills to amend or repeal the timelines have been introduced in Congress, and some states have used the "compelling reasons" exception more generously, for example by treating a parent's incarceration or pending treatment as a presumptive reason not to file. As of late 2026, the federal timeline remained law. From termination to prevention: later statutes Congress has continued to adjust the system in ways that pull in different directions. The Fostering Connections to Success and Increasing Adoptions Act of 2008 gave states the option to use federal funds to support permanent guardianship by relatives, required agencies to notify adult relatives within thirty days of a child's removal, required reasonable efforts to place siblings together, promoted educational stability for children in care, and allowed states to extend foster care to age twenty-one. The Preventing Sex Trafficking and Strengthening Families Act of 2014 restricted the use of "another planned permanent living arrangement," a catch-all plan that had often meant long-term foster care with no permanent family, to youth aged sixteen and older, and required states to adopt a "reasonable and prudent parent" standard allowing foster parents to let children participate in ordinary activities such as sleepovers and sports. The most significant recent change is the Family First Prevention Services Act, enacted as part of the Bipartisan Budget Act of 2018. For the first time, it allowed states to use Title IV-E funds, previously available almost exclusively for foster care and adoption, to pay for prevention services: mental health treatment, substance use treatment and in-home parenting programs for children who are candidates for foster care and for their parents or kin caregivers, for up to twelve months at a time. The services must be trauma-informed and evidence-based, rated by a federal clearinghouse as promising, supported or well-supported, and states must spend a set share of their prevention funds on well-supported programs. At the same time, Family First restricted federal reimbursement for group care, limiting it after two weeks to specified settings such as qualified residential treatment programs that meet accreditation, staffing and assessment requirements. Family First represents a partial reversal of the logic of the previous forty years. Federal money had followed children into foster care; now some of it could be spent keeping them out. Implementation has been uneven. The evidence standards are demanding, the list of approved programs is short, and the federal share applies only to services for which the clearinghouse has rated the evidence. But the statute has changed the terms of debate in state legislatures and agencies. Other federal adjustments have followed. In 2023, the Administration for Children and Families issued a final rule allowing states to adopt separate, less burdensome licensing standards for relative and kin foster family homes, so that grandparents and aunts need not satisfy requirements designed for strangers, such as square-footage rules, in order to receive foster care payments. In 2024, it finalized a rule clarifying that Title IV-E administrative funds may be used to pay for independent legal representation of children and parents in dependency proceedings, building on a 2019 policy change. And in January 2025, Congress enacted the Supporting America's Children and Families Act, which reauthorized the Title IV-B child welfare programs that fund family support and caseworker activities. The sequence of statutes is summarized in Table 2. Table 2. Major federal child welfare statutes. Year Statute Principal features Direction of policy 1974 Child Abuse Prevention and Treatment Act Mandated reporting, investigation standards, guardians ad litem Detection 1978 Indian Child Welfare Act Heightened standards and placement preferences for Native children Tribal and family integrity 1980 Adoption Assistance and Child Welfare Act Title IV-E funding, reasonable efforts, case plans, adoption subsidies Reunification 1997 Adoption and Safe Families Act Safety paramount, 15-of-22 month rule, aggravated circumstances, incentives Speed and adoption 2008 Fostering Connections Act Kinship guardianship, relative notice, sibling placement, care to 21 Kinship and transition 2018 Family First Prevention Services Act IV-E funds for prevention services, limits on group care Prevention Who speaks for whom in the courtroom Every dependency case has at least three parties with distinct interests: the agency, the parents and the child. Their representation shapes outcomes as much as any statute. The agency is represented by government lawyers, sometimes from the county attorney's office, sometimes from the state attorney general, sometimes from lawyers employed by the agency itself. In some jurisdictions the caseworker appears without counsel at routine hearings. Parents, as noted in Chapter 1, have no federal constitutional right to counsel, but almost every state now appoints counsel for indigent parents at least in termination proceedings, and most do so from the beginning of the dependency case. The quality of that representation varies enormously. In some counties, parents are represented by contract lawyers paid a flat fee per case, carrying caseloads in the hundreds, who meet their clients for the first time in the courthouse hallway. In others, parents are represented by specialized offices using an interdisciplinary model that pairs the lawyer with a social worker and a parent advocate who has personal experience of the system. Evidence on the difference is striking. A study of New York City cases by Lucas Gerber and colleagues, published in 2019 in Children and Youth Services Review, found that children whose parents received interdisciplinary representation spent substantially less time in foster care, roughly four months on average, with no increase in later maltreatment. Findings of this kind helped persuade federal officials to open Title IV-E funds to parental representation. Children's representation is even more varied. CAPTA requires states, as a condition of funding, to appoint a guardian ad litem for every child in a judicial proceeding involving abuse or neglect, but the guardian may be a lawyer or a trained lay volunteer, such as those in Court Appointed Special Advocate programs. A guardian ad litem traditionally advocates for what he or she believes to be in the child's best interest. A lawyer for the child, in what is sometimes called a client-directed model, advocates for what the child wants, at least when the child is old enough to express a considered view. States have adopted every combination of these roles. Some provide both a lawyer and a guardian; some provide only a volunteer; a few provide nothing meaningful for young children. In Kenny A. v. Perdue (2005), a federal district court held that children in Georgia dependency proceedings had a right to counsel under the due process clause of the Georgia Constitution, and the resulting consent decree required representation. But there is no federal constitutional rule. The comparative view It is instructive to compare the American approach with England's, which faces similar problems with different tools. Under the Children Act 1989, a court may make a care order only if it finds that a child is suffering, or is likely to suffer, significant harm attributable to the care given by the parents. Since the Children and Families Act 2014, care proceedings must ordinarily conclude within twenty-six weeks, a far tighter clock than the American one, although extensions are common. Parents in care proceedings receive non-means-tested legal aid, so representation is effectively universal. And adoption without parental consent is permitted when the child's welfare requires it, under the Adoption and Children Act 2002, which makes England one of the few European countries that routinely places children from care for adoption against their parents' wishes. The UK Supreme Court and Court of Appeal responded to concerns about that practice in 2013, in Re B and Re B-S, holding that adoption against a parent's will is a last resort, justified only when "nothing else will do," and requiring courts to weigh every realistic option explicitly. The comparison illuminates the American system's distinctive feature. England imposes a strict clock on the court proceedings but guarantees representation and demands a rigorous analysis of alternatives before adoption. The United States imposes a clock on the family's time in care, leaves representation to the states, and relies on a reasonable-efforts requirement that is rarely enforced. The American design places the burden of delay on the family rather than on the institutions responsible for it. Whether that is the right allocation is the question at the heart of the next chapter, which examines the moment when the clock runs out. Chapter 4: Terminating Parental Rights The termination of parental rights is the point of no return. Everything before it, including removal, foster placement, case plans and permanency hearings, is provisional. A child in foster care still has parents in law, and those parents may regain custody if circumstances change. Termination ends that possibility. It frees the child for adoption, but it does so by declaring that the child's legal relationship with the parent no longer exists. The Supreme Court has compared its severity to that of a criminal sentence, and for many parents it is experienced as worse. Because the stakes are so high, one might expect termination to be rare and hard-won. It is neither. American courts terminate the parental rights of tens of thousands of parents every year; federal foster care data have for years recorded on the order of sixty thousand children annually whose parents' rights were terminated. Most of those terminations are uncontested in any meaningful sense. The parent may have disappeared, may have stopped attending hearings, may have agreed to relinquish under pressure, or may have lost at a trial lasting less than a day. The law of termination is therefore as much about what happens in thin, routine cases as about the landmark decisions. Voluntary and involuntary termination Termination takes two forms. In a voluntary relinquishment, the parent consents to the end of the relationship, usually to allow a specific adoption. Relinquishments are common in private infant adoptions and also occur in foster care cases, sometimes as part of a negotiated arrangement in which the parent agrees to relinquish in exchange for the possibility of continued contact or a say in the choice of adoptive family. State laws regulate the circumstances of consent, often forbidding it until some period after birth and allowing revocation for a short time. A relinquishment obtained by fraud or duress can be set aside, but courts are reluctant to disturb consents once an adoption has proceeded. Involuntary termination is a judicial determination, over the parent's objection or in the parent's absence, that the relationship should end. It is the subject of most of this chapter. Nearly every state structures it as a two-step inquiry. First, the state must prove, by at least clear and convincing evidence, that one or more statutory grounds for termination exist. Second, the court must find that termination is in the child's best interests. Some states require the best-interests finding also to be made by clear and convincing evidence; others apply a preponderance standard at that stage, reasoning that Santosky v. Kramer addressed only the question of parental unfitness. The statutory grounds State statutes list grounds for termination that vary in wording but fall into recognizable families. Abandonment is the oldest. A parent who has had no contact with the child and provided no support for a specified period, commonly six months or a year, may be found to have abandoned the child. Courts differ on how much effort counts as contact: a single card, a few phone calls, or an attempt blocked by the agency may or may not defeat an abandonment finding. Severe or chronic abuse and the aggravated circumstances recognized by ASFA form a second group. Where a parent has tortured or sexually abused a child, killed or seriously injured a sibling, or subjected the child to chronic abuse, termination may follow without any requirement that the state first try to reunify the family. Failure to remedy the conditions that led to removal is the most commonly invoked ground in neglect cases. The state must show that the child has been in care for a specified period, that the conditions that led to removal persist or that new conditions of a similar nature have arisen, and that there is little likelihood they will be remedied in the near future. This is where the case plan becomes evidence. If the parent failed to complete a substance abuse program, did not obtain stable housing, missed visits or failed drug tests, the agency will rely on those failures to prove that reunification is not reasonably likely. And if the agency did not provide the services the plan called for, the parent will argue that the failure was the agency's rather than his or her own. Parental incapacity is a fourth ground: a mental illness, intellectual disability or chronic substance use disorder so severe that the parent is unable to care for the child and is unlikely to become able within a reasonable time. These provisions are among the most controversial, because they risk judging parents by their diagnoses rather than their conduct. Many states add grounds based on time in care, mirroring ASFA's fifteen-of-twenty-two-month rule, and grounds based on a prior involuntary termination of rights to another child, which can create a presumption of unfitness that the parent must rebut. Some states include felony convictions or incarceration for a substantial part of the child's minority. Because grounds are defined by statute, a parent can lose rights through a combination of findings that individually seem modest. A mother who relapsed twice, missed a series of visits while she was in jail on a probation violation, and was evicted from housing she had obtained may satisfy the failure-to-remedy ground without anyone concluding that she harmed or would harm her child. The clear-and-convincing standard applies to the existence of the ground, not to the ultimate question of whether the child would be endangered at home. Best interests and the problem of the foster bond If the state establishes a statutory ground, the court moves to the best-interests determination. Here the court considers the child's needs, the child's relationship with the parent, the stability of the current placement, the likelihood of adoption, the child's own wishes if the child is old enough, and any other factor the court finds relevant. The best-interests standard is famously indeterminate; the legal scholar Robert Mnookin observed decades ago that it asks judges to make predictions that no one has the knowledge to make with confidence. The standard also contains a structural trap. One of the most powerful considerations in the best-interests analysis is the child's attachment to his or her current caregivers. The influential 1973 book Beyond the Best Interests of the Child, by Joseph Goldstein, Anna Freud and Albert Solnit, argued that children need continuity of relationships with a "psychological parent" and that the law should protect the relationship the child actually has rather than the biological one. The idea was absorbed into the law and into practice, including the widespread use of "bonding evaluations" in which psychologists observe the child with the birth parent and with the foster parent. The difficulty is that the length of time a child spends with a foster parent is determined largely by the state's own decisions: the removal, the placement, the pace of services, the frequency of visits, and the speed of the court. The longer a case takes, the stronger the child's attachment to the foster family becomes and the weaker the bond with the birth parent, especially if visits are limited to an hour a week in an agency office. A best-interests analysis that turns on attachment thus tends to ratify whatever the system has already done. The state removes the child, delays reunification, and then points to the child's bond with the foster parents as a reason why reunification would now be harmful. Some states have responded directly. New Jersey's termination statute had long directed courts to consider whether separating the child from foster parents would cause serious and enduring emotional or psychological harm. In 2021, the legislature removed that language, so that the harm inquiry focuses on the parent's conduct rather than on the attachments that formed while the child was in state care. Other states have not followed, and the debate over how much weight to give the foster bond remains one of the sharpest in the field. It is not easily resolved, because the child's attachment is real regardless of how it came about, and a court asked to decide what is best for the child going forward cannot simply ignore it. The recurring hard cases Three categories of parents appear so often in termination litigation, and raise such consistent problems, that they deserve separate treatment. Incarcerated parents. A parent in prison cannot attend a parenting class of the agency's choosing, cannot maintain housing, and may be unable to visit a child who is placed hours away. Under ASFA, a sentence longer than fifteen months almost guarantees that the fifteen-of-twenty-two-month period will run. Yet many incarcerated parents maintain real relationships with their children and will be released while the children are still young. Courts have begun to insist that the state make genuine efforts. In In re Mason (2010), the Michigan Supreme Court reversed a termination where the incarcerated father had not been given the opportunity to participate in the proceedings or to receive services, emphasizing that incarceration alone is not a ground for termination and that the state must show more than the fact of imprisonment. New York's legislature enacted the ASFA Expanded Discretion Act in 2010, allowing agencies not to file for termination when a parent's incarceration or residential drug treatment is a significant factor in the child's time in care and the parent maintains a meaningful role in the child's life. Other states have adopted similar provisions. Parents with disabilities. Parents with intellectual, psychiatric or physical disabilities are overrepresented among those who lose their children, and research has long suggested that disability itself, rather than parenting conduct, drives some of these outcomes. Title II of the Americans with Disabilities Act and Section 504 of the Rehabilitation Act prohibit public entities and federally funded programs from discriminating on the basis of disability, and both the Department of Justice and the Department of Health and Human Services have taken the position that these laws apply to child welfare agencies and courts. In 2015, the two departments jointly found that the Massachusetts Department of Children and Families had violated the ADA in removing a newborn from a mother with a developmental disability without assessing whether supports would have enabled her to parent safely. In 2024, HHS finalized an updated Section 504 rule that includes provisions specific to child welfare, prohibiting agencies from making decisions based on stereotypes about parents with disabilities and requiring individualized assessment and reasonable modifications. The rule was challenged in litigation brought by a group of states, but its child welfare provisions largely codified obligations the federal agencies had long asserted. Courts have been slower to treat the ADA as a defense in termination proceedings themselves. Many state appellate courts hold that the ADA does not provide a defense to termination as such, although they allow its requirements to inform whether the agency made reasonable efforts. The practical question in most cases is whether the agency tailored its services to the parent's disability, for example by providing hands-on parenting instruction rather than a lecture course, or by allowing more time. Where it did not, the reasonable-efforts requirement offers the stronger argument. Parents struggling with poverty. Almost no statute authorizes termination because a family is poor, and several expressly forbid it. But poverty operates indirectly through nearly every ground. A parent who cannot find housing may fail to remedy the conditions of removal. A parent who works irregular hours may miss visits. A parent who cannot afford a car may not reach a treatment program. Courts are asked to decide whether these failures reflect unwillingness or inability, and whether the agency's services addressed the actual barriers. Some appellate courts have reversed terminations that rested essentially on inadequate housing. Many more have affirmed them on records where housing was one of several problems. After termination Once a termination order is entered, the parent's legal relationship with the child ends. The parent may appeal, and the appeal must be available regardless of ability to pay under M.L.B. v. S.L.J. Many states expedite termination appeals, and several recognize a statutory or constitutional right to effective assistance of counsel, allowing a parent to challenge a termination on the ground that appointed counsel performed deficiently. Reversal rates vary by state and are difficult to measure, but reversals do occur, most often for failures of notice, inadequate reasonable-efforts findings or procedural defects such as failure to follow the Indian Child Welfare Act. If the appeal fails, or if none is taken, the child is legally free for adoption. But not all such children are adopted. Older children, sibling groups and children with significant disabilities may wait years, and some never find an adoptive family. Tens of thousands of children in foster care at any time are waiting for adoption with parental rights already terminated, and every year many thousands of young people age out of care without any legal family at all. This has prompted a quiet innovation: the reinstatement of parental rights. Beginning with California in the mid-2000s, a growing number of states allow a court to restore the rights of a parent whose rights were terminated, typically when the child has not been adopted, some period has passed, and the parent's circumstances have changed so that reunification would now be in the child's best interests. Often the petition is brought by the child, now an adolescent, who wants to go home. Reinstatement is used rarely, but its existence is an admission by the law itself that some terminations produced not permanence but an empty legal space. States have also expanded the possibility of post-termination contact. Many allow birth parents and adoptive parents to enter into post-adoption contact agreements, and a growing number allow courts to order continued contact with siblings or, in some circumstances, birth parents where it serves the child. These arrangements reflect a recognition, grounded in research on adoption and identity, that severing every tie is not always best for the child, even when legal permanency is. The European mirror A comparative note sharpens the picture. In Strand Lobben and Others v. Norway (2019), the Grand Chamber of the European Court of Human Rights held that Norway had violated the right to respect for family life under Article 8 of the European Convention on Human Rights when its courts authorized the adoption of a boy by his foster parents against his mother's wishes. The Court found that the Norwegian authorities had not genuinely weighed the competing interests, had based their assessment on limited contact that the authorities themselves had imposed, and had not given adequate consideration to the possibility of reunification. The decision was followed by a series of further judgments against Norway on similar facts. The reasoning of Strand Lobben is strikingly relevant to American practice. The European Court treated the limited contact between mother and child as the state's responsibility rather than evidence against the mother, and it required that adoption against a parent's will be justified by an overriding requirement pertaining to the child's best interests. Those are exactly the issues that American critics raise about bonding analyses and the ASFA timeline. The difference is that in Europe a supranational court with the authority to find violations of human rights has imposed a demanding standard, while in the United States the standard is set by fifty state legislatures and courts operating under the minimal federal constitutional floor described in Chapter 1. That difference is not only a matter of legal culture. It reflects the fact that American constitutional doctrine protects parental rights vigorously in the abstract but polices the actual processes of termination lightly. The system relies on state law, state courts and state-funded lawyers to get these decisions right. When they do, termination is a painful but legitimate act that allows a child to move into a permanent family. When they do not, the error becomes permanent the moment an adoption is finalized. Nowhere is the danger of systemic error clearer, or the law's response more distinctive, than in the treatment of Native American children, the subject of the next chapter. Chapter 5: Identity, Culture and Placement: The Indian Child Welfare Act and Beyond Placement decisions in child welfare are usually described in the language of individual welfare: which home will best meet this child's needs. But children do not arrive in the system as isolated individuals. They belong to families, communities, cultures, faiths and, in the case of Native American children, to sovereign nations. Whether and how the law should take those affiliations into account is among the most contested questions in the field. For Native American children, Congress answered it in 1978 with the Indian Child Welfare Act, the most distinctive statute in American family law. For other children, Congress answered it in the opposite way, forbidding agencies to consider race in most placement decisions. And in recent years, a third front has opened over religious belief and sexual orientation, with the federal government reversing its position twice in as many years. The history behind the statute The Indian Child Welfare Act cannot be understood apart from the history that produced it. For much of the nineteenth and twentieth centuries, federal policy sought to assimilate Native Americans by separating children from their families and cultures. Federal boarding schools removed tens of thousands of children from their communities, often far from home, with the explicit aim of eradicating their languages and traditions. The Department of the Interior's Federal Indian Boarding School Initiative, launched in 2021, documented hundreds of such schools and deaths of children in their custody. As boarding schools declined, the removal continued through child welfare agencies and adoption. Between 1958 and 1967, the Indian Adoption Project, a collaboration between the Bureau of Indian Affairs and the Child Welfare League of America, placed several hundred Native children with non-Native families, mostly in the eastern United States, as a deliberate policy. State and private agencies removed far more. Surveys conducted by the Association on American Indian Affairs in 1969 and 1974 found that roughly 25 to 35 percent of all Native American children had been separated from their families and placed in foster homes, adoptive homes or institutions, and that the great majority of those placements were in non-Native homes. Congressional hearings in the mid-1970s heard testimony that many removals rested on cultural misunderstanding, such as treating care by grandparents or extended family as abandonment, or on standards of housing and poverty that virtually no reservation family could meet. Congress responded with a finding, written into the statute, that an alarmingly high percentage of Indian families were being broken up by the removal, often unwarranted, of their children by nontribal public and private agencies, and that states had often failed to recognize the essential tribal relations of Indian people and the cultural and social standards prevailing in Indian communities and families. What the statute does The Indian Child Welfare Act applies to any child custody proceeding, including foster care placement, termination of parental rights and adoption, involving an "Indian child," defined as an unmarried person under eighteen who is either a member of a federally recognized tribe or eligible for membership and the biological child of a member. The definition turns on political affiliation with a tribe, not on ancestry alone. It does not apply to custody disputes between parents in divorce or to delinquency cases. The Act operates on three levels. First, it allocates jurisdiction. Tribal courts have exclusive jurisdiction over custody proceedings involving Indian children who reside or are domiciled on the reservation, or who are wards of the tribal court. For children living elsewhere, state courts must transfer proceedings to tribal court on request of a parent or the tribe, unless either parent objects, the tribal court declines, or there is good cause not to transfer. The tribe has a right to intervene in state proceedings at any point, and state courts must give full faith and credit to tribal court judgments. Second, it raises the procedural and substantive standards for removal and termination. The state must notify the parents and the tribe of any involuntary proceeding. Indigent parents have a right to appointed counsel. Before placing a child in foster care or terminating parental rights, the state must prove that it made "active efforts" to provide remedial services and rehabilitative programs designed to prevent the breakup of the family, and that those efforts were unsuccessful. Active efforts are more demanding than the reasonable efforts required by general federal law: they contemplate affirmative, thorough and timely help, such as helping a parent obtain services rather than handing over a list of referrals. Foster care placement requires clear and convincing evidence, including the testimony of a qualified expert witness, that continued custody by the parent is likely to result in serious emotional or physical damage to the child. Termination requires the same showing beyond a reasonable doubt, the criminal standard. Third, it establishes placement preferences. In any adoptive placement under state law, absent good cause to the contrary, preference must be given to a member of the child's extended family, then to other members of the child's tribe, then to other Indian families. Foster care placements have a similar hierarchy. Tribes may establish different orders of preference by resolution. The Act also regulates voluntary proceedings. A parent's consent to foster care or relinquishment must be executed before a judge, who must certify that the terms were explained and understood; consent given before or within ten days after birth is invalid; and consent to termination or adoption may be withdrawn at any time before a final decree. A parent, child or tribe may petition to invalidate any placement or termination that violated core provisions of the Act. The difference between ICWA and the ordinary rules is substantial, as Table 3 shows. Table 3. Principal protections in ordinary dependency law and under the Indian Child Welfare Act. Issue Ordinary state and federal law Indian Child Welfare Act Efforts to keep family together Reasonable efforts; may be excused for aggravated circumstances Active efforts, shown to have been unsuccessful Standard for foster placement Varies; often preponderance Clear and convincing evidence, with qualified expert Standard for termination Clear and convincing evidence Beyond a reasonable doubt, with qualified expert Counsel for indigent parents Set by state law Guaranteed by statute Jurisdiction State court Tribal court exclusive or presumptive in many cases Adoptive placement Best interests of the child Statutory preferences: family, tribe, other Indian families Voluntary consent State rules; often irrevocable after short period Before a judge; revocable until final decree The Supreme Court and the statute ICWA has reached the Supreme Court three times, and each decision reflects a different stage in the law's political life. The first, Mississippi Band of Choctaw Indians v. Holyfield (1989), strongly affirmed tribal authority. Twin babies were born to parents who lived on the Choctaw reservation in Mississippi. The mother deliberately gave birth off the reservation, and both parents consented to the babies' adoption by a non-Native couple through a state court. The tribe moved to vacate the adoption, arguing that the children were domiciled on the reservation and that the tribal court had exclusive jurisdiction. The Supreme Court agreed. Domicile, it held, is a matter of federal law under ICWA, and a child's domicile follows the parents'; the parents could not defeat tribal jurisdiction by giving birth off the reservation. The Court emphasized that the statute protects the interests of the tribe as well as those of parents and children, and that the tribe's interest in its children is distinct from the parents'. It acknowledged the pain of disrupting a placement that had by then lasted three years, but held that the tribal court was the right forum to decide what should happen. The tribal court ultimately allowed the twins to remain with their adoptive mother. The second decision, Adoptive Couple v. Baby Girl (2013), went the other way. A non-Native mother, pregnant by a father who was a member of the Cherokee Nation, arranged a private adoption with a couple in South Carolina. The father had indicated in text messages that he would relinquish his rights, and he provided no support during the pregnancy, but when he learned of the adoption he objected and invoked ICWA. The South Carolina courts ruled for him, and the child, who had lived with the adoptive couple for over two years, went to live with him. The Supreme Court reversed by a five-to-four vote. It held that ICWA's provision requiring proof of likely serious damage from "continued custody" did not apply to a father who had never had custody, that the active-efforts requirement did not apply where the parent had abandoned the child before birth and never had custody, and that the placement preferences applied only if an alternative party eligible under the preferences had formally sought to adopt. The child was returned to the adoptive couple. The decision narrowed ICWA's reach in private infant adoptions and was widely read as signaling doubts about the statute's scope. Those doubts came to a head in Haaland v. Brackeen (2023). Texas and several non-Native families who had fostered or sought to adopt Native children brought a sweeping constitutional challenge. They argued that ICWA exceeded Congress's power, that it unconstitutionally commandeered state courts and agencies to carry out federal policy, and that its placement preferences were racial classifications that violated equal protection. The Fifth Circuit, sitting en banc, produced a fractured set of opinions that left some provisions standing and struck others. The Supreme Court, in an opinion by Justice Barrett joined by six other justices, rejected the Article I and anticommandeering challenges in their entirety. Congress's power over Indian affairs, the Court held, is broad enough to support ICWA, and the requirements it places on state courts are valid federal law binding on state judges under the Supremacy Clause. The requirements that apply to agencies, such as active efforts and the recordkeeping obligations, apply equally to private parties and so do not single out the states. The Court did not reach the merits of the equal protection claim, holding that the individual petitioners lacked standing to raise it because a ruling in their favor would not bind the state officials who actually implement the placement preferences, and that Texas lacked standing to assert equal protection rights on behalf of its citizens. Brackeen thus upheld ICWA without resolving the most serious constitutional question hanging over it: whether the placement preferences, which favor Native families over non-Native ones, are political classifications based on tribal membership and therefore subject to deferential review under Morton v. Mancari (1974), or racial classifications subject to strict scrutiny. Justice Kavanaugh wrote separately to say he regarded the equal protection issue as serious and expected it to return in a case with proper standing. Justice Gorsuch wrote separately to emphasize the history of federal child removal policies and to defend the statute as an exercise of tribal sovereignty. That question is the principal legal vulnerability that remains for ICWA, and litigants have continued to press it in state courts. Implementation after Brackeen Even where ICWA's constitutionality is settled, compliance has long been uneven. Studies of state courts have found frequent failures to ask whether a child might be an Indian child, to give proper notice to tribes, and to make active efforts. Much reversible error in termination cases, in states with significant Native populations, arises from ICWA noncompliance, and because a violation of core provisions permits invalidation of the placement, noncompliance creates exactly the kind of late disruption the statute's critics complain about. The federal government has tried to improve compliance. In 2016, the Bureau of Indian Affairs issued the first binding regulations interpreting ICWA, which clarified when a court has "reason to know" a child is an Indian child, what counts as good cause to depart from placement preferences, and who qualifies as an expert witness. In December 2024, the Administration for Children and Families finalized a rule requiring states to report ICWA-related data elements through the federal foster care data system, which will allow, for the first time, systematic national measurement of how often the statute's protections are being applied. States have also acted. Several, including California, Washington, Michigan and Minnesota, have enacted state-law versions of ICWA that codify its protections, sometimes extend them, and would continue to protect Native families even if parts of the federal statute were later invalidated. The movement to enact state ICWAs accelerated around the time of Brackeen. A comparative note shows that the underlying idea is not uniquely American. In 2019 Canada enacted An Act respecting First Nations, Inuit and Métis children, youth and families, which affirms Indigenous jurisdiction over child and family services and sets national standards, including placement priorities that favor family and community. In 2024 the Supreme Court of Canada upheld the Act as a valid exercise of federal power. Australia's Aboriginal and Torres Strait Islander Child Placement Principle, adopted in varying forms by the states and territories after the 1997 Bringing Them Home report on the Stolen Generations, pursues similar goals. Race, religion and placement for everyone else For children who are not Native American, federal law takes a strikingly different approach to identity. In 1972 the National Association of Black Social Workers issued a statement opposing the placement of Black children with white adoptive families, and for two decades many agencies practiced some form of race matching, holding Black children in foster care while they searched for same-race adoptive families. Congress rejected that practice in the Multiethnic Placement Act of 1994 and, more firmly, in the Interethnic Adoption Provisions of 1996, which prohibit any agency receiving federal funds from denying or delaying a foster or adoptive placement on the basis of the race, color or national origin of the child or the prospective parent. The prohibition does not apply to ICWA cases. Federal enforcement has been episodic but real; in 2003, for example, the Department of Health and Human Services imposed a substantial financial penalty on Hamilton County, Ohio, for placement practices it found to violate the statute. The constitutional backdrop is Palmore v. Sidoti (1984), in which the Supreme Court held that a court could not transfer custody of a child from her white mother to her father because the mother had married a Black man and the child might suffer social stigma. The reality of private biases, the Court said, cannot justify racial classifications removing a child from a parent. The Interethnic Adoption Provisions extend that principle to placement decisions by agencies. The tension between the two federal regimes, color-blindness for most children and affiliation-based preferences for Native children, is part of what made Brackeen so charged. Religion has become the newest battleground. Many foster care and adoption agencies are religiously affiliated, and some decline to certify same-sex couples as foster parents. In Fulton v. City of Philadelphia (2021), the Supreme Court unanimously held that Philadelphia violated the Free Exercise Clause when it stopped referring children to Catholic Social Services because the agency would not certify same-sex couples. The Court's reasoning was narrow: because the city's contract allowed a commissioner to grant exceptions to its nondiscrimination requirement at his discretion, the policy was not generally applicable, and the city had not shown a compelling reason to refuse an exception to this agency. The decision left states free to write genuinely neutral rules but signaled that religious agencies would be protected against selective enforcement. The federal executive has swung between positions. In April 2024, the Administration for Children and Families issued a rule requiring states to ensure that "designated placements" were available for children who identify as lesbian, gay, bisexual, transgender, queer or intersex, meaning foster homes trained to support them. A federal court vacated that rule in June 2025, concluding that it exceeded the department's statutory authority; the department proposed formally rescinding it in March 2026 and finalized the rescission in July 2026. Meanwhile, in November 2025, the President issued Executive Order 14359, titled Fostering the Future for American Children and Families, which among other things directed the Secretary of Health and Human Services to address state and local policies that exclude prospective foster parents on the basis of sincerely held religious beliefs and to expand partnerships with faith-based providers. Litigation brought by religious prospective foster parents who were denied licenses in several states because of their views on gender identity was continuing in the lower courts. What unites these fights is a question the child-centered language of best interests tends to obscure: whose values should shape the home a child is placed in when the state is choosing it? For Native children, federal law gives a large part of the answer to the tribe. For other children, it forbids consideration of race, permits consideration of culture and religion within limits that are still being worked out, and leaves the rest to caseworkers and judges. The one thing every side agrees on is that the choice matters. Whatever home the state chooses will be the one in which the child grows up, and when things go wrong in that home, the state is responsible in a way it is not when the child lives with a parent. That responsibility is the subject of the next chapter. Chapter 6: When the State Is the Parent At the moment of removal, the state makes an implicit promise. It tells a court, and through the court a family, that the child will be safer in its care than at home. For many children the promise is kept. Foster parents, relatives and caseworkers provide stability and affection that the children had not known, and many former foster youth describe their foster families as the people who saved them. But the promise is not self-executing, and the history of American foster care includes long periods in which whole state systems broke it on a large scale: children moved through a dozen placements in a year, abused in foster homes the state had licensed and failed to monitor, housed in hotels, offices and juvenile detention centers because no placement could be found, and medicated with powerful psychiatric drugs in combinations no one was tracking. This chapter asks what legal recourse exists when the state is the harmful parent. The answer is that the law recognizes a real duty, enforces it through tools that are cumbersome and slow, and has achieved genuine but partial reforms at enormous cost in time and litigation. The scale and shape of the problem At any given time in recent years, roughly 340,000 to 370,000 children have been in foster care in the United States, with a larger number passing through care over the course of a year. About a third or more live with relatives. Most of the rest live with licensed non-relative foster families, and a smaller share live in group homes and residential treatment facilities. The median stay is around a year and a half, but the distribution is skewed: many children return home within months, while others remain for years. The failures that have drawn legal challenges follow recurring patterns. The first is placement instability. Children who move repeatedly lose schools, friends, therapists and siblings, and each move deepens the trauma that brought them into care. Research consistently associates multiple placements with worse behavioral and educational outcomes, although causation runs in both directions. The second is a shortage of placements for children with significant needs, particularly adolescents with mental health or behavioral problems. When a state reduces reliance on group care without building enough therapeutic foster homes, children end up in improvised settings. In the late 2010s and early 2020s, press reports and court filings documented children sleeping in state offices, hotel rooms and emergency shelters in states including Texas, Washington, Oregon and Illinois, sometimes for weeks. The third is maltreatment in care itself. Children in foster care can be abused by foster parents, by staff in residential facilities, or by other children. Agencies that are understaffed and overburdened may fail to conduct adequate licensing checks, fail to visit, or fail to investigate complaints. The fourth is overmedication. A 2011 report by the Government Accountability Office found that children in foster care in the states it examined were prescribed psychotropic drugs at markedly higher rates than other children covered by Medicaid, sometimes in doses exceeding maximum guidelines or in combinations of five or more drugs at once. Congress responded by requiring states to develop oversight plans for psychotropic medication, and many states now require additional review for young children or unusual prescribing patterns. The fifth is the treatment of children's own money. Many foster children are entitled to Social Security survivor benefits or Supplemental Security Income because a parent has died or because they have disabilities. States routinely apply to become the children's representative payees and then use the benefits to reimburse themselves for the cost of foster care. In Washington State Department of Social and Health Services v. Guardianship Estate of Keffeler (2003), the Supreme Court held that this practice does not violate the Social Security Act's anti-attachment provision. Investigative reporting in 2021 by The Marshall Project and NPR documented how widespread the practice was, and the legal scholar Daniel Hatcher had earlier criticized it at length. Beginning with Maryland in 2023, several states have enacted laws requiring that at least some of these benefits be conserved for the child rather than taken by the state. The sixth is what happens at the end. Every year, roughly twenty thousand young people leave foster care by aging out, without reunification, adoption or guardianship. Longitudinal studies, most notably the Midwest Evaluation of the Adult Functioning of Former Foster Youth led by Mark Courtney, have documented high rates of homelessness, incarceration, unemployment and early parenthood among this group. Federal law, through the Chafee program created in 1999, funds transition services and education vouchers, and more than half the states now extend foster care beyond eighteen under the option created in 2008. The Executive Order issued in November 2025 directed further federal support for former foster youth, including an online platform consolidating information on federal and state programs. The constitutional duty Chapter 1 noted that DeShaney v. Winnebago County denied any constitutional duty to protect a child left at home but acknowledged that a duty arises when the state takes a person into custody and holds him against his will. The source of that duty is Youngberg v. Romeo (1982), in which the Supreme Court held that a man involuntarily committed to a state institution for people with intellectual disabilities retained substantive due process rights to reasonably safe conditions of confinement, freedom from unreasonable bodily restraint, and such minimally adequate training as was necessary to ensure those rights. The Court held that the state satisfies those obligations if its decisions reflect the exercise of professional judgment. Lower federal courts extended the principle to foster care even before DeShaney. In Doe v. New York City Department of Social Services (1981), the Second Circuit held that a child agency could be liable when its deliberate indifference to known abuse by a foster parent caused harm to the child. In Taylor v. Ledbetter (1987), the Eleventh Circuit, sitting en banc, held that a child involuntarily placed in a foster home is in a situation analogous to that of the committed patient in Youngberg, and has a constitutional right to be free from unnecessary pain and a fundamental right to physical safety. In K.H. ex rel. Murphy v. Morgan (1990), the Seventh Circuit, in an opinion by Judge Richard Posner, held that Illinois officials could be liable for shuttling a young girl through a series of placements, some abusive, reasoning that a state which removes a child from her parents cannot then place her, with deliberate indifference, where she is likely to be harmed. After DeShaney, courts have consistently read its custody exception to cover children in state foster care, and every federal circuit to address the issue recognizes some version of a substantive due process right to safety in care. They disagree on the standard. Some ask whether officials were deliberately indifferent to a known or obvious risk; others apply Youngberg's professional judgment standard, asking whether the officials' decisions departed substantially from accepted professional practice. The difference matters, because deliberate indifference is harder for plaintiffs to prove. Courts also disagree about the scope of the right: whether it covers only physical safety or extends to protection from psychological harm, to the provision of necessary services, or to a stable placement. The distinction between the duty owed a child in foster care and the absence of any duty to a child at home creates an odd asymmetry. A caseworker who fails to remove a child from a dangerous parent faces, under DeShaney, no federal constitutional liability at all. A caseworker who places a child in a foster home that turns out to be dangerous may face liability. Some commentators have worried that this asymmetry encourages removal. In practice, qualified immunity and the demanding standards of proof make individual damages liability rare in either direction. Systemic litigation The most significant legal tool for reforming foster care has been not individual damages suits but class actions seeking injunctive relief against entire state systems. Beginning in the late 1970s and accelerating in the 1990s and 2000s, advocacy organizations, most prominently the national nonprofit Children's Rights, sued state and local systems on behalf of all children in their custody, alleging violations of substantive due process, federal child welfare statutes and, in some cases, state law. Most of these cases ended in consent decrees: court-enforceable settlements in which the state committed to specific reforms, such as caseload limits for caseworkers, maximum numbers of placements, timeliness standards for investigations and visits, and data reporting, monitored by a neutral expert who reports to the court. The early landmark was LaShawn A. v. Barry, a challenge to the District of Columbia's child welfare system. In 1991 the district court found that the system violated federal statutory requirements and the children's constitutional rights, and in 1995, after the city failed to comply with the remedial order, the court placed the agency in receivership, transferring its management to a court-appointed receiver. The District regained control of its agency in the early 2000s, and the case continued for many years under monitoring. Other significant cases include Braam v. State of Washington (2003), in which the Washington Supreme Court held that children in foster care have a substantive due process right under the federal Constitution to be free from unreasonable risks of harm, including from repeated placement moves, leading to a settlement in 2004; Kenny A. v. Perdue, whose 2005 consent decree required caseload limits and other reforms in two Georgia counties and which produced a Supreme Court decision in 2010 on the calculation of the plaintiffs' attorney's fees; Olivia Y. v. Barbour, a challenge to Mississippi's statewide system settled in 2008; and Jonathan R. v. Justice, filed against West Virginia in 2019. The most prolonged recent case is M.D. v. Abbott, filed in 2011 on behalf of children in Texas's long-term foster care. In 2015, Judge Janis Graham Jack of the Southern District of Texas found that the state had violated the children's substantive due process right to be free from an unreasonable risk of harm, citing among other things excessive caseworker caseloads, inadequate monitoring of placements and the practice of placing children in group settings without adequate supervision. The Fifth Circuit affirmed the core liability findings in 2018 but narrowed the remedy, and the court appointed monitors whose reports documented continuing problems, including children without placement who slept in offices and hotels. The district court held the state in contempt more than once, and the case continued into the mid-2020s amid disputes over compliance and over the appropriate scope of federal supervision. These cases illustrate both the power and the limits of structural litigation. They have unquestionably produced improvements. Caseloads fell, data systems were built, monitoring became routine, and legislatures appropriated money they would not otherwise have spent. But the decrees often lasted decades, the gains sometimes eroded after court supervision ended, and critics argued that the decrees froze particular policy choices in place and substituted the judgment of monitors for that of elected officials. The Supreme Court has encouraged federal courts to release states from such decrees once the underlying violations have been remedied. In Horne v. Flores (2009), a case about English-language instruction in Arizona schools, the Court held that institutional reform decrees must be modified when changed circumstances mean continued enforcement is no longer necessary to remedy a violation of federal law, emphasizing that such decrees raise federalism concerns because they bind state officials and future legislatures. States under long-running child welfare decrees have invoked Horne to seek release. The doctrinal obstacles Structural litigation in this area faces several recurring legal hurdles, which help explain why some systems have been sued repeatedly while others have never been sued at all. The first is abstention. Because children in foster care are subject to continuing state court jurisdiction in their individual dependency cases, states argue that federal courts should decline to intervene under Younger v. Harris (1971), which counsels federal restraint when there are ongoing state proceedings. The circuits have split. In 31 Foster Children v. Bush (2003), the Eleventh Circuit held that a federal challenge to Florida's foster care system should be dismissed under Younger because the relief sought would interfere with the state dependency courts' supervision of each child. In Jonathan R. v. Justice (2022), the Fourth Circuit reached the opposite conclusion, holding that the West Virginia plaintiffs' claims against executive officials did not threaten to interfere with the individual state court proceedings and that abstention was inappropriate. The Supreme Court's decision in Sprint Communications v. Jacobs (2013), which confined Younger to a narrow set of proceedings, has strengthened the plaintiffs' position, but the issue is not settled nationally. The second is whether federal child welfare statutes create rights that children or foster parents can enforce. Suter v. Artist M. held that the reasonable-efforts provision does not. Gonzaga University v. Doe (2002) tightened the test, requiring that a statute unambiguously confer an individual right. In Health and Hospital Corporation of Marion County v. Talevski (2023), the Supreme Court reaffirmed that Spending Clause statutes can create privately enforceable rights under section 1983 if they meet that demanding standard, but in Medina v. Planned Parenthood South Atlantic (2025) it emphasized how rarely such statutes do. The lower courts have divided over particular provisions of Title IV-E, most notably the requirement that states make foster care maintenance payments to foster parents covering the cost of food, clothing, shelter and other needs. The Ninth Circuit held in 2010 that foster parents could enforce that provision, the Sixth Circuit held in 2017 that relative caregivers could enforce a right to payments, and the Eighth Circuit reached the opposite conclusion in 2013. The third is the scope of the substantive due process right itself. Courts have been more willing to recognize a right to protection from physical harm than a right to services, treatment or a permanent family, and several have rejected claims to a constitutional right to a stable placement or to be reunified with family. The routes available to children harmed in care, and the obstacles each faces, are compared in Table 4. Table 4. Legal routes for challenging harm in foster care. Route Legal basis Typical relief Principal obstacle Individual damages suit against officials Section 1983, substantive due process Money damages Qualified immunity; deliberate indifference standard Damages suit against local agency Section 1983, municipal liability Money damages Must show policy or custom caused harm Systemic class action Due process; federal statutes Injunction or consent decree Abstention; enforceability of statutes; duration State tort claim Negligence, state statutes Money damages Sovereign or governmental immunity; damages caps Federal administrative review Title IV-E and IV-B conditions Program improvement plans; funding penalties Weak penalties; long review cycles State oversight bodies Ombudsman or inspector statutes Investigation and public reports Usually no binding authority Administrative oversight and its weakness The federal government has its own oversight tool: the Child and Family Services Reviews, periodic audits in which federal and state reviewers assess each state's performance on outcomes such as safety, permanency and well-being. States that fall short must adopt program improvement plans, and federal law authorizes financial penalties for failure to improve. In practice, no state has been found in full conformity with all the outcomes measured in the reviews, and penalties have been modest and frequently suspended while states improve. The November 2025 Executive Order directed the Department of Health and Human Services to publish annual scorecards of state performance, a step toward greater transparency whose effect remains to be seen. Many states have created independent children's ombudsmen or inspectors general for their child welfare agencies. These offices investigate complaints and publish reports, and some have been influential. But most lack binding authority. What the pattern shows The legal architecture described here has an internal logic. The Constitution recognizes a duty once the state takes custody. Class actions give that duty teeth when a system fails broadly enough, and consent decrees translate it into measurable standards. Federal audits and state oversight supplement the courts. But every layer is slow, the most powerful tools are available only after failure has become systemic and visible, and none addresses the fundamental difficulty that states often lack the placements, caseworkers and services they need. For the argument of this book, the most important consequence is this. The harms suffered in foster care are not separate from the question of termination and adoption; they feed into it. A child who has been moved through eight placements and has developed serious behavioral problems is harder to reunify and harder to adopt. A parent whose visits were cancelled because the agency lacked transport has a weaker record at trial. The failures of the state as parent become evidence in the case against the birth parent, and they shape the population of children who will eventually be available for adoption. It is to adoption itself that we now turn. Chapter 7: Adoption at Home: Consent, Fathers, Subsidies and Breakdown Adoption is the legal act that turns the provisional arrangements of foster care, or the private decision of a birth parent, into a permanent family. It is a judicial decree, not a contract, and it produces a status rather than an agreement: the adoptive parents become the child's parents for every legal purpose, and in most states the child's prior legal family disappears from the record. Because it is permanent, adoption depends entirely on the validity of what preceded it. A defective termination, an invalid consent or an overlooked father can unravel an adoption that everyone believed was final, and the law of domestic adoption is to a remarkable degree the law of making sure that does not happen. A short history of a modern institution Adoption as a judicial process is an American invention of the nineteenth century. English common law did not recognize it at all until 1926, preferring to protect inheritance through blood. In 1851, Massachusetts enacted what is generally regarded as the first modern adoption statute, requiring a judge to approve each adoption and to find that the adoptive parents were fit and that the adoption was proper. The statute placed the child's welfare, rather than the needs of the adults or the transfer of labor, at the center of the process, and other states quickly copied it. For much of the twentieth century, American adoption was shaped by secrecy. Beginning in the 1930s and 1940s, states sealed adoption records and issued amended birth certificates that listed the adoptive parents as though they were the birth parents. The practice was justified as protecting the child from the stigma of illegitimacy, the birth mother from exposure and the adoptive family from interference. In the decades after the Second World War, often called the "baby scoop era," large numbers of unmarried young women were pressured into relinquishing infants, frequently in maternity homes, under conditions that many later described as coercive. The oral histories collected by Ann Fessler in The Girls Who Went Away (2006) are a leading account. The sealed-records regime has been eroding for decades. Kansas and Alaska never sealed original birth certificates for adult adoptees. Oregon voters approved Measure 58 in 1998, restoring access, and many states have since followed, including New York, which opened original birth certificates to adult adoptees in January 2020. Direct-to-consumer genetic testing has made secrecy impossible to guarantee in any event. The law has moved, in other words, from treating the adoptee's origins as something to be erased toward treating them as something the adoptee has a right to know. Pathways into adoption Adoptions in the United States fall into several categories. Stepparent adoptions, in which a parent's new spouse adopts the child, are among the most common and usually the simplest, although they require either the consent of the other birth parent or the termination of that parent's rights. Relative adoptions by grandparents, aunts or uncles are also common. Adoptions from foster care, which follow an involuntary termination or a relinquishment in a dependency case, number roughly fifty thousand a year, and the large majority of those children are adopted by their foster parents or relatives. Private infant adoptions, arranged either through licensed agencies or independently through lawyers, involve a birth mother's decision to place a newborn and number far fewer than they did in the mid-twentieth century. Intercountry adoptions, discussed in the next chapter, have fallen steeply. Each pathway raises different legal problems. In foster care adoptions, the key questions have usually been resolved in the dependency and termination proceedings, and the adoption itself is often an uncontested ceremony. In private infant adoptions, the key questions are about consent: who must give it, when it can be given, and when it can be withdrawn. Consent and its limits Every state requires the consent of the birth mother to an adoption, unless her rights have been terminated, and every state regulates when and how she may give it. Most prohibit consent before birth and many require a waiting period after birth, ranging from a day or two to more than a week, to protect against decisions made under the immediate strain of delivery. Consent must usually be in writing and witnessed or taken before a judge or notary. States differ widely on revocation. Some make consent irrevocable once given except for fraud or duress; others allow revocation for a fixed period, commonly a few days to a month; a few allow revocation until the adoption is finalized if revocation would be in the child's best interests. Consent obtained by fraud, duress or undue influence can be set aside, but courts construe those terms narrowly. Emotional pressure from family or agency staff, promises about future contact that go unfulfilled, and the mother's own later regret are generally not enough. The result is a body of law that strongly favors finality once a consent has been executed properly, on the theory that a child cannot be left in legal limbo while a birth parent reconsiders. Every state also forbids buying and selling children. Adoptive parents may pay certain expenses, typically the birth mother's medical costs, reasonable living expenses during pregnancy, counseling and legal fees, but payments beyond those limits are criminal in most states and can invalidate a consent. The line between permitted support and prohibited inducement is not always clear, and the growth of adoption facilitators and online matching services has made enforcement more difficult. When a child is placed across state lines, as happens frequently in private adoptions, the Interstate Compact on the Placement of Children applies. The Compact, adopted by every state, requires the sending state to obtain approval from the receiving state before a child is placed there, and it gives the receiving state an opportunity to assess the prospective home. The Compact also governs many foster care placements across state lines. Its procedures are notoriously slow, and delays in interstate placement are a persistent source of frustration in foster care cases, where a willing relative in another state may wait months for approval. The problem of fathers The most contested consent question involves unmarried fathers. Chapter 1 described the constitutional framework: under Lehr v. Robertson, an unwed father who has not established a relationship with his child is not entitled to the same protection as one who has, and states may use putative father registries to identify which fathers must be notified of an adoption. But the constitutional minimum leaves room for very different state approaches, and the differences have produced some of the most painful cases in American family law. Two cases in the early 1990s defined the problem for a generation. In the case known as "Baby Jessica," an Iowa mother placed her newborn with a Michigan couple and named the wrong man as the father. The biological father learned of the birth within weeks, sought custody and never consented. After more than two years of litigation in the courts of both states, the Michigan Supreme Court, in In re Clausen (1993), held that Iowa's courts had jurisdiction and that their order returning the child to her biological parents had to be enforced. The child was transferred to her father at the age of two and a half, in scenes broadcast nationally. In the case known as "Baby Richard," an Illinois mother told the father that the baby had died, and placed the child for adoption. When the father learned the truth, he sought custody. The Illinois Supreme Court held in 1994 that the adoption was invalid because the father had not consented and had not been shown to be unfit, and the child was returned to him at nearly four years of age. These cases produced a legislative reaction in two directions. Some states tightened their rules to protect finality, requiring fathers to register or file paternity actions within strict deadlines, sometimes before birth or within days afterward, on pain of losing any right to notice or to object. Utah became known for a particularly strict regime, and a number of adoption disputes involved mothers who traveled to Utah to place children for adoption there, over the objections of fathers in other states who had not complied with Utah's requirements. Other states took the opposite approach, requiring more diligent efforts to identify and notify fathers before an adoption can proceed. The underlying tension has not been resolved. A father who does not know about a pregnancy cannot register, and one who is deliberately deceived cannot easily protect his rights. On the other hand, a child placed with an adoptive family at birth develops attachments that make a later transfer traumatic, and the law has a legitimate interest in knowing quickly who must consent. The best answer most states have found is procedural: requiring reasonable efforts to identify and give notice to known fathers, while allowing adoptions to proceed promptly when fathers who have notice fail to act. Who may adopt For most of the twentieth century, agencies and courts screened adoptive parents in ways that excluded large groups of people: single adults, older couples, people with disabilities, people of different faiths from the child's birth family, and above all same-sex couples. Most of these exclusions have fallen. Florida's statutory ban on adoption by gay men and lesbians, enacted in 1977, was held unconstitutional by a Florida appellate court in 2010. Mississippi's ban on adoption by same-sex couples was enjoined by a federal district court in 2016. After Obergefell v. Hodges (2015) required states to license and recognize same-sex marriages, the Supreme Court made clear that its consequences extended to parenthood. In V.L. v. E.L. (2016), it summarily reversed an Alabama Supreme Court decision that had refused to recognize a Georgia adoption by a woman's same-sex partner, holding that Alabama was required to give full faith and credit to the Georgia judgment. In Pavan v. Smith (2017), it held that Arkansas could not refuse to list a mother's female spouse on a child's birth certificate when it listed husbands in the same circumstances. Second-parent adoptions, in which a parent's partner adopts without the first parent giving up rights, are now available in most states and remain an important protection for families formed through assisted reproduction, even where both spouses are already presumed parents. The questions that remain concern religious liberty rather than eligibility. As discussed in Chapter 5, the Supreme Court's decision in Fulton v. City of Philadelphia protected a religious agency's refusal to certify same-sex couples under a contract that allowed discretionary exceptions, and several states have enacted laws expressly protecting faith-based agencies' ability to decline placements that conflict with their beliefs. Money: subsidies and tax credits Adoption from foster care is financially supported in ways that private adoption largely is not. Since 1980, Title IV-E has funded adoption assistance payments for children with "special needs," a term that sounds narrow but in practice covers most children adopted from foster care. A state may designate a child as having special needs because of age, membership in a sibling group, minority status, or a physical, mental or emotional condition, provided the state first determines that the child cannot be returned home and that reasonable efforts to place the child without a subsidy were unsuccessful. Adoptive parents negotiate an agreement with the state setting a monthly payment, which may not exceed the foster care payment the child would have received, and the child usually retains Medicaid eligibility. Payments generally continue until the child turns eighteen and in many states to twenty-one. These subsidies matter. Many foster parents could not afford to adopt without them, and some would be financially worse off adopting than remaining foster parents. Adoption assistance also has an important legal feature: because it is governed by the agreement signed before finalization, adoptive parents who fail to negotiate adequate terms, or who are not told of their eligibility, may find it difficult to obtain assistance later. Federal policy allows modification in some circumstances, and some courts have required states to honor subsidy obligations they failed to disclose. Separately, federal tax law has provided an adoption tax credit since 1996, covering qualified adoption expenses up to a cap that is indexed for inflation and now exceeds fifteen thousand dollars per child, with the full credit available for special-needs adoptions from foster care regardless of actual expenses. Legislation enacted in 2025 made a portion of the credit refundable, which matters to lower-income families who owe little or no income tax and had been unable to use it. When adoptions break down The permanence of adoption is a legal fact, not a guarantee of family stability, and a minority of adoptions fail. Practitioners distinguish between disruption, in which a placement ends before the adoption is finalized, and dissolution, in which a finalized adoption is legally ended and the child returns to state custody or goes to another family. Research reviews by federal agencies and academic researchers have commonly estimated disruption rates in the range of about ten to twenty-five percent of adoptive placements and dissolution rates in the low single digits, with higher rates for older children and those with serious behavioral needs. These figures vary widely across studies, and many post-adoption breakdowns, where the child leaves the adoptive home without a formal legal change, are not captured at all. The most troubling form of breakdown came to public attention in 2013, when a Reuters investigation led by Megan Twohey, titled "The Child Exchange," documented a practice of "private rehoming": adoptive parents, often of children adopted internationally, advertising the children on internet message boards and transferring them to strangers using only a notarized power of attorney, without any court or agency involvement. Some children were transferred to people who abused them. The practice exploited a gap in the law: a parent may delegate care of a child to another person temporarily by power of attorney, and no state then required any review of such transfers. In response, a number of states, beginning with Wisconsin in 2014, enacted laws prohibiting unregulated custody transfers and advertising of children, and federal agencies strengthened requirements for pre-adoption training and post-adoption support in intercountry adoption. Adoptive parents who were misled about a child's history have also turned to the courts. In Burr v. Board of County Commissioners of Stark County (1986), the Ohio Supreme Court recognized a claim for "wrongful adoption" against an agency that had fraudulently concealed a child's serious medical and psychiatric history from his adoptive parents. Courts in many states have since recognized similar claims based on intentional or negligent misrepresentation, and many states now require agencies to disclose available health and social history before an adoption is finalized. The existence of dissolution, rehoming and wrongful adoption claims tells us something important about the institution. Adoption's legal permanence is valuable precisely because children need to know that the family they are in is theirs for good. But permanence works only if the adoption was built on accurate information, adequate preparation and continuing support. When those are missing, the law's promise of finality can conceal a placement that is failing. Safe havens and the edges of adoption One further development illustrates how adoption law has adapted to public concern. Beginning with Texas in 1999, every state has enacted a "safe haven" law allowing a parent to surrender an infant anonymously at a hospital, fire station or other designated location within a specified period after birth, without prosecution for abandonment. The infant is taken into state custody, parental rights are terminated after a period, and the child is placed for adoption. Some states have authorized "baby boxes," climate-controlled receptacles in which infants can be left without any contact with staff. Safe haven laws attracted renewed attention after the Supreme Court's 2022 decision in Dobbs v. Jackson Women's Health Organization, in which the availability of such laws was raised as a consideration. Safe haven laws show adoption at its most stripped-down: a severance that requires no investigation, no consent proceeding and no identified father. They are widely supported, and they have saved lives. But they also raise the questions this book keeps returning to. A father whose child is surrendered anonymously may never learn the child exists. A child adopted after an anonymous surrender may never know anything about her origins. The legal simplicity that makes safe haven laws effective is also the simplicity that removes the safeguards on which adoption's legitimacy ordinarily depends. That tension is at its most extreme, and its consequences most global, in intercountry adoption. Chapter 8: Adoption Across Borders: The Hague Convention and Its Discontents Intercountry adoption is the most international and the most contested form of family formation. It moves children from one legal system to another, typically from poorer countries to richer ones, and it depends on the cooperation of governments, agencies, courts and immigration authorities in at least two states. At its best, it has given hundreds of thousands of children families when their own countries could not. At its worst, it has created markets in children, with falsified papers, coerced relinquishments and outright abductions laundered through legal processes into adoptions that the receiving country then treats as valid and final. The central legal problem of intercountry adoption is the one this book has traced throughout: an adoption is only as legitimate as the severance that preceded it. In a domestic case, the court that grants the adoption can examine the termination or consent on which it rests. In an intercountry case, the severance happens in one country and the adoption is recognized in another, often by officials who cannot read the documents, cannot interview the birth family and have every incentive to take the paperwork at face value. The 1993 Hague Adoption Convention is an attempt to solve that problem by making the country of origin responsible for guaranteeing the severance and making the receiving country rely on that guarantee. Its history is the history of how well that design has worked. From war orphans to a global market Modern intercountry adoption began in the aftermath of the Second World War and grew with the Korean War. In the mid-1950s, Harry and Bertha Holt, an Oregon farming couple, adopted eight Korean children with the help of a special act of Congress and went on to found an agency that became one of the largest intermediaries in the world. Over the following decades, South Korea sent an estimated two hundred thousand children abroad, roughly two-thirds of them to the United States, long after the war that began the practice had ended. The flow continued for reasons that had less to do with orphanhood than with poverty, the stigma of single motherhood, the absence of a domestic welfare system and the financial interests of private agencies. Other countries followed as sources, often in waves tied to war, political upheaval or economic crisis: Vietnam at the end of the war there, Latin American countries in the 1970s and 1980s, Romania after the fall of Ceaușescu in 1989 revealed institutions holding tens of thousands of neglected children, Russia and other former Soviet states in the 1990s, China from the early 1990s, and later Guatemala, Ethiopia and others. By 2004, the peak year, American families adopted roughly twenty-three thousand children from abroad, according to the State Department. Since then the number has fallen dramatically, to fewer than two thousand a year in the early 2020s. The decline has several causes: the growth of domestic adoption and foster care in countries of origin, falling birth rates and rising incomes, political conflicts, the costs of compliance with the Hague regime and, above all, the closure of programs after scandals. The international framework The first modern international instrument to address intercountry adoption directly was the United Nations Convention on the Rights of the Child, adopted in 1989. Article 21 recognizes intercountry adoption as an alternative means of care if a child cannot be placed in a foster or adoptive family or cared for in any suitable manner in the child's country of origin. This is the principle of subsidiarity: care within the child's own country, and especially within a family, comes first. Article 21 also requires that intercountry adoption not result in improper financial gain for those involved. The United States signed the Convention on the Rights of the Child but, alone among the members of the United Nations, has never ratified it. The Hague Convention on Protection of Children and Co-operation in Respect of Intercountry Adoption, concluded in May 1993 under the auspices of the Hague Conference on Private International Law, turned these principles into a working system. It has about 107 contracting states. The United States signed it in 1994, ratified it in December 2007, and it entered into force for the United States on 1 April 2008. The Convention has three stated objects: to establish safeguards ensuring that intercountry adoptions take place in the best interests of the child and with respect for the child's fundamental rights; to establish a system of cooperation among contracting states to ensure that those safeguards are respected and thereby prevent the abduction, sale of or traffic in children; and to secure the recognition in contracting states of adoptions made in accordance with the Convention. Its architecture divides responsibility between the two states involved. Under Article 4, an adoption may take place only if the competent authorities of the state of origin have established that the child is adoptable; have determined, after giving due consideration to possibilities for placement within the state of origin, that intercountry adoption is in the child's best interests; and have ensured that the necessary consents have been freely given, in writing, after counseling, without inducement by payment or compensation of any kind, and that the mother's consent was given only after the birth of the child. Where appropriate given the child's age and maturity, the child's own wishes and consent must be considered. Under Article 5, the receiving state's authorities must determine that the prospective adoptive parents are eligible and suited to adopt, that they have been counseled, and that the child will be authorized to enter and reside permanently. Each contracting state must designate a Central Authority to carry out the Convention's obligations and cooperate with its counterparts. Under Article 17, a child may be entrusted to prospective adoptive parents only if both Central Authorities have agreed that the adoption may proceed. Under Article 23, an adoption certified by the competent authority of the state where it was made as having been made in accordance with the Convention must be recognized by operation of law in every other contracting state, and under Article 24 recognition may be refused only if the adoption is manifestly contrary to public policy, taking into account the child's best interests. Article 29 generally forbids contact between prospective adoptive parents and the child's parents or caregivers until the key requirements have been met, a rule aimed at preventing private deals. Article 32 forbids improper financial or other gain, while allowing reasonable costs and professional fees. The Convention is thus a treaty of trust. The receiving state relies on the state of origin's certification that the child was lawfully available for adoption, and in exchange the Convention gives the resulting adoption near-automatic recognition. The weakness of the design follows directly: if the state of origin's authorities are corrupt, under-resourced or captured by the adoption industry, the Convention's machinery will certify and then universally recognize adoptions that should never have occurred. How the United States implements the Convention Congress implemented the Convention through the Intercountry Adoption Act of 2000, which designated the Department of State as the United States Central Authority, required that agencies and individuals providing adoption services in Convention cases be accredited or approved, and established a federal framework for recognition of Convention adoptions. The accreditation standards appear in federal regulations and cover agency licensing, finances, fee disclosure, training of prospective parents, information about the child, and the handling of complaints. In 2012, Congress passed the Intercountry Adoption Universal Accreditation Act, which took effect in 2014 and extended the accreditation requirement to adoptions from countries that are not parties to the Convention. Since 2017, accreditation has been administered by a designated accrediting entity under contract with the State Department. Immigration law provides the other half of the process. Children adopted from abroad need an immigrant visa to enter the United States as the children of their adoptive parents, and the path depends on whether the country of origin is a Convention country. Table 5 compares the two principal routes. Table 5. The two principal immigration routes for intercountry adoption into the United States. Feature Convention (Hague) process Non-Convention (orphan) process Governing definition "Convention adoptee" "Orphan" under immigration law Parent suitability filing Form I-800A Form I-600A Child eligibility filing Form I-800, before adoption or custody Form I-600, usually after adoption Who determines adoptability Country of origin Central Authority, verified by U.S. U.S. immigration officials, applying U.S. orphan definition Agency requirement Accredited or approved primary provider Accredited primary provider since 2014 Recognition Hague certificate; recognized in all Convention states Depends on foreign decree and state law A critical difference lies in who decides whether the child was eligible. In the orphan process, American immigration officials determine whether the child meets the statutory definition of an orphan, which requires either the death, disappearance, abandonment or desertion of, or separation or loss from, both parents, or a sole or surviving parent who is incapable of providing proper care and has irrevocably released the child. In the Convention process, the Central Authority of the state of origin makes the adoptability determination, and the United States confirms it before the adoption proceeds. The Convention thus moves the key decision earlier and places it formally in the hands of the country where the child's family lives. One further piece of American law has proved consequential for adoptees. The Child Citizenship Act of 2000, effective in February 2001, grants automatic citizenship to foreign-born children adopted by American citizens, provided they are under eighteen, are lawful permanent residents and are in the legal and physical custody of a citizen parent. The Act did not apply to adoptees who were already eighteen when it took effect, and some adoptive parents of earlier generations never completed naturalization for their children. As a result, some adults who were brought to the United States as infants, raised by American families and never knew they were not citizens, have been deported after criminal convictions to countries where they have no family and do not speak the language. The case of Adam Crapser, adopted from South Korea as a child and deported there in 2016, drew particular attention. Legislation to extend automatic citizenship to all international adoptees, generally called the Adoptee Citizenship Act, has been introduced in successive Congresses but, as of this writing, has not been enacted. Scandal and closure The history of intercountry adoption since the 1990s can be read as a series of national programs that grew rapidly, attracted money, generated evidence of fraud and closed. Cambodia is an early example. In 2001, the United States suspended processing of adoptions from Cambodia amid evidence that children were being bought from poor families and supplied with false papers. A federal investigation led to the prosecution of an American adoption facilitator, Lauryn Galindo, who pleaded guilty in 2004 to visa fraud and money laundering conspiracy charges in connection with Cambodian adoptions. Guatemala is the clearest case of a market. In the mid-2000s, Guatemala sent more children to the United States per capita than any other country; by some estimates, roughly one in every hundred children born there was being adopted by an American family. Adoptions were processed largely through private notaries rather than courts, with birth mothers' relinquishments obtained by intermediaries. Investigations documented payments to birth mothers, falsified DNA samples and children stolen from their parents. Guatemala acceded to the Hague Convention, and the United States, finding that Guatemala could not meet its Convention obligations, stopped processing new adoptions from Guatemala after 2007. The program never reopened. Ethiopia grew rapidly as a source in the late 2000s, with thousands of children adopted by American families, many of them described as orphans who in fact had living parents who had been misled about what adoption meant, believing their children would receive an education abroad and return. In 2014, federal prosecutors charged employees of an American agency, Celebrate Children International, in connection with bribery of Ethiopian officials to facilitate adoptions, and several pleaded guilty. In January 2018, Ethiopia's parliament banned intercountry adoption altogether. Russia closed to American families for political reasons, though the pretext drew on adoption failures. In 2010, a Tennessee adoptive mother put her seven-year-old son, adopted from Russia, on a flight to Moscow alone with a note saying she no longer wanted him. The case caused outrage in Russia and contributed to a bilateral adoption agreement signed in 2011. In December 2012, in retaliation for the Magnitsky Act, which imposed American sanctions on Russian officials, Russia enacted a law banning adoption of Russian children by American citizens. The law was named after Dima Yakovlev, a Russian-born toddler who had died in 2008 after his adoptive father left him in a car in Virginia. China was for two decades the largest source country. Adoptions grew in the 1990s against the background of the one-child policy and the abandonment of girls, and Chinese adoptions were widely regarded as well regulated. But reporting in 2005 about orphanages in Hunan province buying babies from traffickers, and later studies documenting how local family-planning officials had seized children from families, complicated that picture. After China's own birth rate collapsed and the one-child policy was abandoned, the number of children placed abroad fell steeply. In September 2024, China announced that it was ending intercountry adoption, except for adoptions by blood relatives and stepchildren. Families whose adoptions were in process when the announcement was made were left without recourse. South Korea offers perhaps the most consequential reckoning. Korean adoptees, many now in middle age, have for years pressed claims that their records were falsified: that children with known parents were registered as abandoned orphans, that identities were swapped, and that agencies placed children abroad without valid consent. Beginning in 2022, hundreds of adoptees petitioned South Korea's Truth and Reconciliation Commission to investigate. In March 2025, the Commission concluded that the Korean state bore responsibility for human rights violations in the overseas adoption program, including falsified documentation, inadequate verification of consent and the delegation of state responsibilities to private agencies operating for profit, and it recommended an official apology and remedial measures. South Korea, which had signed the Hague Convention in 2013, ratified it in June 2025, and the Convention entered into force for South Korea on 1 October 2025. Accompanying domestic reforms shifted responsibility for adoption decisions from private agencies to public authorities. The reckoning has not been confined to countries of origin. In 2021, an official Dutch committee of inquiry reported that serious abuses had occurred in intercountry adoptions into the Netherlands over decades, and that the Dutch government had known of many of them and failed to act. The Dutch government suspended intercountry adoptions and in 2024 announced its intention to end them. Denmark's only authorized intercountry adoption agency announced in 2024 that it would wind down its operations. In 2022, several United Nations human rights bodies issued a joint statement calling on states to prevent and remedy illegal intercountry adoptions and to recognize that such adoptions can amount to serious human rights violations, including enforced disappearance where state officials were involved. The debate over what comes next These events have divided scholars and advocates into roughly two camps. One, associated with the Harvard law professor Elizabeth Bartholet, emphasizes the harms of institutional care. The Bucharest Early Intervention Project, a randomized study of young children in Romanian institutions led by Charles Nelson, Nathan Fox and Charles Zeanah, found that institutional rearing caused serious deficits in cognitive and emotional development and that placement in families could partially remedy them, especially if it occurred early. From this perspective, restrictions on intercountry adoption, however well motivated, leave children in institutions who could have had families, and the right response to abuses is better regulation, not closure. The other camp, associated with the law professor David Smolin, who coined the term "child laundering" to describe the way the adoption system can transform illicitly obtained children into legally adopted ones, emphasizes that the demand for young, healthy children in wealthy countries has repeatedly generated supply through illicit means, and that the Hague system has not reliably prevented it. From this perspective, the priority is to strengthen family preservation and domestic alternatives in countries of origin, to acknowledge and remedy past abuses, and to accept that intercountry adoption should be a narrow exception, used mainly for children with special needs or older children for whom no domestic family can be found. The trajectory of the law has favored the second camp. The Hague Conference has issued guidance on preventing and addressing illicit practices, receiving countries have tightened their procedures or withdrawn altogether, and countries of origin have used the Convention's subsidiarity principle to prioritize domestic placements. The children who are still adopted internationally are increasingly older, part of sibling groups or have medical needs. The era of intercountry adoption as a large-scale route to parenthood for childless couples in wealthy countries appears to be ending. One irony deserves mention. The United States is not only a receiving country; it is also a country of origin. Every year, American children, a disproportionate number of them Black infants, are adopted by families in Canada, the Netherlands and other countries. The United States, which for decades relied on other countries to certify that their children were lawfully available for adoption, is now itself certifying the adoptability of its own children for families abroad. The same Convention obligations apply. So does the question with which this book began: whether the severance on which the adoption rests was sound. Conclusion: Building Adoption on Sound Ground The law of child protection and adoption is often discussed as though it contained two separate subjects: a punitive or protective system that removes children from families, and a benevolent one that places children in new families. The preceding chapters have argued that this is one subject. Adoption, whether from an American foster care system or from an orphanage abroad, inherits everything that happened before it. When the earlier stages of the process are accurate and fair, adoption gives a child something of immense value, a family that is legally and permanently hers. When they are not, adoption converts a mistake or an injustice into a permanent fact, and the permanence that makes adoption good for children makes the error almost impossible to correct. Several conclusions follow from taking that seriously. Invest where the error begins The first is that the quality of the system depends overwhelmingly on its front end, and the law currently invests least there. The Supreme Court demands clear and convincing evidence at the termination stage, but the decisions that determine whether termination will ever be sought are made at the hotline, in the investigation and at the first hearing, on standards of reasonable suspicion, probable cause or a preponderance of the evidence, often without a lawyer in the room. Much of what enters the system as neglect is poverty, addiction or illness that could be addressed without removal. The reforms of the early 2020s, narrowing definitions of neglect, raising the standard for emergency removal and funding prevention services under the Family First Prevention Services Act, point in the right direction. They deserve to be evaluated rigorously and extended where they work. The single most effective front-end reform may be the least glamorous: giving parents competent lawyers from the first hearing onward. The evidence from interdisciplinary representation suggests that it shortens children's stays in foster care without compromising safety. It does so not by winning trials but by making the system do what it is already supposed to do, providing services, arranging visits, identifying relatives and making genuine reasonable efforts. The federal decision to open Title IV-E funds to legal representation made this financially possible for states. Whether states use the opportunity is now a question of political will. Reconsider the clock The second conclusion concerns time. The Adoption and Safe Families Act rightly recognized that children cannot wait indefinitely while adults struggle, and that foster care drift was a real harm. But the fifteen-of-twenty-two-month rule measures the wrong thing. It counts the time a child spends in care without asking why the time passed: whether the parent was failing, or whether the agency failed to provide treatment, housing, visits or transport. A clock that runs against parents regardless of the state's own delay rewards institutional failure. A better design would measure the state's performance as well as the parent's. It would treat unavailable services as a presumptive reason to extend the timeline, as New York's discretion law does for incarcerated parents and those in residential treatment, and it would require courts to make genuine rather than formulaic findings about reasonable efforts before a termination petition is filed. It would also take seriously the problem of legal orphans, recognizing that terminating parental rights without a realistic prospect of adoption does not create permanence and may destroy the only family ties a child has. The growth of reinstatement statutes and post-termination contact orders shows that states have already begun to reach this conclusion case by case. Accept that the state as parent must be accountable The third conclusion is that when the state takes custody of a child, it must be held to the standard it applies to parents. The constitutional duty recognized after DeShaney is real, and structural litigation has produced significant improvements in some systems. But the tools are slow, available only after failure is widespread and visible, and increasingly hemmed in by doctrines of abstention and limits on enforcing federal statutes. Children who spend nights in offices, who are moved through a dozen placements, or whose Social Security benefits are taken to pay for their own care are experiencing harms that would support a neglect finding against a private parent. Better accountability would include transparent public reporting of placement stability, maltreatment in care, psychotropic prescribing and outcomes for youth leaving care; independent oversight bodies with real investigative powers; and the conservation of children's own resources for their benefit. It would also recognize that the failures of the foster care system are not separate from termination and adoption. They shape the evidence against parents and the population of children who become available for adoption. Treat identity as part of welfare The fourth conclusion concerns the fights over identity in placement. The Indian Child Welfare Act embodies a judgment that the welfare of Native children cannot be separated from their relationship to their tribes and that the history of removal justifies special protection. The Supreme Court upheld the statute against most constitutional challenges in Haaland v. Brackeen, but the equal protection question remains open, and the best protection for Native families may now lie in state-law versions of ICWA and in the new federal data that will measure compliance. For other children, federal law forbids race-based placement but leaves questions of culture, religion, sexual orientation and gender identity to a shifting mixture of constitutional doctrine, executive policy and state law. These disputes will not be resolved soon. What can be said is that every placement embodies someone's view of what a child needs, and the law does better when it makes those judgments openly than when it hides them within an indeterminate best-interests standard. Adoption abroad: the lesson of the reckonings The final conclusion concerns intercountry adoption, where the argument of this book is most starkly confirmed. The Hague Convention's design is sound in principle: it places the responsibility for verifying that a child was lawfully available for adoption on the country of origin and requires the receiving country to cooperate. But the history of Cambodia, Guatemala, Ethiopia, China and South Korea shows that the design fails whenever money, demand and weak institutions combine. The reckonings now under way, from South Korea's truth commission to the Dutch inquiry, reveal harms that the law's recognition of those adoptions had rendered invisible for decades. What remains of intercountry adoption should be shaped by those lessons: strict subsidiarity, public rather than private control of adoptability decisions, full transparency of fees and records, genuine counseling for birth parents, and a presumption that adoptees have a right to their origins. For the United States, it should also include resolving the citizenship status of international adoptees who were brought here as children and never naturalized. A country that recognized their adoptions as creating permanent family bonds cannot coherently deport them as strangers. What remains unsettled Several questions will define the next decade. Whether the Supreme Court will take up the equal protection challenge to ICWA's placement preferences. Whether Congress will revise the ASFA timeline. Whether the prevention funding under Family First will grow into a genuine alternative to foster care or remain a modest supplement. Whether federal oversight of state systems will become more demanding under the scorecard approach announced in 2025, or whether federal policy will prioritize other goals. Whether the litigation over religious liberty and LGBTQ youth in foster care will produce a stable settlement. And whether intercountry adoption will survive as a narrow, carefully regulated practice or continue its decline toward disappearance. The answers will differ, but the test for each should be the same. Every legal change in this field ought to be judged by whether it makes the severance of families more accurate and more just, because every adoption rests on that severance. A child adopted after a fair process, in which the state removed her only because it had to, tried genuinely to bring her home, and ended her first family's legal ties only when that could not be done, can build a life on solid ground. A child adopted after a process that failed her first family inherits that failure, however loving her new family may be. The law's task is to ensure that the first kind of adoption is the only kind it produces. Notes Chapter 1 1. Meyer v. Nebraska, 262 U.S. 390 (1923); Pierce v. Society of Sisters, 268 U.S. 510 (1925). 1. Prince v. Massachusetts, 321 U.S. 158 (1944). 2. Parham v. J.R., 442 U.S. 584 (1979). 3. Troxel v. Granville, 530 U.S. 57 (2000) (plurality opinion). 4. Mahmoud v. Taylor, No. 24-297 (U.S. June 27, 2025); Wisconsin v. Yoder, 406 U.S. 205 (1972). 5. Stanley v. Illinois, 405 U.S. 645 (1972); Quilloin v. Walcott, 434 U.S. 246 (1978); Caban v. Mohammed, 441 U.S. 380 (1979); Lehr v. Robertson, 463 U.S. 248 (1983); Michael H. v. Gerald D., 491 U.S. 110 (1989). 6. Smith v. Organization of Foster Families for Equality and Reform, 431 U.S. 816 (1977). 7. Lassiter v. Department of Social Services, 452 U.S. 18 (1981); Mathews v. Eldridge, 424 U.S. 319 (1976). 8. Santosky v. Kramer, 455 U.S. 745 (1982). 9. M.L.B. v. S.L.J., 519 U.S. 102 (1996). 10. DeShaney v. Winnebago County Department of Social Services, 489 U.S. 189 (1989); Town of Castle Rock v. Gonzales, 545 U.S. 748 (2005). Chapter 2 1. C. Henry Kempe, Frederic N. Silverman, Brandt F. Steele, William Droegemueller and Henry K. Silver, "The Battered-Child Syndrome," Journal of the American Medical Association 181 (1962): 17–24. 1. Child Abuse Prevention and Treatment Act, Pub. L. No. 93-247 (1974), as amended, 42 U.S.C. § 5101 et seq. 2. U.S. Department of Health and Human Services, Administration for Children and Families, Children's Bureau, Child Maltreatment (annual reports). 3. Hyunil Kim, Christopher Wildeman, Melissa Jonson-Reid and Brett Drake, "Lifetime Prevalence of Investigating Child Maltreatment Among US Children," American Journal of Public Health 107, no. 2 (2017): 274–280. 4. N.Y. Family Court Act § 1012(f). 5. Texas H.B. 567, 87th Legislature (2021); Washington H.B. 1227, Keeping Families Together Act (2021). 6. Nicholson v. Williams, 203 F. Supp. 2d 153 (E.D.N.Y. 2002); Nicholson v. Scoppetta, 3 N.Y.3d 357 (2004). 7. Comprehensive Addiction and Recovery Act of 2016, Pub. L. No. 114-198; Ferguson v. City of Charleston, 532 U.S. 67 (2001). 8. Calabretta v. Floyd, 189 F.3d 808 (9th Cir. 1999). 9. Doe v. Heck, 327 F.3d 492 (7th Cir. 2003); Greene v. Camreta, 588 F.3d 1011 (9th Cir. 2009); Camreta v. Greene, 563 U.S. 692 (2011). 10. Tenenbaum v. Williams, 193 F.3d 581 (2d Cir. 1999); Wallis v. Spencer, 202 F.3d 1126 (9th Cir. 2000). 11. Josh Gupta-Kagan, "America's Hidden Foster Care System," Stanford Law Review 72 (2020): 841. 12. Dupuy v. Samuels, 397 F.3d 493 (7th Cir. 2005). 13. Valmonte v. Bane, 18 F.3d 992 (2d Cir. 1994); Los Angeles County v. Humphries, 562 U.S. 29 (2010). Chapter 3 1. Adoption Assistance and Child Welfare Act of 1980, Pub. L. No. 96-272. 1. Suter v. Artist M., 503 U.S. 347 (1992). 2. Adoption and Safe Families Act of 1997, Pub. L. No. 105-89; 42 U.S.C. § 675(5)(E). 3. Fostering Connections to Success and Increasing Adoptions Act of 2008, Pub. L. No. 110-351; Preventing Sex Trafficking and Strengthening Families Act, Pub. L. No. 113-183 (2014). 4. Family First Prevention Services Act, enacted as Division E, Title VII of the Bipartisan Budget Act of 2018, Pub. L. No. 115-123. 5. Administration for Children and Families, final rule, "Separate Licensing or Approval Standards for Relative or Kinship Foster Family Homes" (2023); final rule, "Foster Care Legal Representation" (2024). 6. Supporting America's Children and Families Act (enacted January 2025). 7. Lucas A. Gerber, Yuk C. Pang, Timothy Ross, Martin Guggenheim, Peter J. Pecora and Joel Miller, "Effects of an Interdisciplinary Approach to Parental Representation in Child Welfare," Children and Youth Services Review 102 (2019): 42–55. 8. Kenny A. v. Perdue, 356 F. Supp. 2d 1353 (N.D. Ga. 2005). 9. Children Act 1989, s. 31; Children and Families Act 2014, s. 14; Adoption and Children Act 2002, s. 52; In re B (Care Proceedings: Appeal) [2013] UKSC 33; Re B-S (Children) [2013] EWCA Civ 1146. Chapter 4 1. Robert H. Mnookin, "Child-Custody Adjudication: Judicial Functions in the Face of Indeterminacy," Law and Contemporary Problems 39 (1975): 226. 1. Joseph Goldstein, Anna Freud and Albert J. Solnit, Beyond the Best Interests of the Child (New York: Free Press, 1973). 2. N.J. Stat. Ann. § 30:4C-15.1, as amended in 2021. 3. In re Mason, 486 Mich. 142 (2010). 4. New York ASFA Expanded Discretion Act (2010), amending N.Y. Social Services Law § 384-b. 5. U.S. Department of Justice and U.S. Department of Health and Human Services, letter of findings to the Massachusetts Department of Children and Families (January 2015); HHS, final rule, "Nondiscrimination on the Basis of Disability in Programs or Activities Receiving Federal Financial Assistance" (May 2024), 45 C.F.R. Part 84. 6. Strand Lobben and Others v. Norway, App. No. 37283/13 (European Court of Human Rights, Grand Chamber, 10 September 2019). Chapter 5 1. Indian Child Welfare Act of 1978, 25 U.S.C. §§ 1901–1963. 1. U.S. Department of the Interior, Federal Indian Boarding School Initiative Investigative Report, vol. 1 (2022) and vol. 2 (2024). 2. Mississippi Band of Choctaw Indians v. Holyfield, 490 U.S. 30 (1989). 3. Adoptive Couple v. Baby Girl, 570 U.S. 637 (2013). 4. Haaland v. Brackeen, 599 U.S. 255 (2023); Morton v. Mancari, 417 U.S. 535 (1974). 5. Bureau of Indian Affairs, Indian Child Welfare Act Proceedings, 25 C.F.R. Part 23 (2016); Administration for Children and Families, Adoption and Foster Care Analysis and Reporting System final rule (December 2024). 6. An Act respecting First Nations, Inuit and Métis children, youth and families, S.C. 2019, c. 24; Reference re An Act respecting First Nations, Inuit and Métis children, youth and families, 2024 SCC 5. 7. Human Rights and Equal Opportunity Commission, Bringing Them Home: Report of the National Inquiry into the Separation of Aboriginal and Torres Strait Islander Children from Their Families (1997). 8. Multiethnic Placement Act of 1994, Pub. L. No. 103-382; Interethnic Adoption Provisions, Small Business Job Protection Act of 1996, Pub. L. No. 104-188, § 1808. 9. Palmore v. Sidoti, 466 U.S. 429 (1984). 10. Fulton v. City of Philadelphia, 593 U.S. 522 (2021). 11. Administration for Children and Families, final rule, "Designated Placement Requirements Under Titles IV-E and IV-B for LGBTQI+ Children," Federal Register (April 30, 2024); proposed rescission (March 6, 2026); final rescission (July 14, 2026). 12. Executive Order 14359, "Fostering the Future for American Children and Families" (November 13, 2025). Chapter 6 1. U.S. Government Accountability Office, Foster Children: HHS Guidance Could Help States Improve Oversight of Psychotropic Prescriptions, GAO-12-201 (December 2011). 1. Washington State Department of Social and Health Services v. Guardianship Estate of Keffeler, 537 U.S. 371 (2003); Daniel L. Hatcher, The Poverty Industry: The Exploitation of America's Most Vulnerable Citizens (New York: NYU Press, 2016). 2. Mark E. Courtney and colleagues, Midwest Evaluation of the Adult Functioning of Former Foster Youth (Chapin Hall at the University of Chicago, multiple waves, 2004–2011). 3. Youngberg v. Romeo, 457 U.S. 307 (1982). 4. Doe v. New York City Department of Social Services, 649 F.2d 134 (2d Cir. 1981); Taylor v. Ledbetter, 818 F.2d 791 (11th Cir. 1987) (en banc); K.H. ex rel. Murphy v. Morgan, 914 F.2d 846 (7th Cir. 1990). 5. LaShawn A. v. Dixon, 762 F. Supp. 959 (D.D.C. 1991), later captioned LaShawn A. v. Barry. 6. Braam ex rel. Braam v. State, 150 Wash. 2d 689 (2003). 7. Perdue v. Kenny A., 559 U.S. 542 (2010). 8. M.D. v. Perry, 152 F. Supp. 3d 684 (S.D. Tex. 2015); M.D. ex rel. Stukenberg v. Abbott, 907 F.3d 237 (5th Cir. 2018). 9. Horne v. Flores, 557 U.S. 433 (2009). 10. Younger v. Harris, 401 U.S. 37 (1971); 31 Foster Children v. Bush, 329 F.3d 1255 (11th Cir. 2003); Jonathan R. v. Justice, 41 F.4th 316 (4th Cir. 2022); Sprint Communications, Inc. v. Jacobs, 571 U.S. 69 (2013). 11. Gonzaga University v. Doe, 536 U.S. 273 (2002); Health and Hospital Corporation of Marion County v. Talevski, 599 U.S. 166 (2023); Medina v. Planned Parenthood South Atlantic, No. 23-1275 (U.S. June 26, 2025). 12. California State Foster Parent Association v. Wagner, 624 F.3d 974 (9th Cir. 2010); D.O. v. Glisson, 847 F.3d 374 (6th Cir. 2017); Midwest Foster Care and Adoption Association v. Kincade, 712 F.3d 1190 (8th Cir. 2013). Chapter 7 1. Massachusetts Adoption of Children Act, 1851 Mass. Acts ch. 324. 1. Ann Fessler, The Girls Who Went Away: The Hidden History of Women Who Surrendered Children for Adoption in the Decades Before Roe v. Wade (New York: Penguin Press, 2006). 2. In re Clausen, 442 Mich. 648 (1993); In re Petition of Doe, 159 Ill. 2d 347 (1994). 3. Florida Department of Children and Families v. Adoption of X.X.G., 45 So. 3d 79 (Fla. Dist. Ct. App. 2010); Campaign for Southern Equality v. Mississippi Department of Human Services, 175 F. Supp. 3d 691 (S.D. Miss. 2016). 4. Obergefell v. Hodges, 576 U.S. 644 (2015); V.L. v. E.L., 577 U.S. 404 (2016); Pavan v. Smith, 582 U.S. 563 (2017). 5. Megan Twohey, "The Child Exchange: Inside America's Underground Market for Adopted Children," Reuters (September 2013). 6. Burr v. Board of County Commissioners of Stark County, 23 Ohio St. 3d 69 (1986). 7. Dobbs v. Jackson Women's Health Organization, 597 U.S. 215 (2022). Chapter 8 1. United Nations Convention on the Rights of the Child (1989), art. 21. 1. Hague Convention of 29 May 1993 on Protection of Children and Co-operation in Respect of Intercountry Adoption; Hague Conference on Private International Law, status table for the Convention. 2. Intercountry Adoption Act of 2000, Pub. L. No. 106-279; Intercountry Adoption Universal Accreditation Act of 2012, Pub. L. No. 112-276; 22 C.F.R. Part 96. 3. Immigration and Nationality Act § 101(b)(1)(F)–(G), 8 U.S.C. § 1101(b)(1)(F)–(G). 4. Child Citizenship Act of 2000, Pub. L. No. 106-395. 5. U.S. Department of State, Bureau of Consular Affairs, annual reports on intercountry adoption. 6. Barbara Demick, "The End of Adoptions from China," The New Yorker (September 2024). 7. Committee Investigating Intercountry Adoption (Netherlands), report (February 2021). 8. David M. Smolin, "Child Laundering: How the Intercountry Adoption System Legitimizes and Incentivizes the Practices of Buying, Trafficking, Kidnapping, and Stealing Children," Wayne Law Review 52 (2006): 113. 9. Elizabeth Bartholet, "International Adoption: Thoughts on the Human Rights Issues," Buffalo Human Rights Law Review 13 (2007): 151. 10. Charles A. Nelson, Nathan A. Fox and Charles H. Zeanah, Romania's Abandoned Children: Deprivation, Brain Development, and the Struggle for Recovery (Cambridge, MA: Harvard University Press, 2014). Further Reading Dorothy Roberts, Shattered Bonds: The Color of Child Welfare (Basic Books, 2002), and Torn Apart: How the Child Welfare System Destroys Black Families and How Abolition Can Build a Safer World (Basic Books, 2022).** The most influential critiques of the system's racial disparities, the second arguing for dismantling rather than reforming it. Elizabeth Bartholet, Nobody's Children: Abuse and Neglect, Foster Drift, and the Adoption Alternative (Beacon Press, 1999).** The leading argument for prioritizing children's safety and permanency, including through adoption, over family preservation. A necessary counterweight to Roberts. Martin Guggenheim, What's Wrong with Children's Rights (Harvard University Press, 2005).** A sharp examination of how the language of children's rights is used to expand state power over families, by a leading parents' advocate. Josh Gupta-Kagan, "America's Hidden Foster Care System," Stanford Law Review 72 (2020).** The essential account of informal safety plans and the separations that occur outside the formal system. Nina Bernstein, The Lost Children of Wilder: The Epic Struggle to Change Foster Care (Pantheon, 2001).** A narrative history of the decades-long New York litigation over religious foster care agencies, and a vivid picture of structural reform litigation. Daniel L. Hatcher, The Poverty Industry: The Exploitation of America's Most Vulnerable Citizens (NYU Press, 2016).** Documents how states and contractors capture funds intended for vulnerable people, including foster children's Social Security benefits. Laura Briggs, Somebody's Children: The Politics of Transracial and Transnational Adoption (Duke University Press, 2012).** A history of how race, poverty and international politics shaped the movement of children into adoptive families. Kathryn Joyce, The Child Catchers: Rescue, Trafficking, and the New Gospel of Adoption (PublicAffairs, 2013).** Investigative reporting on the evangelical adoption movement and the scandals in Ethiopia, Liberia and elsewhere. Eleana J. Kim, Adopted Territory: Transnational Korean Adoptees and the Politics of Belonging (Duke University Press, 2010).** An anthropological study of Korean adoptees and their return to Korea, essential background to the recent truth commission findings. Charles A. Nelson, Nathan A. Fox and Charles H. Zeanah, Romania's Abandoned Children (Harvard University Press, 2014).** The findings of the Bucharest Early Intervention Project on the effects of institutional care, central to the debate over intercountry adoption. David M. Smolin, "Child Laundering," Wayne Law Review 52 (2006).** The classic legal analysis of how illicitly obtained children are converted into legally adopted ones. Hague Conference on Private International Law, The Implementation and Operation of the 1993 Hague Intercountry Adoption Convention: Guide to Good Practice No. 1 (2008).** The official guide to how the Convention is meant to work, freely available from the Hague Conference. Child Welfare Information Gateway (childwelfare.gov). A federal clearinghouse offering plain-language summaries of state laws on reporting, grounds for termination, consent to adoption, access to adoption records and more. The best starting point for any state-specific question. Hashtags: #ChildProtectionLaw #AdoptionLaw #ChildWelfareLaw #ChildrensRights #FamilyLaw #ChildProtection #ChildWelfare #ChildSafety #ChildAbusePrevention #ChildNeglect #ChildProtectiveServices #CPS #StateIntervention #ParentalRights #ConstitutionalRights #DueProcess #FamilyPreservation #FamilyReunification #FosterCare #FosterCareSystem #KinshipCare #KinshipPlacement #DependencyCourt #JuvenileCourt #MandatoryReporting #EmergencyRemoval #ReasonableEfforts #PermanencyPlanning #TerminationOfParentalRights #BestInterestsOfTheChild #AdoptionAndSafeFamiliesAct #ASFA #CAPTA #FamilyFirstPreventionServicesAct #IndianChildWelfareAct #ICWA #TribalSovereignty #DomesticAdoption #InternationalAdoption #IntercountryAdoption #HagueAdoptionConvention #HagueConvention #AdoptionEthics #AdoptionConsent #AdoptionFraud #AdoptionSafeguards #AdoptionAssistance #FosterCareReform #ChildWelfareReform #LegalPermanency
- Distributed Model Training (Tensor, Pipeline, and Sequence Parallelism)
Download the Book (PDF): Introduction A modern frontier language model is trained on a machine that does not exist as a single object. It exists as a schedule. Thousands of accelerators, each with its own small pool of fast memory, sit in racks connected by copper inside a server and by optical fibre between servers. None of them can hold the model. None of them could finish the computation alone in any useful span of time. The thing we call "training the model" is a choreography in which each device holds a fragment of the parameters, a fragment of the activations, and a fragment of the optimizer's bookkeeping, and in which those fragments are exchanged, summed, and re-assembled millions of times over a period of weeks. When Meta trained the 405-billion-parameter Llama 3 model in 2024, it ran on up to 16,384 NVIDIA H100 GPUs at once. Each GPU had 80 gigabytes of memory. The model's weights alone, stored in 16-bit precision, occupy about 810 gigabytes, ten times what any single GPU can hold, before a single byte has been spent on gradients, optimizer state, or the intermediate values that backpropagation needs. The job was laid out as eight-way tensor parallelism inside each server, sixteen-way pipeline parallelism across servers, and data parallelism across the remaining dimension, with context parallelism added for the long-sequence phase. That configuration was not an accident of preference. It was the answer to a set of constraints that can be stated precisely, and the reasoning behind it generalises to almost every large training run. The controlling idea This book argues one thing and teaches it from several sides: distributed training is the problem of deciding where every tensor lives and what it costs to move it, and a good parallel layout is one that matches each kind of communication to the tier of the network fast enough to carry it. Everything else follows from that sentence. There are only a handful of things to split: the batch of training examples, the weight matrices, the sequence of tokens, the layers of the network, and, in mixture-of-experts models, the experts themselves. Each split frees memory or adds compute in a particular way, and each imposes a particular communication pattern with a particular volume and a particular sensitivity to latency. Splitting a weight matrix across devices requires those devices to exchange partial results inside every layer, twice in the forward pass and twice in the backward pass, which only works over the fastest links available. Splitting the network into sequential stages requires devices to pass only the activations at the boundary between stages, a small and infrequent message that tolerates slower links, but it creates idle time that must be engineered away. Splitting the batch requires one large exchange of gradients per step, which can be overlapped with computation and so tolerates the slowest links of all. The hardware, meanwhile, offers a steep hierarchy. Inside a GPU, memory bandwidth is measured in terabytes per second. Between GPUs in the same server, connected by NVLink and NVSwitch, bandwidth is in the hundreds of gigabytes per second per GPU. Between servers, over InfiniBand or RDMA-capable Ethernet, a single network card delivers on the order of fifty gigabytes per second in each direction, and the path may cross several switches. Each step down that hierarchy costs roughly an order of magnitude in bandwidth and adds latency. The art of the parallel layout is to put the chattiest split on the fastest tier and the quietest split on the slowest, and to arrange the work so that the communication that cannot be avoided is hidden behind computation that must happen anyway. Once you see the problem this way, the alphabet soup of the field resolves into a small number of moves. ZeRO and fully sharded data parallelism are ways of removing redundant copies of state at the price of more communication along the data-parallel dimension. Tensor parallelism and sequence parallelism are two halves of one technique for splitting a transformer layer across a tightly coupled group of devices. Context parallelism extends the split to the attention computation over very long sequences. Pipeline parallelism trades a fixed amount of idle time, the bubble, for very cheap communication. Expert parallelism adds an all-to-all exchange that is unlike any of the others. The "3D" or "4D" layout is simply the composition of these moves, ordered by how much bandwidth each one consumes. Who this book is for The book is written for engineers, researchers, technical managers, and serious students who understand roughly how a neural network is trained, what a gradient is, what a matrix multiplication does, and what a GPU is for, but who have not had to make a model fit on a cluster. It assumes no prior knowledge of collective communication, of the memory accounting of mixed-precision training, or of any particular framework. Every term is defined where it first appears and collected in a glossary at the end. It is equally written for the practitioner who has run a training job with a framework's default settings and wants to know why the settings are what they are, why throughput collapses when a configuration is changed, or why a job that ran at forty per cent of peak on 512 GPUs runs at twenty-five per cent on 4,096. The quantities involved are not mysterious. A reader who can multiply the number of parameters by the number of bytes per parameter, and divide a message size by a link bandwidth, can predict most of what matters about a distributed training job before it is launched. The examples are drawn mostly from transformer language models, because that is where the techniques were developed and refined and where the scale is greatest. The ideas apply with modest changes to vision transformers, diffusion models, recommendation models with large embedding tables, and multimodal systems. Where a technique is specific to transformers, the text says so. How the argument unfolds The first two chapters establish the two quantities that govern everything else: memory and communication. Chapter 1 accounts for every byte that a training step requires on a GPU, from the parameters and gradients through the optimizer's moment estimates to the activations stored for the backward pass, and shows why a model with a few billion parameters already overflows a single device. Chapter 2 introduces the collective communication primitives (all-reduce, reduce-scatter, all-gather, broadcast, and all-to-all), the algorithms that implement them, a simple cost model that predicts their running time, and the physical hierarchy of links over which they run. Chapters 3 and 4 treat the batch dimension. Data parallelism is the oldest and simplest strategy and still the outermost layer of nearly every large job; Chapter 3 explains how it works, how its gradient exchange is overlapped with computation, and why it eventually runs out of room, both because every device must hold a full copy of the model and because the global batch cannot grow without limit. Chapter 4 covers the family of techniques, ZeRO in its three stages and PyTorch's Fully Sharded Data Parallel, that remove the redundant copies and turn data parallelism into a memory-scaling technique. Chapters 5 and 6 split the layer itself. Chapter 5 derives tensor parallelism for the transformer's attention and feed-forward blocks, counts its communication, and explains why it almost never crosses a server boundary. Chapter 6 treats the sequence dimension: first the sequence parallelism that complements tensor parallelism by sharding the parts of the layer it leaves replicated, then the context parallelism, in its ring and all-to-all forms, that makes training on sequences of hundreds of thousands of tokens possible. Chapter 7 splits the model by depth. Pipeline parallelism is the cheapest strategy in communication and the most intricate in scheduling, and the chapter follows the line of schedules from GPipe through one-forward-one-backward, interleaved virtual stages, and the zero-bubble schedules that split the backward pass in two. Chapter 8 introduces the fifth axis, expert parallelism for mixture-of-experts models, whose all-to-all communication has reshaped how the largest open models are trained. Chapter 9 assembles the pieces. It sets out the rules for ordering the parallel dimensions onto the hardware hierarchy, works through the planning of a concrete layout step by step, and examines the configurations of several real training runs to show the rules at work. Chapter 10 turns from design to operation: what goes wrong when a job runs on thousands of GPUs for months, how checkpointing and failure recovery are engineered, how to measure efficiency honestly, and how to find the bottleneck when the numbers disappoint. The conclusion draws out what follows from the whole argument for the person who must plan, buy, or debug a large training system. A note on numbers Distributed training is a quantitative discipline, and this book uses numbers freely. Where a figure comes from a published paper or technical report, the source is named in the text and collected in the further reading list. Where a figure is the result of arithmetic in the text, the arithmetic is shown. Hardware specifications change quickly, and the specific bandwidths and capacities quoted for current accelerators will date; the ratios between tiers have been far more stable than the absolute values, and it is the ratios that drive the design decisions. A reader who replaces every hardware figure in this book with that of next year's machine will find that the reasoning still holds, and that the answers shift in predictable directions. Chapter 1. What Lives on a GPU: The Arithmetic of Training State There are two reasons to spread a training job across many devices, and they are worth keeping apart because they lead to different techniques. The first is compute: the job requires more floating-point operations than one device can perform in an acceptable time. The second is memory: the job requires more bytes of state than one device can hold. Data parallelism, in its simple form, addresses only the first. Almost every other technique in this book exists because of the second. Before any of them can be understood, it is necessary to know exactly what occupies a GPU's memory during training, how much of it there is, and which parts grow with which quantities. The compute wall Start with compute, because the arithmetic is short. For a dense transformer, a widely used approximation, introduced in the scaling-law work of Kaplan and colleagues at OpenAI in 2020, is that training costs about six floating-point operations per parameter per token: two for the forward pass (one multiply and one add for each weight applied to each token) and four for the backward pass, which must compute gradients with respect to both the activations and the weights. The approximation ignores the attention computation's quadratic term, which matters for long sequences, but it is accurate enough for planning. Take a 70-billion-parameter model trained on 15 trillion tokens, a scale typical of open models in 2024 and 2025. The total is about 6 × 70 × 10⁹ × 15 × 10¹² = 6.3 × 10²⁴ operations. An H100 GPU has a peak dense throughput of roughly 989 teraflops in 16-bit brain floating point (BF16), and a well-tuned training job sustains around 40 per cent of that, so call it 400 teraflops. At that rate, one GPU would need about 1.6 × 10¹⁰ seconds, which is roughly five hundred years. Ten thousand GPUs, if they could be kept equally busy, would need about eighteen days. That last condition, "if they could be kept equally busy", is the whole engineering problem. Dividing the work is easy in principle; doing so without the devices spending most of their time waiting for one another is not. But notice that the compute argument alone says nothing about how the model must be split. If each GPU could hold the entire model and its training state, one could simply give each GPU a different slice of the data and average the gradients. The memory wall is what forces the model itself to be divided. Model state: sixteen bytes per parameter The standard recipe for training large models is mixed-precision training with the Adam optimizer, or its weight-decay variant AdamW. Mixed precision, described by Micikevicius and colleagues at NVIDIA and Baidu in 2017, performs the forward and backward computations in a 16-bit format, originally IEEE half precision (FP16) and now usually BF16, while keeping a master copy of the weights in 32-bit single precision (FP32) so that small updates are not lost to rounding. Adam keeps two additional quantities for every parameter: a running average of the gradient (the first moment, or momentum) and a running average of the squared gradient (the second moment, or variance). These are kept in FP32. Counting the bytes per parameter gives the accounting that the ZeRO paper of Rajbhandari and colleagues at Microsoft made standard in 2019, summarised in Table 1. The 16-bit working copy of the parameters costs 2 bytes. The 16-bit gradients cost 2 bytes. The FP32 master weights cost 4 bytes, the FP32 momentum 4 bytes, and the FP32 variance 4 bytes. The total is 16 bytes per parameter, of which 12 bytes, three-quarters of the whole, belong to the optimizer. Table 1. Memory for model state in mixed-precision training with Adam, per parameter and for a 70-billion-parameter model (accounting follows Rajbhandari et al., 2019). Component Precision Bytes per parameter 70B model Working parameters BF16 or FP16 2 140 GB Gradients BF16 or FP16 2 140 GB Master parameters FP32 4 280 GB Adam first moment FP32 4 280 GB Adam second moment FP32 4 280 GB Total model state 16 1,120 GB The consequence is stark. A model of 5 billion parameters needs 80 gigabytes of model state, the entire memory of an 80-gigabyte H100, with nothing left over for activations, temporary buffers, or the memory the framework and communication library reserve for themselves. In practice the largest model one can train with plain Adam on a single 80-gigabyte device, using no memory-saving techniques, is in the low single-digit billions of parameters. The ZeRO paper's running example was a 7.5-billion-parameter model, which needs 120 gigabytes of model state, already beyond any single GPU of its time. Variations on the recipe shift the numbers without changing the conclusion. Some implementations keep gradients in FP32 for accumulation, adding two bytes. Some optimizers keep less state: Adafactor factorises the second moment, and 8-bit optimizers quantise the moments, cutting the optimizer's share substantially. Training with FP8 arithmetic for the matrix multiplications, as DeepSeek did for its V3 model and as NVIDIA's Transformer Engine supports on Hopper and Blackwell GPUs, reduces the cost of activations and of some communication, but the master weights and optimizer moments generally remain in higher precision because they must accumulate tiny increments faithfully. The sixteen-byte figure is a good default for planning, and the important structural fact survives every variation: optimizer state is the largest single block, and it is touched only once per step, at the very end. That second observation is the seed of ZeRO. The optimizer state is needed only during the update, and each parameter's update depends only on that parameter's own gradient and moments. There is no reason, in principle, for every device to hold the optimizer state for every parameter. Chapter 4 develops this into a family of techniques. Activations: the state that grows with the batch Model state depends only on the number of parameters. The second great consumer of memory depends on how much data passes through the model at once. During the forward pass, each layer produces outputs that are needed again during the backward pass to compute gradients. These stored intermediate values are the activations, and for large transformers trained on long sequences they frequently exceed the model state. The most careful public accounting is due to Korthikanti and colleagues at NVIDIA, whose 2022 paper on reducing activation recomputation derived the activation memory of a standard transformer layer. Using s for the sequence length, b for the micro-batch size (the number of sequences processed together), h for the hidden dimension, and a for the number of attention heads, they found that one layer stores approximately s·b·h·(34 + 5·a·s/h) bytes in 16-bit training without any memory-saving tricks. The two terms behave very differently. The first, 34·s·b·h, comes from the inputs to the various linear layers, the layer normalisations, the GELU activation, and the dropout masks; it grows linearly with sequence length. The second, 5·a·s²·b, comes from the attention mechanism: the matrix of attention scores, its softmax, and the dropout mask applied to it each have one entry for every pair of positions in the sequence, for every head. It grows with the square of the sequence length. To see the magnitudes, take a layer shaped like GPT-3's largest model: hidden dimension 12,288, 96 attention heads, sequence length 2,048, and a micro-batch of a single sequence. The product s·b·h is about 25 million. The linear term is 34 times that, about 0.86 gigabytes. The ratio 5·a·s/h is 5 × 96 × 2,048 / 12,288 = 80, so the quadratic term is about 2.0 gigabytes. One layer therefore stores roughly 2.9 gigabytes of activations for a single 2,048-token sequence, and the 96-layer model stores about 275 gigabytes. That is more than the full model state of a 17-billion-parameter model, and it is for one sequence. Two developments since GPT-3 changed this picture in important ways. The first is FlashAttention, introduced by Dao and colleagues in 2022 and now the default attention implementation almost everywhere. FlashAttention computes attention in tiles that fit in the GPU's on-chip memory and never materialises the full score matrix in high-bandwidth memory; during the backward pass it recomputes the scores from the stored queries, keys, and values. This removes most of the quadratic term from activation memory, leaving activation memory roughly linear in sequence length. The quadratic cost does not disappear from the arithmetic, since attention still performs work proportional to s², but it no longer dominates storage. The second development is the steady growth of context length. Models trained in 2020 used sequences of 2,048 tokens; models trained in 2024 and 2025 routinely include phases at 32,000, 128,000 or more. Even with the quadratic term removed, the linear term at s = 131,072 is sixty-four times what it was at 2,048. Long-context training is therefore an activation-memory problem first, and Chapter 6 is devoted to it. Trading compute for memory: activation recomputation The oldest remedy for activation memory is not to store activations at all. Activation recomputation, also called gradient checkpointing, was described for neural networks by Chen and colleagues in 2016. The forward pass stores only a few checkpoints, typically the input to each transformer layer, and discards everything else. During the backward pass, when a layer's gradients are needed, the layer's forward computation is run again from its stored input to regenerate the intermediates. Full recomputation of every layer reduces activation memory to roughly 2·s·b·h bytes per layer, the size of the layer's input, which is tiny. The cost is an extra forward pass for the whole network, which raises the total compute per step from about three forward-pass equivalents (one forward, two backward) to about four, a 33 per cent increase. For many years this was simply accepted as the price of training large models. The Korthikanti paper's contribution was to observe that the cost and the benefit are unevenly distributed within a layer. The attention scores, softmax outputs, and attention dropout masks, the quadratic term, are large in memory but cheap to recompute, because the operations that produce them are elementwise or are small batched matrix multiplications compared with the big linear layers. The linear term's activations are small per byte of compute saved. So they proposed selective activation recomputation: store everything except the attention core, and recompute only that. In their experiments this cut activation memory by roughly a factor of five while adding only a few per cent of compute, compared with the 30 to 40 per cent overhead of full recomputation. FlashAttention, by design, is a form of selective recomputation applied to exactly that part of the layer. Modern frameworks offer a range of granularities: full recomputation, selective recomputation of attention, recomputation of only the activation function or normalisation layers, or offloading activations to host memory over the PCIe bus. Choosing among them is part of the layout problem, because every byte of activation memory saved is a byte that can be spent on a larger micro-batch, a smaller degree of model parallelism, or fewer pipeline stages, and each of those choices changes the communication pattern. Everything else on the device Model state and activations are the two large blocks, but planning that counts only them will overestimate what fits. Several other consumers take their share. Temporary buffers are allocated during computation. A matrix multiplication may need workspace; a fused kernel may need scratch space; the cross-entropy loss over a vocabulary of 128,000 or more tokens produces a logits tensor of size s·b·V, which for a large vocabulary and long sequence can run to several gigabytes on its own and is often computed in chunks for that reason. Communication buffers are allocated by the collective communication library. NVIDIA's NCCL reserves memory for each communicator, and frameworks that overlap communication with computation keep gradient buckets or prefetched parameter shards in flight, which means holding extra copies for a while. Fragmentation wastes memory. The PyTorch caching allocator requests large blocks from the device and sub-allocates them; when tensors of many different sizes are freed and reallocated in irregular patterns, free memory becomes scattered in pieces too small to use. Large jobs routinely lose several gigabytes this way, and out-of-memory errors that occur with gigabytes nominally free are usually fragmentation. Setting the allocator to use expandable segments, a newer PyTorch option, mitigates this considerably. Finally, the runtime itself, the CUDA context and the libraries' kernels, occupies some hundreds of megabytes to a gigabyte or more before the model is even loaded. A rule of thumb among practitioners is to plan for no more than about 85 to 90 per cent of nominal device memory to be usable for model state and activations combined, and to verify with a short run before committing to a configuration. What the accounting tells us Laying out the memory budget this way reveals the structure of the problem that the rest of the book solves. There are four large blocks: parameters, gradients, optimizer state, and activations. Each can be divided among devices in a different way. The optimizer state, gradients, and parameters can be divided among the devices of a data-parallel group, at the price of extra collective communication of the parameters and gradients. This is ZeRO and fully sharded data parallelism, and it scales memory almost perfectly with the number of devices while keeping the computation unchanged. The parameters, and with them the gradients and optimizer state, can also be divided by splitting each weight matrix, which also divides the computation and some of the activations. This is tensor parallelism. It requires communication inside every layer. The activations can be divided along the sequence dimension, which is sequence parallelism and context parallelism. This requires communication to reassemble or exchange the parts of the sequence that each layer needs. The layers themselves can be divided among devices, each holding a contiguous slice of the network with its parameters, gradients, optimizer state, and the activations of the micro-batches currently in flight. This is pipeline parallelism, and it requires only point-to-point communication at stage boundaries. None of these is free, and the costs are almost entirely costs of communication: how many bytes must move, between which devices, how often, and whether the movement can be overlapped with useful arithmetic. To compare them, we need a vocabulary and a cost model for communication itself. That is the subject of the next chapter. Chapter 2. Moving Bytes: Collective Communication and the Bandwidth Hierarchy Every parallel training technique can be described as a pattern of communication. Data parallelism sums gradients across replicas. Fully sharded data parallelism gathers parameters before they are used and scatters gradients after they are computed. Tensor parallelism sums partial results inside each layer. Pipeline parallelism passes activations from one stage to the next. Expert parallelism sends each token to the device that holds its chosen expert and brings the result back. To reason about any of them, one needs a small vocabulary of communication operations, an understanding of how those operations are implemented, and a model of how long they take on real hardware. This chapter supplies all three. The collective primitives Most communication in distributed training is collective: a group of devices, called ranks, all participate in a single coordinated operation. The vocabulary comes from the Message Passing Interface (MPI) standard, developed for scientific supercomputing in the 1990s, and has been adopted wholesale by the libraries used for deep learning, most importantly NVIDIA's Collective Communications Library (NCCL), AMD's RCCL, and the communication backends of PyTorch, JAX, and their frameworks. Imagine N ranks, each holding a buffer of S bytes. The five operations that matter most are these. Broadcast. One rank, the root, holds a buffer; after the operation, every rank holds a copy. Broadcast is used at initialisation to make sure all replicas start from the same weights. Reduce. Every rank holds a buffer of the same shape; after the operation, the root holds the elementwise sum (or another reduction such as the maximum) of all of them. Reduce on its own is rare in training, but it is a building block. All-reduce. Every rank holds a buffer; after the operation, every rank holds the elementwise sum of all buffers. This is the workhorse of data parallelism: each replica computes gradients on its own data, and all-reduce gives every replica the summed gradient so that all can apply the same update. It is also the operation at the heart of tensor parallelism. Reduce-scatter. Every rank holds a buffer of S bytes, logically divided into N chunks. After the operation, rank i holds only the i-th chunk of the sum. Each rank ends up with a fully reduced shard of size S/N. All-gather. Every rank holds a shard of S/N bytes; after the operation, every rank holds the concatenation of all shards, the full S bytes. The relationship among the last three is the single most useful fact in this chapter: an all-reduce is exactly a reduce-scatter followed by an all-gather. After the reduce-scatter, each rank holds one fully summed shard; after the all-gather, every rank holds every summed shard. This decomposition is not just a mathematical identity. It is how the most efficient all-reduce algorithms work, and it is the insight that ZeRO, fully sharded data parallelism, and Megatron's sequence parallelism all exploit. If some ranks only need part of the result, or need it at different times, one can perform the two halves separately, at different points in the computation, and pay for each half only when it is needed. All-to-all. Every rank holds N chunks, one destined for each rank; after the operation, rank i holds the i-th chunk from every rank. It is a transpose of data across the group. All-to-all is the communication pattern of expert parallelism, where each device holds tokens destined for experts on every other device, and of the Ulysses form of context parallelism, where data distributed by sequence position must be redistributed by attention head. Finally, there is plain point-to-point communication: one rank sends a buffer, another receives it. Pipeline parallelism uses this to move activations forward and gradients backward between adjacent stages. Table 2 summarises the operations, the bytes each rank must send under the standard bandwidth-optimal algorithms described below, and where each appears in training. Table 2. Collective operations for N ranks and a full buffer of S bytes: per-rank data sent under bandwidth-optimal algorithms, and principal uses in training. Operation Each rank starts with Each rank ends with Bytes sent per rank Main use in training All-reduce Full buffer Full sum 2(N−1)/N × S Data-parallel gradients; tensor parallelism Reduce-scatter Full buffer One summed shard (N−1)/N × S Sharded gradients (ZeRO, FSDP); sequence parallelism All-gather One shard Full buffer (N−1)/N × S Sharded parameters (ZeRO-3, FSDP); sequence parallelism All-to-all N chunks N chunks, transposed (N−1)/N × S Expert parallelism; Ulysses context parallelism Point-to-point Buffer (sender) Buffer (receiver) S (sender only) Pipeline stage boundaries; ring attention How all-reduce is implemented The naive implementation of all-reduce sends every rank's buffer to one root, sums them there, and broadcasts the result. The root must receive (N−1)·S bytes and send (N−1)·S bytes, so its link becomes a bottleneck that grows linearly with the number of ranks. For large N this is hopeless. The ring algorithm removes the bottleneck. Arrange the N ranks in a logical ring, each sending to its successor and receiving from its predecessor. Divide each rank's buffer into N chunks. In the reduce-scatter phase, which takes N−1 steps, each rank sends one chunk to its neighbour, receives a chunk from its other neighbour, adds the received chunk to its own corresponding chunk, and passes the partial sum along in the next step. After N−1 steps, each rank holds one chunk that contains the complete sum over all ranks. In the all-gather phase, another N−1 steps, the completed chunks are passed around the ring until every rank has all of them. In each of the 2(N−1) steps, every rank sends and receives one chunk of S/N bytes, simultaneously and on different links. The total each rank sends is 2(N−1)/N × S, which approaches 2S as N grows. The remarkable property is that this is independent of N: doubling the number of ranks does not increase the bytes each rank sends. Patarasuk and Yuan showed in 2009 that this is optimal in bandwidth for all-reduce; no algorithm can have every rank send less than about 2S bytes in the general case. The ring algorithm was brought into deep learning by Baidu's research group in 2017 and popularised by Uber's Horovod library the same year, and it remains the basis of large-message all-reduce. The ring's weakness is latency. It takes 2(N−1) sequential steps, and each step incurs a fixed start-up cost regardless of message size. For small messages, or for very large N, the steps dominate. Tree algorithms address this by organising ranks into a tree, reducing up toward the root and broadcasting down, which takes a number of steps proportional to the logarithm of N rather than to N itself. NCCL implements a double binary tree, introduced in NCCL 2.4 in 2019, in which two complementary trees each carry half the data so that every rank's links are used in both directions. Trees win for small messages and very large groups; rings win for large messages on moderate groups. NCCL chooses between them, and among other algorithms, automatically based on message size and topology. A third family moves the reduction into the network itself. NVIDIA's SHARP (Scalable Hierarchical Aggregation and Reduction Protocol) allows InfiniBand switches to perform the summation as data passes through them, so that each rank sends its buffer once and receives the result once, roughly halving the traffic compared with a ring. On servers with third-generation and later NVSwitch, the analogous NVLink SHARP (NVLS) lets the switch chips inside the server perform reductions. These in-network reductions matter increasingly as clusters grow, because they reduce both traffic and the number of sequential steps. Real all-reduce implementations are also hierarchical. A cluster is not a flat set of equal links: GPUs inside a server are connected far more richly than GPUs in different servers. A hierarchical all-reduce first reduces within each server over the fast local links, then all-reduces across servers using one or several representatives per server over the network, then broadcasts within each server again. The cross-server phase carries the same total data but involves far fewer participants per ring. Modern NCCL topologies effectively do this automatically, building rings and trees that cross the slow links as few times as possible. A cost model: alpha and beta To predict how long communication takes, practitioners use the alpha-beta model, a simple linear model from the parallel computing literature. Sending a message of n bytes over a link costs α + n/β, where α is the latency (the fixed cost of starting a transfer, including software overhead, synchronisation and propagation) and β is the bandwidth in bytes per second. For a ring all-reduce of S bytes across N ranks, the model gives a time of about 2(N−1)·α + 2·(N−1)/N · S/β. For large messages the second term dominates, and the time is roughly 2S/β regardless of N. For small messages the first term dominates and grows with N. Two consequences follow immediately. First, large messages are efficient and small messages are not. The latency on a modern InfiniBand fabric, including software, is a few microseconds per step; at 50 gigabytes per second, a one-megabyte message takes about 20 microseconds to transmit. So a step that moves a one-megabyte chunk spends a substantial fraction of its time on latency, while a step that moves a hundred-megabyte chunk does not. This is why frameworks group many small gradient tensors into larger buckets before communicating them, a technique Chapter 3 discusses. Second, the bandwidth that matters is that of the slowest link the algorithm must cross. A ring that includes one cross-server hop runs at the speed of that hop, however fast the other links are. This is the quantitative core of the argument that runs through this book: communication patterns that must cross slow links should be designed to cross them rarely and with large, overlappable messages. When measuring collective performance, two bandwidth figures are reported, and confusing them causes endless misunderstanding. Algorithm bandwidth is simply the buffer size divided by the elapsed time. Bus bandwidth, the figure reported by NVIDIA's nccl-tests benchmarks, corrects for the algorithm's inherent traffic so that the number can be compared with the hardware's link speed. For all-reduce, bus bandwidth equals algorithm bandwidth multiplied by 2(N−1)/N. A system whose all-reduce shows a bus bandwidth close to the link's rated speed is performing well; a much lower figure signals a problem in topology, configuration, or hardware. The bandwidth hierarchy The alpha-beta model is only as useful as the values of α and β one feeds it, and those values differ by orders of magnitude depending on where in the system the communication happens. Table 3 gives representative figures for an NVIDIA H100 SXM server, the workhorse of large training runs in 2023 to 2025, and the corresponding figures for its Blackwell successor. The values are per GPU, from NVIDIA's published specifications. Table 3. The bandwidth hierarchy of an H100-class training cluster, per GPU, with Blackwell-generation figures for comparison (NVIDIA product specifications). Tier Technology H100 SXM (per GPU) B200 (per GPU) Typical use On-device memory HBM3 / HBM3e 3.35 TB/s about 8 TB/s Every kernel GPU to GPU, same server NVLink 4 / NVLink 5 via NVSwitch 900 GB/s total (450 GB/s each way) 1.8 TB/s total Tensor, sequence, expert parallelism GPU to host PCIe Gen5 x16 about 64 GB/s each way about 64 GB/s each way Offloading, data loading Server to server 400 Gb/s InfiniBand NDR or RoCE, one NIC per GPU 50 GB/s each way 50 to 100 GB/s each way Data, pipeline, context parallelism The ratio that matters most is between the second and fourth rows. Within a server, each H100 can exchange data with its seven peers at up to 450 gigabytes per second in each direction through the NVSwitch fabric. Across servers, each H100 has one 400-gigabit network interface, delivering 50 gigabytes per second in each direction. The intra-server link is nine times faster, and its latency is lower too, because it involves no network interface card, no switch hops beyond the NVSwitch chips, and no protocol stack beyond NVLink itself. That factor of nine, or ten, or more depending on generation and configuration, is the most important number in distributed training. It determines which techniques can run where. Any parallelism strategy that needs to communicate inside every layer must keep its communicating group within the fast domain. Strategies that communicate once per step, or only at boundaries, can tolerate the slower tier. The size of the fast domain has itself become a design variable. For most of the history of large-model training, the NVLink domain was a single server of eight GPUs. NVIDIA's GB200 NVL72 system, shipping since late 2024, connects 72 Blackwell GPUs in a single NVLink domain spanning a rack, using NVLink switch trays. Google's TPU pods take a different approach, connecting thousands of chips with a direct inter-chip interconnect arranged as a three-dimensional torus, supplemented by optical circuit switches in the TPU v4 and later generations. These architectures change where the boundaries fall, and therefore which degree of tensor or expert parallelism is affordable, but they do not abolish the hierarchy. There is always a tier above the fastest one. The shape of the network Beyond the server, the topology of the network matters as much as the speed of its links. Large training clusters are built as multi-level fat trees, or Clos networks, in which servers connect to leaf switches, leaf switches to spine switches, and in the largest clusters spines to a third layer. A fully provisioned fat tree gives every server the same bandwidth to every other server. Many clusters are deliberately oversubscribed at upper layers to save cost: Meta's 24,000-GPU RoCE cluster used for Llama 3 had, according to the Llama 3 paper, a 1:7 oversubscription ratio at its aggregation layer, meaning that traffic crossing that layer has only a seventh of the bandwidth available to traffic that stays below it. Job schedulers and parallel layouts therefore try to place the most communication-intensive groups close together in the topology. Most modern GPU clusters are also rail-optimised. In an eight-GPU server with eight network cards, GPU 0's card in every server connects to one set of leaf switches, GPU 1's card to another, and so on, forming eight "rails". Communication between GPU 0 in one server and GPU 0 in another stays on a single rail and crosses only one switch tier. Communication between GPU 0 in one server and GPU 3 in another must either cross the spine or hop through NVLink inside a server first. Collective libraries and parallel layouts that are aware of rails arrange for cross-server traffic to flow between same-index GPUs, and the standard layouts described in Chapter 9 do this naturally, because the data-parallel and pipeline-parallel groups connect corresponding GPUs in different servers. The choice between InfiniBand and Ethernet, long settled in favour of InfiniBand for high-performance training, reopened in the 2020s. RDMA over Converged Ethernet (RoCE) offers comparable bandwidth and remote direct memory access at lower cost and with broader vendor choice, but requires careful attention to congestion and load balancing because Ethernet was not designed for the lossless, low-jitter behaviour that collective communication wants. Meta trained Llama 3 405B on RoCE and its smaller models on InfiniBand, reporting that both could be made to work at scale. The Ultra Ethernet Consortium, formed in 2023, is standardising Ethernet extensions aimed at this workload. For the purposes of parallel layout design, the two are similar: what matters is the per-GPU bandwidth, the latency, and the topology. Overlap: the most important optimisation The alpha-beta model gives the time a communication takes in isolation. In a well-designed training system, much of that time is hidden. GPUs can compute and communicate at the same time: the data movement is performed by the network card and by dedicated copy engines or a small number of streaming multiprocessors, while the remaining multiprocessors continue executing compute kernels. If the communication for one part of the model can begin while computation for another part continues, the communication's cost disappears from the step time, provided the computation takes longer. Overlap is not free. Communication kernels that run on the GPU's streaming multiprocessors take some of them away from computation; DeepSeek reported dedicating 20 of the H800's 132 multiprocessors to communication in its V3 training. Communication and computation both consume memory bandwidth, and running them concurrently can slow each. And overlap requires a dependency structure that allows it: one cannot overlap the communication of a result with the computation that needs that result. Every technique in the following chapters can be judged by two questions. How many bytes must it move, and across which tier of the hierarchy? And how much of that movement can be overlapped with computation that must happen anyway? A technique that moves many bytes over fast links, and can overlap most of them, may be cheaper in practice than one that moves few bytes over slow links on the critical path. The rest of this book applies these two questions repeatedly, beginning with the simplest and most widely used strategy of all. Chapter 3. Data Parallelism: The Default and Its Ceiling Data parallelism is the first technique anyone reaches for, and it remains the outermost layer of nearly every large training job, including those that also use every other technique in this book. The idea is simple enough to state in a sentence: give every device a full copy of the model, give each a different slice of each batch, and combine their gradients before every update. The details, though, determine whether a data-parallel job runs at ninety per cent scaling efficiency or at forty, and its limits explain why the rest of the field exists. From parameter servers to all-reduce The first large distributed deep learning systems, notably Google's DistBelief, described by Dean and colleagues in 2012, used a parameter server architecture. A set of server machines held the model's parameters. Worker machines fetched the current parameters, computed gradients on their own data, and pushed the gradients back to the servers, which applied them. Often the process was asynchronous: workers did not wait for each other, and a worker's gradient might be computed against parameters that other workers had already updated several times. Asynchrony tolerated slow machines and unreliable networks, which mattered on the commodity clusters of the time, but it introduced staleness, gradients that pointed in a direction appropriate for an older version of the model, which harmed convergence in ways that became harder to manage as models and clusters grew. By the late 2010s the field had largely settled on synchronous data parallelism with all-reduce. In each training step, every worker computes gradients on its own micro-batch, the gradients are summed (or averaged) across all workers with an all-reduce, and every worker applies the identical update to its identical copy of the model. Because every replica starts from the same weights and applies the same updates, the replicas remain bit-for-bit identical, or nearly so, throughout training. Mathematically, the result is the same as training on one enormous device with a batch equal to the sum of all workers' batches. This equivalence is the great virtue of synchronous data parallelism: it changes the speed of training but not its mathematics, so hyperparameters, convergence behaviour, and debugging intuitions carry over. The cost is that every step proceeds at the pace of the slowest worker. A single slow GPU, a congested network link, or a worker stalled on data loading holds up the entire job. At the scale of thousands of devices, managing such stragglers becomes a significant operational concern, taken up in Chapter 10. Inside a data-parallel step PyTorch's DistributedDataParallel (DDP), described by Li and colleagues in a 2020 paper, is the reference implementation, and its design shows the engineering that turns the simple idea into an efficient one. At start-up, DDP broadcasts the parameters from one rank to all others, guaranteeing identical initialisation. During the forward pass, nothing distributed happens: each replica processes its own data independently. The interesting work is in the backward pass. A naive implementation would wait until the backward pass had finished, then all-reduce all the gradients at once. That puts the entire communication on the critical path, after all the computation. DDP instead exploits the structure of backpropagation. Gradients are computed layer by layer from the output back to the input, so the gradients for the last layers are ready long before those for the first. DDP registers a hook on each parameter that fires when its gradient is ready. As soon as enough gradients are ready, DDP launches an all-reduce for them, asynchronously, while the backward pass continues computing gradients for earlier layers. By the time the backward pass reaches the input layer, most of the gradients have already been reduced, and only the last bucket's communication remains exposed. The grouping into buckets is essential. A large model has thousands of parameter tensors, many of them small (biases, normalisation scales). Launching a separate all-reduce for each would incur the latency cost thousands of times. DDP concatenates gradients into buckets, by default about 25 megabytes each, and all-reduces each bucket as a unit once all its gradients are ready. Buckets are filled in roughly the reverse order of the parameters' definition, which approximates the order in which gradients become available. The bucket size is a tuning parameter: larger buckets amortise latency better but delay the start of communication and reduce overlap; smaller buckets start sooner but pay more latency. DDP also handles a subtle problem. If some parameters receive no gradient in a given step, because part of the model was not used, their hooks never fire and their buckets never complete, and the job hangs. An option tells DDP to detect unused parameters by traversing the autograd graph after the forward pass, at some cost. Many models with conditional computation have tripped over this, and it is an early example of a general truth about distributed training: every rank must make exactly the same sequence of collective calls, in the same order, or the job deadlocks. Gradient accumulation Gradient accumulation decouples the batch size used for each optimiser step from the batch size each device processes at once. A worker runs forward and backward on several micro-batches in succession, adding the gradients together locally, and only after the last micro-batch does it participate in the all-reduce and apply the update. DDP provides a context in which the gradient synchronisation is suppressed for the intermediate micro-batches. Accumulation serves two purposes. It allows a large effective batch on devices whose memory can hold only a small micro-batch of activations. And it reduces the frequency of communication relative to computation: the all-reduce happens once per optimiser step regardless of how many micro-batches are accumulated, so accumulating more micro-batches increases the computation that the communication must be hidden behind. The second purpose leads directly to the central quantitative question about data parallelism. When can the gradient exchange be hidden? Consider a model with Ψ parameters, with gradients exchanged in BF16, 2 bytes per parameter. A ring all-reduce has each rank send and receive about twice the buffer size, so each GPU moves about 2 × 2Ψ = 4Ψ bytes per step. At a cross-server bandwidth of β bytes per second, the communication takes roughly 4Ψ/β seconds. The computation, by the six-operations-per-parameter-per-token rule from Chapter 1, takes 6Ψ·T/F seconds, where T is the number of tokens each GPU processes per step and F is its sustained throughput in operations per second. For the communication to be completely hidden behind computation, the computation must take longer: 6Ψ·T/F > 4Ψ/β. The model size Ψ cancels from both sides, leaving T > (2/3)·F/β. This is a striking result. Whether data-parallel gradient communication can be hidden does not depend on the size of the model. It depends only on the ratio of each GPU's compute speed to its network bandwidth, and on how many tokens each GPU processes per step. For an H100 sustaining 400 teraflops with 50 gigabytes per second of network bandwidth, the threshold is (2/3) × 400 × 10¹² / 50 × 10⁹, about 5,300 tokens per GPU per step. A GPU processing two sequences of 4,096 tokens per step, either at once or through accumulation, is comfortably over the line. Real systems need a margin above this threshold, because overlap is never perfect: the last bucket's communication is always exposed, communication kernels take some compute resources, and the ring's latency term adds to the bandwidth term. But the result explains why data parallelism scales so well in practice when each GPU has enough work. It also shows how the balance shifts with hardware generations. Each new GPU generation has raised compute throughput faster than network bandwidth per GPU, which raises the threshold: the faster the GPU relative to its network, the more tokens each GPU must process per step to keep the network off the critical path. The first ceiling: the global batch The threshold above concerns tokens per GPU. Multiply by the number of GPUs and one obtains the global batch size, the number of tokens contributing to each optimiser step. Here data parallelism meets its first fundamental ceiling: the global batch cannot grow without limit. The effect of batch size on training was studied systematically in the late 2010s. Goyal and colleagues at Facebook showed in 2017 that ResNet-50 could be trained on ImageNet in one hour on 256 GPUs with a batch of 8,192 images, matching the accuracy of small-batch training, provided the learning rate was scaled linearly with the batch size and warmed up gradually over the first few epochs. Specialised optimisers such as LARS and LAMB, developed by You and colleagues, pushed the usable batch further, LAMB famously reducing BERT pre-training to 76 minutes on a large TPU pod. But these results also made clear that the benefits diminish. Beyond a certain batch size, doubling the batch no longer halves the number of steps needed; the extra examples per step add little new information, and the total computation required to reach a given loss starts to rise. McCandlish and colleagues at OpenAI formalised this in 2018 with the notion of a critical batch size, which they related to the gradient noise scale: a measure of how noisy the gradient estimate is relative to its magnitude. Below the critical batch size, larger batches give nearly linear speedups in steps; above it, they give diminishing returns, and the computation is increasingly wasted. The critical batch size is not fixed: it grows as training progresses and the loss falls, which is one reason many large runs increase their batch size during training. GPT-3 ramped its batch size up over the early part of training; the Llama 3 405B run, according to its paper, began with a batch of 4 million tokens and increased it in stages to 16 million. For large language models, global batch sizes in the range of several million to a few tens of millions of tokens are typical. Sixteen million tokens across 16,384 GPUs is fewer than a thousand tokens per GPU, far below the 5,300-token threshold computed above, and far below what an H100 can usefully process in a step. A purely data-parallel job at that scale would spend most of its time waiting on the network, and many of its GPUs would have less than one sequence each. This is a less widely appreciated reason for model parallelism. It is usually explained as a memory technique, and it is one. But it is also how a large job keeps the number of independent data-parallel replicas small enough that each replica gets a healthy share of the global batch. If eight-way tensor parallelism and sixteen-way pipeline parallelism group 128 GPUs into one model replica, then 16,384 GPUs form only 128 replicas, and each replica processes 125,000 tokens per step. Because the replica's parameters are divided among its 128 GPUs, each GPU's share of the gradient exchange shrinks by the same factor as its share of the computation, and the overlap condition becomes a condition on tokens per replica rather than tokens per GPU, now comfortably satisfied. The second ceiling: memory The second ceiling is the one every practitioner hits first. Plain data parallelism replicates everything: every GPU holds the full parameters, the full gradients, and the full optimizer state. By the accounting of Chapter 1, that limits the model to a few billion parameters on an 80-gigabyte device, however many devices one has. Adding GPUs adds compute but not memory capacity for the model. A thousand GPUs running plain data parallelism can train no larger a model than one. This redundancy is enormous. In a 1,024-way data-parallel job, there are 1,024 identical copies of the optimizer state, each updated with identical arithmetic. Removing that redundancy, without giving up the simplicity and scalability of data parallelism, is the subject of the next chapter. Variations at the edges Several variations on data parallelism address its communication cost directly, and are worth knowing even though they are not standard in large-scale pre-training. Gradient compression reduces the bytes exchanged by quantising gradients to fewer bits, sparsifying them by sending only the largest components, or projecting them onto a low-rank approximation. PowerSGD, from Vogels and colleagues in 2019, uses low-rank compression; 1-bit Adam and its successors from the DeepSpeed team quantise aggressively after a warm-up period. Compression has found use where bandwidth is scarce, but in large dense training on well-provisioned clusters, the overlap described above usually hides the full-precision exchange, and compression adds computation and the risk of convergence changes. Its most successful descendant is the quantised communication in ZeRO++, discussed in Chapter 4. Local SGD and its descendants let each replica take several optimiser steps on its own before synchronising, by averaging parameters rather than gradients. DiLoCo, introduced by Douillard and colleagues at Google DeepMind in 2023, runs an inner optimiser locally for hundreds of steps and an outer optimiser on the averaged parameter changes, reducing communication by orders of magnitude. Such methods are attracting attention for training across multiple data centres, or across loosely connected clusters, where the cross-site bandwidth is far below anything inside a cluster. They change the mathematics of training, not just its speed, and so must be validated for each use; that is the price of their dramatic communication savings. Hierarchical and hybrid schemes combine fast synchronous reduction within a well-connected group with slower or less frequent reduction across groups. Hybrid sharded data parallelism, discussed in the next chapter, is the most widely used example. What data parallelism leaves unsolved Synchronous data parallelism with overlapped bucketed all-reduce is close to ideal for what it does. Its communication volume per GPU is independent of the number of GPUs; its communication can be hidden whenever each replica has enough tokens; and it leaves the mathematics of training untouched. Its two ceilings are the memory of a single device, which bounds the model size, and the critical batch size, which bounds the number of replicas that can be usefully employed. The first ceiling can be raised without leaving the data-parallel paradigm, by sharding the replicated state. The second can only be raised by making each replica larger, spreading one copy of the model over several devices. Every remaining technique in this book does one or the other. Chapter 4. ZeRO and Fully Sharded Data Parallelism In 2019 a team at Microsoft led by Samyam Rajbhandari published a paper whose title stated its ambition plainly: "ZeRO: Memory Optimizations Toward Training Trillion Parameter Models." Its central observation was that data parallelism wastes almost all of its memory. In an N-way data-parallel job, there are N complete copies of the parameters, N copies of the gradients, and N copies of the optimizer state, and the copies are identical. The paper asked whether the copies could be removed while keeping the computational structure, the communication efficiency, and the simplicity of data parallelism. The answer, the Zero Redundancy Optimizer, has become one of the foundations of large-scale training, and its ideas are now built into PyTorch as Fully Sharded Data Parallel (FSDP), into JAX through sharding annotations, and into every major training framework. The three stages ZeRO removes redundancy in three progressive stages, each sharding one more block of the model state across the N ranks of a data-parallel group. The beauty of the scheme is that the first two stages cost no extra communication at all, and the third costs only half again as much as plain data parallelism. Stage 1: shard the optimizer state. The optimizer state, the FP32 master weights and the two Adam moments, is twelve of the sixteen bytes per parameter. Each parameter's update depends only on its own gradient and its own optimizer state, so there is no need for every rank to update every parameter. In stage 1, the parameters are divided into N equal partitions, and rank i holds the optimizer state only for partition i. In each step, every rank computes the full gradient as usual. But rather than all-reducing the gradients, the ranks perform a reduce-scatter, so that rank i receives the fully summed gradient only for its own partition. Rank i then updates its partition of the master weights using its partition of the optimizer state, and the ranks perform an all-gather of the updated 16-bit parameters so that every rank again holds the full model for the next forward pass. Recall from Chapter 2 that an all-reduce is a reduce-scatter followed by an all-gather, and that each half moves the same amount of data. Stage 1 performs exactly those two halves, separated by the optimizer step. Its communication volume is therefore identical to plain data parallelism. Yet the optimizer's memory falls from 12 bytes per parameter to 12/N. Stage 2: also shard the gradients. In stage 1, each rank still materialises the full gradient, even though it only ever uses the reduced gradient for its own partition. In stage 2, as each bucket of gradients is reduce-scattered during the backward pass, each rank keeps only its own shard of the result and frees the rest. Gradient memory falls from 2 bytes per parameter to 2/N, again with no change in communication volume. Stage 3: also shard the parameters. In stage 3, each rank holds only its partition of the 16-bit parameters too. Before a layer can be computed in the forward pass, its full parameters must be assembled, so the ranks perform an all-gather for that layer, compute, and then discard the gathered parameters, keeping only their own shard. The backward pass needs the parameters again, so they are all-gathered a second time, the gradients are computed, and the gradients are reduce-scattered so that each rank keeps only its shard. Memory for all model state is now 16/N bytes per parameter: it scales down perfectly with the number of ranks. The communication cost of stage 3 is one all-gather of the parameters in the forward pass, one in the backward pass, and one reduce-scatter of the gradients, three operations each moving about Ψ elements per rank, compared with plain data parallelism's two. Stage 3 thus costs 1.5 times the communication of ordinary data parallelism. Table 4 sets out the memory and communication for each stage, along with the ZeRO paper's illustrative numbers for a 7.5-billion-parameter model trained on 64 GPUs. Table 4. Per-GPU model-state memory and communication volume for the ZeRO stages, for Ψ parameters across N data-parallel ranks, with the ZeRO paper's example of Ψ = 7.5 billion and N = 64 (Rajbhandari et al., 2019). Configuration What is sharded Model-state bytes per GPU Example: 7.5B, 64 GPUs Communication relative to data parallelism Plain data parallelism Nothing 16Ψ 120 GB 1× ZeRO stage 1 Optimizer state 4Ψ + 12Ψ/N 31.4 GB 1× ZeRO stage 2 Optimizer state, gradients 2Ψ + 14Ψ/N 16.6 GB 1× ZeRO stage 3 / FSDP Optimizer state, gradients, parameters 16Ψ/N 1.9 GB 1.5× The example makes the point vividly. A model that needs 120 gigabytes per GPU under plain data parallelism needs under 2 gigabytes per GPU under stage 3 with 64 ranks. The model state of a 70-billion-parameter model, 1,120 gigabytes in total, fits in under 9 gigabytes per GPU across 128 ranks. What stage 3 actually requires The tidy formula 16Ψ/N hides some important realities about what fully sharded training needs in practice. The first is that sharding the model state does nothing for activations. Each rank still processes its own micro-batch through the entire model and stores that micro-batch's activations. For long sequences or large hidden dimensions, the activations can dwarf the sharded model state, and stage 3 offers no relief. This is why sharded data parallelism alone cannot handle long-context training and why it is combined with activation recomputation, tensor parallelism, or context parallelism. The second is that the full parameters of at least one layer must be materialised at a time, and in practice more than one. To overlap the all-gather of a layer's parameters with computation, the system must fetch the next layer's parameters while computing the current one, so at least two layers' worth of full parameters are resident. For a model with very wide layers, or with a single enormous embedding table, this floor can be substantial. Systems group parameters into units, usually one transformer block per unit, and the largest unit sets the floor. The third is that the overlap condition is tighter than for plain data parallelism. Following the reasoning of Chapter 3, stage 3 moves about 3 × 2Ψ = 6Ψ bytes per GPU per step in BF16, against a computation time of 6Ψ·T/F. The model size again cancels, and the condition for full overlap becomes T > F/β: for an H100 at 400 teraflops with 50 gigabytes per second, about 8,000 tokens per GPU per step, compared with 5,300 for plain data parallelism. With gradient accumulation, the parameters must be gathered for every micro-batch, not just once per step, unless the system keeps them resident between micro-batches, which costs memory. Stage 2 has no such issue, because it gathers parameters only once per optimiser step, which is one reason that ZeRO stage 1 or 2 is often preferred when the model state fits without full parameter sharding. The fourth is latency at scale. A ring all-gather across N ranks takes N−1 sequential steps, and in a fully sharded job each rank's shard of a single layer shrinks as N grows. Across 4,096 ranks, a layer with 200 million parameters gives each rank a shard of fewer than 50,000 elements, and the collective becomes dominated by latency rather than bandwidth. Fully sharding across the entire cluster is therefore rarely optimal at very large scale. Fully Sharded Data Parallel in PyTorch PyTorch's FSDP, described in a 2023 paper by Zhao and colleagues at Meta, brought ZeRO stage 3 into the core framework and is now the most common way it is used. Its design choices illustrate how the abstract scheme becomes a practical system. The original FSDP organises parameters into units by wrapping submodules, typically each transformer block, according to a user-specified policy. Within each unit, it flattens all parameters into a single contiguous FlatParameter and shards that evenly across ranks. Flattening makes each all-gather and reduce-scatter a single large collective rather than many small ones, which matters given the alpha-beta model. Before a unit's forward computation, FSDP all-gathers its FlatParameter; afterwards it frees the gathered copy, unless configured to keep it for the backward pass. In the backward pass it gathers again, computes gradients, and reduce-scatters them. To overlap communication, FSDP prefetches. In the forward pass, it issues the all-gather for the next unit before the current unit's computation completes. In the backward pass, where the order of units is reversed, it can issue the all-gather for the previous unit before the current unit's reduce-scatter, a setting called backward prefetch. Because aggressive prefetching can cause a burst of memory allocation if communication runs far ahead of computation, FSDP includes a rate limiter that bounds the number of in-flight all-gathers. FSDP exposes the trade-offs as choices. Setting it not to reshard parameters after the forward pass keeps each unit's full parameters in memory until its backward pass, avoiding the second all-gather at the cost of memory, a configuration equivalent to ZeRO stage 2 in communication. The Llama 3 paper reports using exactly this choice: its FSDP implementation did not reshard after the forward pass, avoiding an extra all-gather during backward. It also reported accumulating and reduce-scattering gradients in FP32 for numerical stability, doubling the gradient communication in exchange for more accurate sums. A redesigned implementation, commonly called FSDP2, became available in PyTorch releases from 2024 onward. It abandons the flattened parameter in favour of sharding each parameter individually along its first dimension, represented using PyTorch's DTensor abstraction for distributed tensors. Per-parameter sharding makes it easier to combine FSDP with tensor parallelism, to apply different precision or freezing to different parameters, and to save and load checkpoints in a sharded form that does not depend on the flattening order. It is the basis of PyTorch's TorchTitan reference implementation for large-model pre-training. Hybrid sharding The latency problem at very large scale, and the general principle of keeping chatty communication on fast links, led to hybrid schemes. In hybrid sharded data parallelism (HSDP), the data-parallel group is divided into subgroups. Model state is fully sharded within each subgroup, and the subgroups are replicas of one another. Within a step, each subgroup performs the all-gathers and reduce-scatters of FSDP among its own members; then the subgroups all-reduce their gradient shards with each other, each rank exchanging only with its counterparts in other subgroups. If the subgroup is chosen to be a single server, or a small set of well-connected servers, then the frequent parameter all-gathers run over NVLink or within a single leaf switch, while only the gradient reduction crosses the wider network. The memory saving is limited to the subgroup size rather than the full data-parallel degree, but if the subgroup is large enough to fit the model, that is all that is needed. Amazon's MiCS system, published in 2022, made the same argument: shard only as widely as memory requires, and replicate beyond that. Offloading and ZeRO++ Two further families extend the ZeRO idea in opposite directions. Offloading moves state off the GPU entirely. ZeRO-Offload, from Ren and colleagues in 2021, keeps the optimizer state and the optimizer computation on the host CPU, sending gradients to the CPU and receiving updated parameters back over PCIe. It enabled training models of over ten billion parameters on a single GPU. ZeRO-Infinity extended offloading to NVMe solid-state storage, aiming at models whose state exceeds even aggregate CPU memory. Offloading is valuable when GPU memory is the binding constraint and GPU count is small: fine-tuning a large model on a single workstation, or research on modest clusters. At the scale of thousands of GPUs, the PCIe link, at roughly a seventh of the NVLink bandwidth per GPU and shared with data loading and network traffic, becomes a bottleneck, and the CPU's arithmetic for the optimizer step can stall the GPU; large pre-training runs generally avoid it. ZeRO++, from Wang and colleagues at Microsoft in 2023, attacks stage 3's communication volume. It combines three techniques: quantising the parameters to 8 bits before the all-gather, keeping a secondary copy of the parameters sharded within each server so that the backward pass's all-gather can run over NVLink rather than the network, and quantising gradients for the reduce-scatter with a scheme designed to avoid accumulating quantisation error. Together these reduced cross-server communication by about four times in the authors' experiments, with the largest benefits on clusters with limited network bandwidth or very small per-GPU batches. The secondary in-server copy is another instance of the hierarchy principle: spend some memory to move communication from the slow tier to the fast one. Sharding as a general abstraction ZeRO was presented as an optimiser for data parallelism, but its underlying idea, that any tensor can be partitioned across a group of devices and gathered on demand, is more general. Google's GSPMD system, described by Xu and colleagues in 2021 and underlying JAX's sharding, lets users annotate how each tensor should be partitioned across a logical mesh of devices, and has the compiler insert the necessary collectives. In that framework, fully sharded data parallelism, tensor parallelism, and their combinations are all simply sharding specifications: shard the parameters along the data axis of the mesh, or along the model axis, or both. Google's PaLM, trained on 6,144 TPU v4 chips across two pods in 2022, used a layout that its paper described as 12-way model parallelism combined with 256-way fully sharded data parallelism within each pod, with data parallelism across the two pods. PyTorch's DTensor and DeviceMesh abstractions move in the same direction. A device mesh is a multi-dimensional arrangement of ranks, for instance eight ranks by sixteen by 128, and each dimension can be assigned to a different form of parallelism. FSDP2 shards along one dimension of the mesh; tensor parallelism shards along another. This mesh view is the natural language for the composite layouts of Chapter 9. The place of sharded data parallelism ZeRO and FSDP solve the memory ceiling of data parallelism for model state, at little or no cost in communication volume. Stage 1 is nearly free and is used almost universally in large jobs; BLOOM's 176-billion-parameter training, for example, used ZeRO stage 1 alongside tensor and pipeline parallelism, and DeepSeek-V3 used ZeRO-1-style data parallelism alongside pipeline and expert parallelism. Full sharding is the simplest route to training models of tens of billions of parameters on a few hundred GPUs, and with hybrid sharding and careful overlap it scales further. What sharded data parallelism cannot do is reduce activation memory or reduce the computation per replica. Each rank still runs the entire model on its own micro-batch. When the activations of a single sequence do not fit on a device, or when the global batch is too small to give every rank a useful micro-batch, a single sample's computation must be divided among devices. That requires splitting the layers themselves, which is the subject of the next chapter. Chapter 5. Tensor Parallelism: Splitting the Matrix Multiply Sharded data parallelism divides the storage of a model across devices but leaves the computation whole: every rank still runs every layer on its own data, gathering the parameters it needs just in time. Tensor parallelism takes the opposite approach to a single layer. It divides the layer's computation itself, so that several devices cooperate on the same data, each computing a part of every matrix multiplication with its own part of the weights. The weights are never gathered; instead, the partial results are combined. This divides not only the parameter storage but also the arithmetic and a large share of the activations, and it allows a single training example to be processed by a group of devices too large for any one of them to manage. The technique in its modern form was laid out in the 2019 Megatron-LM paper by Shoeybi and colleagues at NVIDIA. Its achievement was not the general idea of splitting matrices across processors, which is old in scientific computing, but the identification of a particular way of splitting the transformer layer that requires remarkably little communication and almost no change to the model's code. Splitting one matrix multiplication Begin with a single linear layer: an input X, with one row per token and h columns, multiplied by a weight matrix A of size h by h′, to produce an output Y = XA. There are two natural ways to divide A between two devices. The first is to split A by columns: A = [A₁, A₂], each half of size h by h′/2. If both devices hold the full input X, device 1 computes XA₁ and device 2 computes XA₂. Each device then holds half of the output's columns. No communication is needed to compute these halves; communication is needed only if some later computation needs the full Y. The second is to split A by rows: A is stacked as A₁ over A₂, each of size h/2 by h′. For this to work, the input must be split correspondingly by columns, X = [X₁, X₂]. Device 1 computes X₁A₁ and device 2 computes X₂A₂, and the true output is their sum: Y = X₁A₁ + X₂A₂. Each device holds a partial sum of the full output, and an all-reduce produces the complete Y on both. The insight of Megatron-LM is that these two splits fit together perfectly when two linear layers are applied in sequence. A column-parallel layer produces an output split by columns, which is exactly the input a row-parallel layer requires. Chaining a column-parallel layer into a row-parallel layer, therefore, needs no communication between them, and only one all-reduce at the end. The transformer's feed-forward block The feed-forward block, or MLP, of a transformer layer applies two linear layers with a nonlinearity between them: Z = GeLU(XA)·B, where A expands the hidden dimension h to an intermediate dimension, typically 4h, and B projects back down to h. (Modern models often use a gated variant such as SwiGLU, with three matrices rather than two, but the parallel structure is the same.) Split A by columns across t devices. Each device computes its slice of XA, a block of the intermediate activations. Now apply the GeLU. Because GeLU acts on each element independently, each device can apply it to its own block without needing the others' blocks. This is the crucial detail. If A had been split by rows, each device would hold a partial sum of XA, and since GeLU(a + b) is not GeLU(a) + GeLU(b), the partial sums would have to be all-reduced before the nonlinearity, costing a synchronisation in the middle of the block. The column split avoids it. Next, split B by rows, matching the column split of its input. Each device multiplies its block of the intermediate activations by its slice of B, producing a partial sum of the output Z. A single all-reduce across the t devices produces the complete Z on every device, ready for the residual connection and the next block. The forward pass of the MLP thus requires one all-reduce. In the backward pass the pattern is mirrored. The gradient flowing back into the block arrives complete on every device; each device computes the gradients for its own slices of B and A; and the gradients with respect to the block's input X, which are partial sums from each device's slice of A, are combined by one all-reduce. Megatron-LM formalised this pairing with two conjugate operators, called f and g in the paper. The operator f, placed at the start of the parallel region, is the identity in the forward pass and an all-reduce in the backward pass. The operator g, placed at the end, is an all-reduce in the forward pass and the identity in the backward pass. Writing the block as f, then column-parallel linear, then the nonlinearity, then row-parallel linear, then g, makes the communication explicit, and the automatic differentiation machinery handles the rest. The attention block The attention block divides even more naturally. Multi-head attention already computes several independent attention operations, one per head, on separate slices of the query, key, and value projections. Tensor parallelism assigns a subset of the heads to each device. The projection matrices that produce queries, keys, and values from the input are split by columns, so that each device computes the queries, keys, and values for its own heads. Each device then performs the attention computation for its heads entirely locally: scores, softmax, and the weighted sum of values, with no communication. Finally, the output projection, which mixes the heads back together, is split by rows, and an all-reduce produces the block's output. So attention, like the MLP, needs one all-reduce in the forward pass and one in the backward pass. A complete transformer layer, with one attention block and one MLP block, requires two all-reduces forward and two backward, each on a tensor of size b·s·h, the full activation for the micro-batch. The head split introduces a constraint that matters in practice: the tensor-parallel degree must divide the number of heads. Modern models complicate this further with grouped-query attention, in which many query heads share a smaller number of key and value heads. Llama 3 70B, for instance, has 64 query heads and 8 key-value heads. A tensor-parallel degree of 8 gives each device one key-value head and eight query heads, a clean division. A degree of 16 would require each key-value head to be replicated on two devices, wasting memory and some computation. The architecture of the model and the parallel layout constrain each other. Embeddings and the output layer The input embedding and the output projection to the vocabulary are large matrices in language models, with one row per vocabulary entry. With a vocabulary of 128,000 tokens and a hidden size of 8,192, each holds about a billion parameters. Megatron splits the embedding table by rows, along the vocabulary dimension, so that each device holds the embeddings for a range of token identifiers. To look up an embedding, each device produces the embedding for tokens in its range and zeros for the others, and an all-reduce assembles the result. The output layer is more interesting. It produces logits over the entire vocabulary, a tensor of size b·s·V, which for long sequences and large vocabularies is enormous. Splitting the output projection by columns gives each device the logits for its slice of the vocabulary. Gathering the full logits onto every device would require moving that huge tensor. Instead, Megatron computes the cross-entropy loss in parallel: each device computes the maximum and the sum of exponentials over its slice, an all-reduce of these per-token scalars yields the global softmax normalisation, and each device computes its contribution to the loss. The communication drops from b·s·V elements to a few per token. This is a small example of a recurring theme: a parallel algorithm should communicate the smallest summary of the data that suffices, not the data itself. What tensor parallelism saves With t-way tensor parallelism, each device holds 1/t of the weights of every linear layer, and with them 1/t of the gradients and optimizer state for those weights. Since linear layers hold almost all of a transformer's parameters, the model state per device falls by nearly a factor of t. The arithmetic per device also falls by nearly a factor of t, since each device performs 1/t of every large matrix multiplication. The activations are divided less completely. Inside the parallel regions, the intermediate activations of the MLP and the per-head attention tensors are split across devices. But the inputs to each block, the layer normalisations, the dropout operations, and the residual stream, sit outside the parallel regions, and every device holds a full copy of them. Korthikanti and colleagues' accounting, mentioned in Chapter 1, shows the result: with t-way tensor parallelism, the activation memory per layer becomes s·b·h·(10 + 24/t + 5·a·s/(h·t)) bytes. The terms divided by t shrink as t grows, but the constant 10·s·b·h does not. With t = 8, the replicated part is already larger than the parallelised linear-layer part. This residue is what sequence parallelism, in the next chapter, removes. The cost: communication on the critical path Tensor parallelism's communication differs from data parallelism's in two ways that make it far more demanding. The first is frequency. Data parallelism communicates once per optimiser step. Tensor parallelism communicates four times per layer per micro-batch: a model with 80 layers and 16 micro-batches per step makes over five thousand tensor-parallel all-reduces per step on each device. The second, more important, is that these all-reduces sit on the critical path. A data-parallel gradient all-reduce can overlap with the backward pass of earlier layers, because nothing needs the reduced gradient until the optimiser step. The tensor-parallel all-reduce at the end of an MLP block produces the input to the next block; nothing can proceed until it completes. In the simplest implementations, the GPU's arithmetic units sit idle while it runs. A rough estimate shows the consequence. In the forward pass of one layer, the large matrix multiplications perform about 24·b·s·h² floating-point operations, divided across t devices. The two all-reduces each operate on 2·b·s·h bytes in BF16, and with a ring algorithm each device moves about 2(t−1)/t times that, so roughly 7·b·s·h bytes per layer for t = 8. The ratio of communication time to computation time is therefore about (7·b·s·h/β) divided by (24·b·s·h²/(t·F)), which simplifies to 7·t·F/(24·h·β). Notice that the batch and sequence dimensions cancel: the overhead depends only on the parallel degree, the hidden size, and the ratio of compute speed to link bandwidth. Put in numbers for an H100 at 400 teraflops sustained, with NVLink at 450 gigabytes per second each way, t = 8, and h = 8,192, the size used in Llama 3 70B: the ratio is 7 × 8 × 400 × 10¹² divided by 24 × 8,192 × 450 × 10⁹, about 0.25. If none of it were overlapped, communication would add a quarter to the computation time of each layer. In practice, NVLink SHARP reductions, overlapping techniques described below, and kernel fusion reduce this considerably, but the estimate correctly conveys that tensor parallelism is expensive even on the fastest links available. Now move the same calculation across servers, where each GPU has 50 gigabytes per second rather than 450. The ratio rises ninefold, to more than two. Communication would take more than twice as long as computation. And raising t to 16 to span two servers doubles the ratio again. This is the quantitative reason for the most consistent rule in large-model training: tensor parallelism stays inside the NVLink domain. For most of the past five years that meant at most eight-way tensor parallelism, matching the eight GPUs of a DGX or HGX server. Megatron-Turing NLG, trained by Microsoft and NVIDIA in 2021, used eight-way tensor parallelism within each DGX A100 server for exactly this reason, as did Llama 3 405B on H100 servers in 2024. The formula also shows that tensor parallelism becomes relatively cheaper as models get wider. Because the overhead falls as 1/h, a model with a hidden size of 16,384 pays half the relative cost of one with 8,192. Wide models were the original target of tensor parallelism, and they remain where it pays best. A second, subtler cost arises from the matrix shapes. Splitting a matrix multiplication eight ways leaves each GPU with one-eighth of the columns or rows. For smaller models, or small micro-batches, the resulting matrices may be too thin to keep the GPU's tensor cores fully occupied, and achieved throughput per GPU falls. The original Megatron-LM paper reported 76 per cent weak-scaling efficiency when training an 8.3-billion-parameter model on 512 V100 GPUs, combining eight-way tensor parallelism with 64-way data parallelism, sustaining 15.1 petaflops in total; the loss of a quarter of ideal efficiency reflects both communication and these smaller kernels. Hiding tensor-parallel communication Because tensor-parallel communication sits on the critical path, hiding it requires restructuring the computation so that independent work exists to overlap with. The most important technique decomposes a matrix multiplication and its adjacent collective into smaller pieces and pipelines them. Suppose a row-parallel matrix multiplication is followed by a reduction. Split the multiplication into several chunks along the token dimension. As soon as the first chunk's partial result is computed, begin reducing it while the second chunk is computed, and so on. The communication of each chunk overlaps the computation of the next, and only the last chunk's communication is exposed. Google described a general version of this decomposition for TPUs in 2023, under the name of collective matmul, and NVIDIA's Transformer Engine and Megatron-Core implement comparable overlaps for GPUs using direct peer-to-peer memory access over NVLink. PyTorch added an asynchronous tensor-parallel mode that fuses all-gather and reduce-scatter with the adjacent matrix multiplications in a similar way. Other techniques reduce the communication rather than hiding it. Performing reductions inside the NVSwitch chips with NVLink SHARP halves the traffic of an all-reduce. Overlapping one micro-batch's tensor-parallel communication with another micro-batch's computation is possible when the pipeline schedule interleaves them, which some recent schedules, including DeepSeek's DualPipe described in Chapter 7, are designed to exploit. Beyond one dimension Megatron's scheme is one-dimensional: each matrix is split along a single axis, and the activations entering and leaving each block are replicated across the tensor-parallel group. Researchers have proposed two- and three-dimensional tensor parallel schemes that split both the weights and the activations along multiple axes, drawing on classical distributed matrix multiplication algorithms such as SUMMA. These reduce memory further and change how communication scales with the number of devices, and implementations exist in research frameworks such as Colossal-AI. In large-scale practice, however, one-dimensional tensor parallelism within a node, combined with the sequence parallelism of the next chapter to shard the replicated activations, has proven sufficient and simpler, and the other axes of scale are handled by data, pipeline, and context parallelism. Hardware is shifting the boundary. In a GB200 NVL72 rack, 72 GPUs share a single NVLink domain at 1.8 terabytes per second per GPU. That makes higher tensor-parallel degrees, or large expert-parallel groups, feasible without crossing to the slower network. The formula above still applies, so going from eight-way to 64-way tensor parallelism multiplies the relative communication cost by eight; the larger domain is often better spent on expert or context parallelism than on extending tensor parallelism itself. But the principle is unchanged: the chattiest split goes where the links are fastest, and the size of the fastest domain sets the limit. What tensor parallelism leaves unsolved Tensor parallelism divides a layer's weights and computation among a small, tightly coupled group of devices, at the price of frequent, latency-sensitive communication that must stay on the fastest links. It cuts model state per device by the tensor-parallel degree and divides the activations inside each block, but leaves a full copy of the activations at the boundaries between blocks on every device, an increasingly large share as the degree grows and as sequences lengthen. Its scale is bounded by the size of the NVLink domain. To go further, the activations themselves must be divided along the sequence, and the model must be divided along its depth. The next two chapters take up these two directions in turn. Chapter 6. Sequence and Context Parallelism: Splitting the Activations Every technique so far has divided either the model's state or the model's arithmetic. None has divided the sequence. Yet the sequence dimension is where activation memory lives, and as training contexts have grown from 2,048 tokens to 128,000 and beyond, it has become the dimension that most often decides whether a configuration fits. Two quite different techniques split along it. The first, which NVIDIA named sequence parallelism, is a refinement of tensor parallelism that shards the activations tensor parallelism leaves replicated. The second, now usually called context parallelism, divides the sequence across devices throughout the whole layer, including the attention computation, which is where the difficulty lies. This chapter treats them in that order, because the first is a small and elegant change and the second is a substantial new axis of parallelism. A word on names is unavoidable. "Sequence parallelism" has been used in the literature for at least three distinct techniques: a 2021 proposal by Li and colleagues for distributing attention across devices with a ring algorithm, NVIDIA's 2022 technique for sharding the non-tensor-parallel regions of a layer, and DeepSpeed's 2023 Ulysses system for long sequences. NVIDIA's Megatron framework later adopted "context parallelism" for the long-sequence form, and that usage has become common. This book follows it: sequence parallelism means the companion to tensor parallelism, and context parallelism means splitting the sequence for long-context training. Sequence parallelism: finishing what tensor parallelism started Chapter 5 ended with a residue. Within a tensor-parallel group, the inputs to each attention and MLP block, the layer normalisations, the dropout operations, and the residual additions are replicated on every device. Each device performs the same layer normalisation on the same full tensor, applies the same dropout, and stores the same activations. In Korthikanti and colleagues' accounting, these replicated regions contribute 10·s·b·h bytes per layer regardless of the tensor-parallel degree. The observation that unlocks this residue is that these operations are independent across positions in the sequence. Layer normalisation normalises each token's vector separately. Dropout masks each element separately. A residual addition adds tokens elementwise. None of them needs to see more than one token at a time. They can therefore be split along the sequence dimension: with t devices, each performs the operation on s/t of the tokens and stores only those tokens' activations. The question is how to move between the two layouts. Inside the tensor-parallel regions, each device needs the full sequence and works on a slice of the hidden dimension. In the sequence-parallel regions, each device needs a slice of the sequence but the full hidden dimension. The transitions turn out to be exactly the two halves of the all-reduce that tensor parallelism was already performing. At the end of a tensor-parallel block, the row-parallel linear layer leaves each device with a partial sum of the full output. Tensor parallelism alone all-reduces these partial sums so every device holds the complete output. Sequence parallelism instead performs a reduce-scatter along the sequence dimension: each device receives the fully summed output for its own s/t tokens only. That is precisely what the following layer normalisation and dropout need. At the start of the next tensor-parallel block, the column-parallel linear layer needs the full sequence on every device, so an all-gather along the sequence dimension reassembles it. Since an all-reduce is a reduce-scatter followed by an all-gather, and each half moves the same data, replacing each all-reduce with a reduce-scatter and moving its all-gather to the start of the next block costs no additional communication. The same identity that made ZeRO stage 1 free makes sequence parallelism free. In the backward pass the roles reverse: the all-gather becomes a reduce-scatter and the reduce-scatter becomes an all-gather. The effect on memory is that every term of the activation formula is now divided by t. Korthikanti and colleagues' expression for a layer with tensor and sequence parallelism becomes s·b·h·(34/t + 5·a·s/(h·t)) bytes. Combined with selective recomputation of the attention core, it falls to 34·s·b·h/t. Their paper reported that this combination reduced activation memory by a factor of about five and eliminated more than 90 per cent of the time overhead of activation recomputation. For a 530-billion-parameter GPT-3-style model on 2,240 A100 GPUs, it raised model FLOPs utilisation to 54.2 per cent, from 42.1 per cent with full recomputation, a 29 per cent speedup obtained with no change in communication volume and no change to the mathematics of training. Sequence parallelism is now the default whenever tensor parallelism is used in Megatron-based frameworks, and it should be thought of as part of tensor parallelism rather than an independent axis. Its degree is always equal to the tensor-parallel degree, it uses the same group of devices, and it inherits the requirement that this group sit inside the fast NVLink domain. Why long contexts need more Tensor and sequence parallelism together divide the activations of a layer by the tensor-parallel degree, typically eight. That is enough for sequences of a few thousand tokens. It is not enough for the long-context training phases that became standard for frontier models in 2024 and 2025. Consider a 70-billion-parameter-class model with hidden size 8,192 trained at a sequence length of 131,072 tokens. With FlashAttention removing the quadratic term and selective recomputation in place, the activation memory per layer is about 34·s·b·h/t. For a single sequence with t = 8, that is 34 × 131,072 × 8,192 / 8, roughly 4.6 gigabytes per layer. With 80 layers, the activations for one sequence reach roughly 365 gigabytes per device, several times an H100's capacity, before any model state is counted. Full recomputation would bring the stored activations down, but the transient memory needed to recompute a single layer, and the extra arithmetic, make that unattractive, and the attention computation itself, quadratic in the sequence length, would still sit on eight GPUs. Raising tensor parallelism beyond eight would cross the server boundary, which Chapter 5 showed to be prohibitive. What is needed is a way to split the sequence across many more devices, potentially spanning servers, for the entire layer. For most of the layer, this is easy: the linear layers, normalisations, and activation functions operate on each token independently, so a device holding a slice of the sequence can apply them to its slice with no communication at all. The obstacle is attention. In self-attention every token's output depends on the keys and values of every other token, or, with a causal mask, every earlier token. A device holding one slice of the queries needs the keys and values of the whole sequence. Context parallelism is the art of providing them efficiently. Two designs dominate. Ring attention The first design passes keys and values around a ring. Its best-known statement is the Ring Attention paper of Liu, Zaharia, and Abbeel from 2023, which built on the blockwise computation of attention that FlashAttention had made standard. Divide the sequence into N contiguous chunks, one per device. Each device computes the queries, keys, and values for its own chunk. Attention is then computed in N rounds. In the first round, each device computes the attention of its queries against its own keys and values. Simultaneously, it sends its key and value chunk to the next device in the ring and receives one from the previous device. In the second round, it computes the attention of its queries against the received keys and values, while passing them on and receiving the next chunk. After N rounds, every device's queries have attended to every chunk of keys and values. Combining the partial results is possible because of the online softmax technique that FlashAttention uses: by keeping a running maximum and running sum of exponentials for each query, partial attention outputs computed against different blocks of keys can be rescaled and combined exactly, giving the same result as attention over the full sequence. Ring attention is therefore exact, not an approximation. The key question is whether the communication in each round can hide behind the computation. In each round a device computes attention between its chunk of c queries and a chunk of c keys and values, about 4·c²·h floating-point operations in the forward pass for hidden size h. It sends a chunk of keys and values, about 2·c·h_kv elements, where h_kv is the total width of the key and value heads. In BF16, that is 4·c·h_kv bytes. The computation takes longer than the communication when 4·c²·h/F > 4·c·h_kv/β, that is, when the chunk length c exceeds (F/β)·(h_kv/h). With standard multi-head attention, h_kv equals h and the threshold is simply F/β, about 8,000 tokens for an H100 communicating across servers at 50 gigabytes per second. With grouped-query attention, which shrinks the key and value width, eight-fold in many current models, the threshold falls to around a thousand tokens. The communication of ring attention thus overlaps comfortably with computation whenever each device's chunk is a few thousand tokens or more, and unlike tensor parallelism, ring attention can cross server boundaries. Because the computation per round grows with the square of the chunk length while communication grows linearly, longer sequences make overlap easier, not harder. One complication is the causal mask. In causal attention, each token attends only to earlier tokens. If the sequence is divided into contiguous chunks, the device holding the first chunk has almost nothing to compute in most rounds, since most key chunks come after its queries, while the device holding the last chunk must compute against every chunk. The work is badly unbalanced, and the whole ring runs at the pace of the busiest device. The standard remedy divides the sequence into 2N chunks and gives device i both chunk i and chunk 2N−1−i, pairing an early chunk with a late one so that every device has roughly the same amount of unmasked work. The Llama 3 paper describes exactly this arrangement for its context parallelism, and similar load-balanced or striped assignments appear in Megatron-Core and other frameworks. Llama 3 also illustrates that the ring is not the only option. Its implementation, rather than passing key and value chunks around a ring in overlapping rounds, all-gathers the full keys and values onto every device in the context-parallel group and then computes attention for the local query chunks. The paper explains the choice: an all-gather makes it easier to support complex attention masks, such as the document masks that prevent tokens in one packed document from attending to another, and grouped-query attention makes the key and value tensors small enough that the exposed communication is modest. This is a fair example of how engineering choices in this field depend on the specific model; with full multi-head attention, the same all-gather would be eight times larger. Ulysses: the all-to-all alternative The second design, introduced by Jacobs and colleagues at Microsoft in 2023 as DeepSpeed Ulysses, takes a different route. Rather than circulating pieces of the keys and values, it rearranges the data so that attention can be computed locally. Outside attention, each device holds a slice of the sequence for all attention heads. Just before attention, Ulysses performs an all-to-all on the queries, keys, and values that changes the layout: afterwards, each device holds the full sequence for a subset of the heads. Since attention heads are independent, each device can compute ordinary attention, with FlashAttention or any other kernel and any mask, over the full sequence for its own heads. A second all-to-all after attention restores the original layout, a slice of the sequence for all heads, ready for the output projection. The communication properties are attractive. In an all-to-all among N devices, each device sends about (N−1)/N of its data, and its data is a 1/N slice of the full tensor. If the sequence length and the number of devices grow together, the communication volume per device stays constant, which the Ulysses authors emphasised. The computation is unmodified attention, so there is no load-balancing problem with causal masks and no need for online-softmax recombination. Ulysses has one hard limit, and it is the same limit that tensor parallelism faces: the degree of parallelism cannot exceed the number of attention heads, since each device must hold at least one whole head. With grouped-query attention it is effectively limited by the number of key and value heads, often eight, unless they are replicated. And all-to-all is more sensitive to network topology than ring communication, since every device communicates with every other device rather than only its neighbours, which makes it best suited to the fast intra-server domain. The two designs are complementary, and several systems combine them. A unified scheme proposed by Fang and Zhao in 2024 uses Ulysses within a server, where the all-to-all runs over NVLink and the head count is not yet a constraint, and ring attention across servers, where neighbour-only communication and computational overlap suit the slower links. This is the hierarchy principle in miniature. Context parallelism in a full layout Context parallelism divides activations across its group, by the context-parallel degree, in every part of the layer. It does not, by itself, divide the weights: every device in a context-parallel group holds the same parameters, as in data parallelism, and processes different tokens. Its gradients must therefore be summed across the context-parallel group, just as data-parallel gradients are summed across replicas. Frameworks such as Megatron-Core handle this by folding the context-parallel group into the gradient reduction and parameter sharding, treating the context and data dimensions together as one larger group for that purpose. The Llama 3 405B training run offers a clean worked example of why context parallelism is introduced and how its degree is chosen. In the main pre-training phase, at sequence length 8,192, the run used no context parallelism: eight-way tensor parallelism, sixteen-way pipeline parallelism, and 128-way data parallelism on 16,384 GPUs. For the long-context phase, at sequence length 131,072, sixteen times longer, it introduced sixteen-way context parallelism. Each device's share of a sequence was therefore 131,072/16 = 8,192 tokens, exactly the per-device length of the short-context phase, so per-device activation memory was essentially unchanged. To keep the total number of GPUs and the number of tokens per batch constant, the data-parallel degree fell from 128 to 8. The reported throughput dropped from 400 to 380 teraflops per GPU, a modest cost for a sixteen-fold increase in context length. The pattern generalises. Context parallelism is added when, and only when, a single sequence's activations do not fit within a tensor-parallel group, and its degree is chosen to bring the per-device sequence length back to a comfortable value. It is drawn from the devices that would otherwise be used for data parallelism, which is acceptable because long sequences already contain many tokens, so fewer replicas are needed to reach the target batch size in tokens. The sequence dimension as a resource The broader lesson of this chapter is that the sequence dimension is a resource to be partitioned like any other, and that the cost of partitioning it depends almost entirely on attention. For everything outside attention, splitting the sequence is free; the transformer's position-wise operations need no communication. Sequence parallelism exploits that within the tensor-parallel group at zero added cost. Context parallelism exploits it across a wider group and pays only for attention, either by circulating keys and values around a ring with overlapped computation or by transposing the data with all-to-all so that attention becomes local. As contexts lengthen, attention's quadratic computation grows faster than the linear communication needed to distribute it, which makes context parallelism more efficient, not less, at exactly the lengths where it is needed. With tensor, sequence, and context parallelism, a layer's weights, arithmetic, and activations can all be divided across a group of devices. What remains is the depth of the network: models with a hundred or more layers, each requiring its share of memory, multiplied by the number of micro-batches in flight. Dividing the layers themselves is the subject of the next chapter. Chapter 7. Pipeline Parallelism and the Bubble Tensor parallelism splits each layer across devices and pays for it with communication inside every layer. Pipeline parallelism splits the model in the other direction, by depth. A model with 96 layers on eight devices becomes eight stages of twelve layers each; each device holds only its stage's parameters, gradients, and optimizer state, and communicates only with its neighbours, passing activations forward and gradients backward across the boundaries between stages. The communication is small and infrequent, so pipeline stages can be placed on different servers connected by the ordinary network. That is pipeline parallelism's great virtue and the reason nearly every very large dense training run uses it. Its great vice is idleness. A pipeline has a beginning and an end, and while the first stage starts work on the first batch, the last stage has nothing to do; when the last stage finishes the final backward pass, the first stage has long since run out of work. This idle time is called the bubble, and the history of pipeline parallelism is largely a history of schedules designed to shrink it without exhausting memory. The naive pipeline and the micro-batch The simplest way to split a model by depth is to place consecutive layers on consecutive devices and run the batch through them in order. Device 1 computes the forward pass of its layers and sends the output to device 2, which computes its layers and sends the output to device 3, and so on to the last device, which computes the loss and begins the backward pass, sending gradients back the way they came. At any moment, exactly one device is working. With p stages, each device is busy only 1/p of the time. This arrangement solves the memory problem, since each device holds 1/p of the model, but wastes nearly all the compute. The fix, introduced in Google's GPipe paper by Huang and colleagues in 2019, is to split each batch into m smaller micro-batches and feed them into the pipeline one after another. As soon as device 1 finishes the forward pass of micro-batch 1 and sends it on, it begins micro-batch 2, while device 2 processes micro-batch 1. After p−1 steps, all devices are busy, each working on a different micro-batch. In GPipe's schedule, every micro-batch's forward pass is completed first, then every micro-batch's backward pass, in reverse; gradients are accumulated across micro-batches and the optimiser step is applied once at the end, so the mathematics is identical to processing the whole batch at once. The pipeline still has to fill and drain. At the start of the forward phase, devices wait for their first input; at the end of the backward phase, devices finish at different times. If each stage's forward and backward work for one micro-batch takes a fixed time, the idle time on each device amounts to p−1 micro-batch slots out of m+p−1 total. The bubble's share of the step time is therefore (p−1)/(m+p−1), or, equivalently, the idle time is (p−1)/m relative to the useful computation. With eight stages and eight micro-batches, devices sit idle nearly half the time; with 32 micro-batches, about 18 per cent; with 64, about 10 per cent. GPipe's lesson is that the bubble is amortised by using many more micro-batches than stages. The cost of many micro-batches is memory. In GPipe's schedule, each device runs the forward pass of all m micro-batches before any backward pass, so it must store the activations of all m micro-batches at once. More micro-batches mean a smaller bubble but proportionally more activation memory. GPipe relied on activation recomputation to make this affordable, storing only each stage's inputs and recomputing the rest during the backward pass. With this approach it trained, among other models, a 128-layer, 6-billion-parameter multilingual translation transformer. One forward, one backward The memory problem was solved by reordering the work. In PipeDream, from Narayanan and colleagues at Microsoft and Stanford in 2019, and in its synchronous successor PipeDream-Flush, each device alternates between forward and backward passes as soon as the pipeline is full, a pattern known as one-forward-one-backward, or 1F1B. The schedule has three phases. In the warm-up phase, each device runs forward passes until the pipeline is full: the first stage runs p forward passes, the second p−1, and so on, with the last stage running just one. In the steady state, each device alternates one forward pass for a new micro-batch with one backward pass for the oldest micro-batch whose gradient has arrived from downstream. In the cool-down phase, the remaining backward passes drain. The bubble is the same as GPipe's: (p−1)/m relative to useful compute. What changes is memory. Because a device starts a backward pass, and frees that micro-batch's activations, as soon as it can, it never holds more than p micro-batches' activations at once, regardless of how many micro-batches the step contains. The memory cost of reducing the bubble by adding micro-batches disappears. With 1F1B, one can use as many micro-batches as the batch size allows, up to the point where each micro-batch becomes too small to keep the GPU efficient. The original PipeDream went further, running asynchronously without ever flushing the pipeline, which required keeping multiple versions of the weights so that each micro-batch's backward pass used the same weights as its forward pass. The asynchronous version changes the optimisation dynamics, and large-scale practice settled on the synchronous 1F1B schedule, which preserves the exact mathematics of mini-batch training. One asymmetry of 1F1B deserves attention. The first stage holds p micro-batches of activations at its peak, while the last stage holds only one. Memory usage is uneven across the pipeline, with the earliest stages the tightest. Practitioners compensate by assigning fewer layers to early stages, or by applying more aggressive recomputation there. Interleaved stages The next improvement, also from Narayanan and colleagues, in their 2021 paper on efficient large-scale training with Megatron-LM, reduced the bubble itself. The bubble arises because the pipeline takes p−1 steps to fill, and the length of each step is the time to process one micro-batch through one stage. Making the stages smaller makes the fill time shorter. In the interleaved schedule, each device holds not one contiguous block of layers but v smaller blocks, called virtual stages or model chunks, drawn from different parts of the network. With four devices and v = 2, a 16-layer model might be split so that device 1 holds layers 1–2 and 9–10, device 2 holds layers 3–4 and 11–12, and so on. A micro-batch travels around the devices twice, once through each round of chunks. Each virtual stage is v times smaller than a regular stage, so the pipeline fills v times faster, and the bubble falls to (p−1)/(v·m) relative to useful compute. The cost is communication. A micro-batch now crosses device boundaries v times as often, so point-to-point traffic rises by a factor of v. Since pipeline communication is small, this is usually affordable, but it is not free, and at large v it can begin to matter. The original schedule also required the number of micro-batches to be a multiple of the number of stages, a constraint that later implementations relaxed. The Llama 3 paper describes modifications of exactly this kind: an interleaved schedule with a tunable number of micro-batches, so that the schedule could work with whatever batch size the training dynamics required. Splitting the backward pass: zero-bubble schedules A conceptual advance came in 2023 from Qi and colleagues at Sea AI Lab, whose paper on zero-bubble pipeline parallelism observed that the backward pass of a layer is really two computations with different dependencies. The backward pass computes two kinds of gradients. The gradient with respect to the layer's input, call it B, must be passed to the previous stage, which is waiting for it; it is on the critical path of the pipeline. The gradient with respect to the layer's weights, call it W, is needed only at the end of the step, for the optimiser update. Nothing downstream depends on it. In a standard schedule, B and W are computed together. But W can be deferred: computed later, whenever the device would otherwise be idle. Deferring W gives the scheduler a supply of flexible work to pour into the bubbles. In the warm-up and cool-down phases, where devices would otherwise wait, they can compute deferred weight gradients. The paper presented a family of schedules. One, which it called ZB-H1, keeps the same peak activation memory as 1F1B and reduces the bubble to about one-third of 1F1B's. Another, ZB-H2, allows somewhat more activation memory, roughly twice that of 1F1B, and drives the bubble essentially to zero. The paper also offered an automatic scheduler that searches for the best arrangement given measured timings of forward, B, and W computations and a memory limit. Removing the bubble entirely exposed a further obstacle. Standard training performs global checks before the optimiser step, such as computing the gradient norm across all stages for clipping and checking for numerical overflow. These checks synchronise all stages and recreate a small bubble at the end of every step. The zero-bubble paper addressed this by performing the optimiser step optimistically and validating it afterwards, rolling back in the rare case that a check fails. Subsequent work by the same group introduced V-shaped schedules, in which each device holds two chunks arranged so that a micro-batch travels down the devices and back up, so that the first and last chunks of the model share a device. Combined with the B/W split, such schedules reach very small bubbles with memory comparable to 1F1B. DeepSeek's DualPipe, introduced for training DeepSeek-V3 in late 2024, is a further development in this line. It feeds micro-batches into both ends of the pipeline simultaneously, so that forward computations travelling in one direction can be paired with backward computations travelling in the other. Each forward and backward chunk is subdivided into attention, expert dispatch communication, feed-forward computation, and expert combine communication, and the schedule overlaps the computation of one with the communication of another. DualPipe was designed primarily to hide the heavy all-to-all communication of a mixture-of-experts model, discussed in Chapter 8, and it pays for its efficiency by keeping two copies of the model's parameters on each device, along with some additional activation memory. Table 5 summarises the main schedules. The bubble is given as idle time relative to useful computation per step, for p stages and m micro-batches, and the activation memory as the number of micro-batches whose activations the busiest device must hold. Table 5. Pipeline schedules compared: bubble overhead, peak activation memory on the busiest stage, and principal cost (from the GPipe, PipeDream, Megatron-LM, zero-bubble and DeepSeek-V3 papers). Schedule Bubble relative to compute Peak activations held Principal cost or constraint GPipe (p−1)/m m micro-batches Memory grows with micro-batch count 1F1B (PipeDream-Flush) (p−1)/m p micro-batches Uneven memory across stages Interleaved 1F1B, v chunks (p−1)/(v·m) Slightly above p v times more point-to-point traffic Zero-bubble ZB-H1 About one-third of 1F1B p micro-batches Requires splitting backward into B and W Zero-bubble ZB-H2 Close to zero About 2p micro-batches More memory; optimistic optimiser step DualPipe Smaller than 1F1B and ZB-H1 About p plus one micro-batch Two copies of parameters per device Communication and placement Pipeline parallelism's communication requirements are modest. At each stage boundary, the forward pass sends the activation tensor of one micro-batch, b·s·h elements, to the next stage, and the backward pass sends a gradient of the same size back. Compare that with the computation within a stage. If a stage holds k transformer layers, each micro-batch requires about 24·k·b·s·h² floating-point operations in the forward pass, while it sends 2·b·s·h bytes in BF16. The ratio of computation time to communication time is about 12·k·h·β/F. For a stage of eight layers with hidden size 16,384, on an H100 at 400 teraflops with 50 gigabytes per second of network bandwidth, the ratio is roughly 12 × 8 × 16,384 × 50 × 10⁹ / (400 × 10¹²), about 200. Pipeline communication takes about half a per cent of the computation time, even over the inter-server network, and it can be overlapped with computation by sending asynchronously. This is why pipeline parallelism goes across servers. Megatron-Turing NLG's 530-billion-parameter model used 35-way pipeline parallelism spanning DGX A100 servers, combined with eight-way tensor parallelism inside each server. Llama 3 405B used 16-way pipeline parallelism across servers. When pipeline parallelism is combined with tensor parallelism, a further optimisation is available. The activation tensor at a stage boundary is replicated across the eight tensor-parallel ranks of a server, since each holds a full copy of the layer's output. Sending eight identical copies across the network would be wasteful. Instead, each rank sends only one-eighth of the tensor to its counterpart in the next stage, using its own network interface, and the receiving server reassembles the full tensor with an all-gather over NVLink. With sequence parallelism this happens naturally, since the activations are already sharded along the sequence. The cross-server traffic falls by the tensor-parallel degree, and it travels on the parallel rails of the network. Balancing the stages A pipeline runs at the pace of its slowest stage. If one stage takes 10 per cent longer than the others, every stage waits for it on every micro-batch, and the whole pipeline loses 10 per cent. Balancing the stages is therefore critical, and it is harder than it looks. The layers of a transformer are nearly identical, but the ends of the model are not. The first stage must perform the embedding lookup; the last must compute the output projection over the whole vocabulary and the loss. With a vocabulary of 128,000 tokens and a hidden size of 16,384, the output projection alone is comparable in arithmetic to several transformer layers, and its logits are large in memory. Llama 3's answer, reported in its paper, was to remove one transformer layer from each of the first and last stages. Other systems assign the embedding and output layers to their own stages, or split the layers unevenly to equalise measured time. Memory imbalance compounds the time imbalance. Under 1F1B, the first stage holds activations for p micro-batches while the last holds one, so the first stage may need more recomputation, which makes it slower. Achieving a balanced pipeline therefore usually involves profiling and adjusting the layer assignment, recomputation policy, and sometimes the stage count until the stages take equal time under the memory limit. Pipelines and the batch size Pipeline parallelism couples tightly with the batch size. To keep the bubble small, each pipeline needs many micro-batches per step: with 1F1B and p = 16, a bubble of 10 per cent requires about 150 micro-batches; with interleaving at v = 4, about 38. The micro-batch cannot be made arbitrarily small, since a micro-batch of fewer than one sequence is meaningless without context parallelism, and very small micro-batches give poor GPU efficiency. So each pipeline replica needs a substantial number of sequences per step. But Chapter 3 established that the global batch is bounded by the critical batch size. The number of sequences per replica is the global batch divided by the data-parallel degree. Increasing the data-parallel degree to use more GPUs reduces the micro-batches per pipeline and enlarges the bubble. The Llama 3 paper shows this tension in its own numbers. Going from 8,192 to 16,384 GPUs at a constant 16 million tokens per batch, the data-parallel degree doubled from 64 to 128 and the batch per data-parallel replica halved from 32 sequences to 16. The reported throughput per GPU fell from 430 to 400 teraflops, which the paper attributes to the smaller batch per replica, that is, to the larger pipeline bubble relative to useful work. This coupling between pipeline depth, data-parallel degree, and batch size is the central constraint of large-scale layout design, and it is one reason that improved pipeline schedules matter so much. Each halving of the bubble allows the same efficiency with half as many micro-batches per replica, and therefore twice as many replicas within the same global batch. What pipelines provide Pipeline parallelism divides a model's depth across devices with very little communication, which lets it span servers and scale to large numbers of stages. It divides the parameters, gradients, and optimizer state by the number of stages, and it divides the activations, though each stage holds activations for several micro-batches at once. Its cost is the bubble, which schedules from GPipe through 1F1B, interleaving, and zero-bubble methods have shrunk from an inescapable tax to a manageable overhead, and a sensitivity to imbalance between stages. It requires many micro-batches per step, which ties it to the global batch size and the data-parallel degree. With tensor, sequence, context, pipeline, and data parallelism, the dense transformer can be fully partitioned. But some of the largest models now in training are not dense. Their sparse, expert-based structure introduces a fifth axis with its own communication pattern, the subject of the next chapter. Chapter 8. Expert Parallelism: Mixture-of-Experts and the All-to-All Every technique in the preceding chapters was developed for dense models, in which every parameter participates in processing every token. A growing share of the largest models are not dense. In a mixture-of-experts (MoE) model, the feed-forward block of some or all layers is replaced by a collection of smaller feed-forward networks, the experts, together with a small router that sends each token to only a few of them. The model can then have a very large number of parameters while spending only a modest amount of computation on each token. That decoupling of parameter count from compute is the reason for MoE's appeal, and it is also why MoE models need their own form of parallelism with a communication pattern unlike any other in this book. Sparse models and why they are attractive The modern lineage of MoE in deep learning begins with the sparsely gated mixture-of-experts layer of Shazeer and colleagues at Google in 2017, which inserted thousands of experts between recurrent layers. Google's GShard, described by Lepikhin and colleagues in 2020, brought experts into the transformer and trained a 600-billion-parameter translation model on 2,048 TPU v3 cores, each token routed to two experts. The Switch Transformer of Fedus, Zoph, and Shazeer in 2021 simplified routing to a single expert per token and scaled to over a trillion parameters. Mistral's Mixtral 8x7B, released in December 2023, made MoE prominent among openly available models, with eight experts per layer and two active per token. DeepSeek-V3, released in December 2024, pushed the design further: 671 billion parameters in total, of which about 37 billion are activated for each token, with each MoE layer containing 256 routed experts and one shared expert that every token uses, and eight routed experts selected per token. The appeal is that, for a fixed computational budget per token, a sparse model can hold far more parameters, and in practice this gives better quality for the same training compute. The cost, from a systems perspective, is that those parameters must live somewhere, and each token must be brought to the experts it has chosen. The structure of expert parallelism Consider a single MoE layer with E experts on a group of E devices, one expert per device. Before the layer, each device holds the hidden states of its own tokens, the micro-batch it is processing. The router on each device computes, for each of its tokens, which k experts should process it. Now the tokens must move. Each device sends each of its tokens to the devices holding that token's chosen experts. Each device then has a collection of tokens from every device in the group, all destined for its expert. It applies its expert to them, a feed-forward computation, and sends each result back to the device the token came from, which combines the k results for each token, weighted by the router's scores. The two movements, called dispatch and combine, are each an all-to-all: every device sends a different chunk of data to every other device. In the backward pass, gradients follow the reverse paths, requiring two more all-to-alls. Each MoE layer therefore performs four all-to-all operations per micro-batch, two in the forward pass and two in the backward. This structure has several properties that distinguish it from the other forms of parallelism. First, expert parallelism divides the expert parameters, gradients, and optimizer state across the expert-parallel group, but it does so without dividing the computation of any single expert, which runs whole on its device. Each device computes only for the tokens routed to it. Second, the attention blocks and other non-expert parameters are typically replicated across the expert-parallel group and treated as data parallel, since each device processes different tokens through them. An expert-parallel group is therefore usually carved out of the data-parallel dimension: devices that are data-parallel replicas for attention are simultaneously an expert-parallel group for the MoE layers. The gradients of each expert need to be reduced only among the devices that hold copies of that same expert, a smaller group than the full data-parallel group. Third, the communication volume and pattern depend on the router's decisions, which change from batch to batch. The amount of data each device sends and receives is not fixed in advance, and if the router sends too many tokens to one expert, that expert's device becomes the bottleneck for the whole group. The arithmetic of the all-to-all The intensity problem of expert parallelism becomes clear in a back-of-the-envelope calculation using DeepSeek-V3's published dimensions. The model's hidden size is 7,168 and each routed expert has an intermediate size of 2,048, using a gated feed-forward with three weight matrices. For one token passing through one expert, the forward computation is about 2 × 3 × 7,168 × 2,048, roughly 88 million floating-point operations. To get there and back, the token's hidden vector, 7,168 elements, must be dispatched and the result combined, which in BF16 is about 14 kilobytes each way, 28 kilobytes in total. On a GPU sustaining 400 teraflops, 88 million operations take about 0.22 microseconds. Over a 50-gigabyte-per-second network link, 28 kilobytes take about 0.56 microseconds. The communication takes more than twice as long as the computation it serves. For fine-grained experts, expert parallelism across servers is fundamentally communication-bound, unlike pipeline parallelism, where computation outweighs communication by two orders of magnitude, or data parallelism, where computation wins whenever the batch per replica is reasonable. The cause is the ratio of data moved to work done. A dense feed-forward block performs a large matrix multiplication for each token's vector, but the vector stays on the device. An expert performs a smaller matrix multiplication and the vector must travel. As MoE designs move toward many small experts, which research has found improves quality, the ratio worsens. The trend that makes MoE models better makes them harder to train. How DeepSeek-V3 made it work DeepSeek-V3 offers the most detailed public account of training a large fine-grained MoE model efficiently, and its techniques are a catalogue of responses to this arithmetic. The model was trained on 2,048 NVIDIA H800 GPUs, a variant of the H100 with reduced NVLink bandwidth sold for the Chinese market. Its technical report describes 16-way pipeline parallelism, 64-way expert parallelism spanning eight servers, and ZeRO-1 data parallelism, with no tensor parallelism at all; the team engineered memory savings so that tensor parallelism was unnecessary. The first technique is node-limited routing. The router is constrained so that each token is sent to experts on at most four servers. A token is sent across the InfiniBand network only once to each destination server and then forwarded over NVLink to the specific GPUs holding its chosen experts within that server. Since the report states NVLink bandwidth of 160 gigabytes per second against InfiniBand's 50 on these machines, cross-server traffic is the scarce resource, and the routing restriction caps it per token while allowing several experts per server to be reached at little extra cost. The second is overlap through scheduling. The DualPipe schedule described in Chapter 7 pairs the forward computation of one micro-batch with the backward computation of another and splits each into attention, dispatch, expert computation, and combine phases, so that the all-to-all communication of one micro-batch runs while another micro-batch computes. The report describes this as achieving near-complete overlap of the all-to-all communication, with a computation-to-communication ratio of roughly one to one, a regime where overlap is the only alternative to halving throughput. The third is dedicated communication hardware within the GPU. The team wrote custom all-to-all kernels and assigned 20 of each H800's 132 streaming multiprocessors to communication, a fixed reservation that keeps communication progressing without contending for resources with the computation kernels. DeepSeek later released a version of this work as an open-source library, DeepEP. The fourth is lower-precision communication. DeepSeek-V3 was trained with FP8 arithmetic for most matrix multiplications, and the activations dispatched to experts were quantised to FP8 before transmission, roughly halving the dispatch volume compared with BF16. The result, according to the report, was a total training cost of about 2.79 million H800 GPU-hours for the full model including pre-training, context extension, and post-training, a figure that drew wide attention because it was far below what many observers had assumed such a model required. The engineering lesson is that a fine-grained MoE is trainable at high efficiency, but only by treating communication as the primary design constraint and shaping the model, the routing, the schedule, and the kernels around it. Load balance Expert parallelism has a failure mode that has no analogue in the dense techniques. If the router sends a disproportionate share of tokens to a few experts, the devices holding those experts must process more tokens, receive more data in the dispatch, and send more in the combine, while other devices sit idle. Since every MoE layer synchronises the group twice in each direction, the whole group runs at the pace of its most loaded device. Imbalance also wastes model capacity, since rarely chosen experts barely learn, and it can become self-reinforcing as favoured experts improve and attract still more tokens. The traditional remedies are auxiliary losses and capacity limits. The Switch Transformer added a load-balancing loss term that penalises uneven routing, pushing the router toward a uniform distribution of tokens across experts. It also set a capacity factor: each expert accepts at most a fixed number of tokens per batch, the expected share multiplied by a factor somewhat above one, and tokens beyond that capacity are dropped, passing through the layer unchanged by the residual connection. Capacity limits make the all-to-all volumes predictable and bound the worst case, which simplifies the systems work, but dropped tokens are a loss of training signal, and the balancing loss pulls the router away from what would be best for model quality. More recent work has moved away from both. MegaBlocks, from Gale and colleagues in 2022, reformulated MoE computation as block-sparse matrix multiplication, allowing every expert to process however many tokens it receives without padding or dropping, so-called dropless MoE. DeepSeek-V3 replaced the auxiliary loss with a bias term added to each expert's routing score, adjusted after each step to raise the scores of underused experts and lower those of overused ones. The bias affects only which experts are selected, not how their outputs are weighted, and the report found that this balanced the load without the quality penalty of a strong auxiliary loss. DeepSeek also balances at the level of devices and servers, not just individual experts, since it is the per-device load that determines the all-to-all's duration. Even with balanced routing on average, the load varies from batch to batch, and the systems design must handle bursts. Some frameworks replicate the most popular experts on additional devices, and DeepSeek described deploying redundant copies of high-load experts during inference for the same reason. Combining experts with the other axes Expert parallelism composes with the other forms of parallelism, but the composition differs from that of dense models in an important way: the MoE layers and the attention layers of the same model have different preferences. Attention layers in an MoE model are like those of a dense model of the same hidden size: they benefit from tensor parallelism if they are large, from context parallelism if sequences are long, and from data parallelism. Expert layers want expert parallelism, ideally within the fastest domain available, and may also want tensor parallelism within each expert if experts are large. Forcing both onto the same arrangement of devices is wasteful. Recent versions of NVIDIA's Megatron-Core address this by allowing the expert layers to use a different decomposition of the same GPUs than the attention layers, so that, for example, a group of GPUs that runs attention with tensor and context parallelism can run the expert layers with a wider expert-parallel group. Hardware with larger NVLink domains changes the calculation significantly. In an eight-GPU server, an expert-parallel group of more than eight devices must cross the network, where the all-to-all is communication-bound. In a 72-GPU NVL72 rack, an expert-parallel group of 64 can run entirely over NVLink at many times the bandwidth. Much of the commercial interest in rack-scale NVLink systems is driven by exactly this: large mixture-of-experts models, in both training and inference, are the workload that most benefits from an all-to-all within a large fast domain. Expert parallelism also affects pipeline parallelism. Because each MoE layer synchronises all devices in its expert group, any variation in the expert computation, from load imbalance or from variations in token counts, propagates into stage timing and enlarges pipeline bubbles. Schedules that overlap all-to-all communication with other micro-batches' computation, like DualPipe, address both problems at once. The fifth axis Expert parallelism is the fifth axis of partitioning, after data, tensor and sequence, context, and pipeline. Its communication pattern is the all-to-all, its volume per unit of computation is higher than that of any other axis, and its load is determined by a learned router rather than by the shape of the model. It is most effective within the fastest interconnect available, it must be balanced both statistically and in its schedule, and it is the main reason that recent training systems invest so heavily in overlapping communication with computation. With all five axes in hand, the question becomes how to combine them: how many of each, in what order, mapped onto which parts of the hardware. That is the subject of the next chapter. Chapter 9. Composing the Layout: 3D, 4D and Beyond The previous chapters introduced each form of parallelism in isolation. No large training run uses only one. The 405-billion-parameter Llama 3 model, the 530-billion-parameter Megatron-Turing NLG, and DeepSeek-V3 each combined three or four forms, arranged in a particular order across the machines. This combination is often called 3D parallelism, for data, tensor, and pipeline, or 4D and 5D when context and expert parallelism are added. The name matters less than the method. This chapter sets out the rules for composing the axes, works through the planning of a concrete layout from first principles, and examines real configurations to see the rules at work. The device mesh The cleanest way to think about a composite layout is as a multi-dimensional grid of devices, called a device mesh. A job on 1,024 GPUs using eight-way tensor parallelism, four-way pipeline parallelism, and 32-way data parallelism arranges its GPUs in an 8 × 4 × 32 grid. Each GPU has a coordinate along each axis. All GPUs that share the same pipeline and data coordinates, and differ only in the tensor coordinate, form one tensor-parallel group of eight; they hold different slices of the same layers and communicate inside every layer. All GPUs that share the same tensor and data coordinates form one pipeline of four stages. All GPUs that share the same tensor and pipeline coordinates form one data-parallel group of 32; they hold the same slice of the same layers and reduce their gradients together. Every GPU belongs to exactly one group along each axis, and every collective operation runs within one group along one axis. Tensor-parallel all-reduces run within tensor groups, pipeline sends run along pipeline groups, gradient reductions run within data groups. The total number of GPUs is the product of the axis sizes. PyTorch's DeviceMesh, JAX's Mesh, and Megatron-Core's process-group initialisation all express this same structure. The crucial design decision is how the logical mesh maps onto the physical machine: which axes vary fastest across neighbouring GPUs, and therefore which groups share a server, a rack, or a leaf switch. Ranks are numbered so that consecutive ranks sit in the same server. The axis assigned to vary fastest, the innermost, will have its groups composed of GPUs within one server. The next axis will have groups that span neighbouring servers. The outermost axis will have groups that span the whole cluster. The ordering rule Each axis has a characteristic communication pattern, and Chapters 3 through 8 estimated how demanding each one is. Those estimates give the ordering rule directly: the axis with the most frequent, most latency-sensitive communication goes innermost, on the fastest links; the axis with the least frequent, most overlappable communication goes outermost, on the slowest. Tensor parallelism, with four blocking all-reduces per layer per micro-batch, is innermost, and its degree is bounded by the size of the NVLink domain. Sequence parallelism shares its group. Context parallelism comes next. Its ring communication overlaps with attention computation when chunks are long enough, and its all-gather variants are modest with grouped-query attention, so it can span servers, though it prefers to be close. Expert parallelism, where present, wants to be as close to the fast domain as possible, given its communication-bound all-to-all. In eight-GPU servers it usually spans several servers, with routing constrained to limit cross-server traffic; in rack-scale NVLink systems it fits inside the rack. Pipeline parallelism, with small point-to-point messages at stage boundaries, comfortably spans servers. Data parallelism, with one large overlappable reduction per step, or with sharded parameter gathers that can be prefetched, is outermost. The Llama 3 paper states its ordering explicitly as [TP, CP, PP, DP], from innermost to outermost, and justifies it in exactly these terms: tensor parallelism is innermost because it requires the highest bandwidth and lowest latency, and data parallelism, implemented as FSDP, is outermost because it tolerates latency through asynchronous prefetching and gradient reduction. One further benefit of this ordering is easy to miss. With tensor parallelism innermost and eight GPUs per server, every pipeline and data-parallel group connects GPUs with the same local index in different servers: GPU 3 in one server communicates with GPU 3 in another. On a rail-optimised network, described in Chapter 2, those GPUs share a rail and communicate through a single tier of switches. The standard ordering thus aligns cross-server traffic with the network's physical structure automatically. The sizing rules Ordering decides where each axis goes. Sizing decides how large each is. Four rules, applied in sequence, cover most cases. First, use the smallest amount of model parallelism that makes the model fit. Tensor, pipeline, and context parallelism all reduce efficiency in some way: tensor parallelism through blocking communication and thinner matrices, pipeline parallelism through the bubble, context parallelism through attention communication. Data parallelism, by contrast, is nearly free when each replica has enough work. So the model-parallel degree should be the minimum that fits memory, and all remaining GPUs should go to data parallelism. Second, among the ways to reduce memory, prefer the cheapest. ZeRO stage 1 is essentially free and should almost always be used. Selective activation recomputation costs a few per cent and should be used before adding model parallelism. Full sharding of parameters costs extra communication but no bubble, and is preferable to pipeline parallelism when the batch per replica is large enough to hide it. Tensor parallelism within a server is preferable to tensor parallelism across servers in every case. Third, check the batch. The global batch in tokens, divided by the data-parallel degree, gives the tokens per replica. That number must be large enough to hide data-parallel communication, as Chapter 3 showed, and, if pipeline parallelism is used, large enough to provide many micro-batches per pipeline, as Chapter 7 showed. If it is not, either the global batch must grow, which is limited by the critical batch size, or the model-parallel degree must grow, which reduces the number of replicas. Fourth, measure and adjust. The estimates in this book give the right shape of the answer and eliminate bad configurations quickly, but the final choice depends on kernel efficiencies, the actual achieved bandwidths, and memory details that only a short profiling run reveals. A worked example Consider a concrete planning problem: a dense 70-billion-parameter transformer, with 80 layers, hidden size 8,192, 64 query heads and 8 key-value heads, to be trained at a sequence length of 8,192 tokens on 1,024 H100 GPUs in 128 eight-GPU servers. The target global batch is 4 million tokens, 512 sequences per step, a common choice for a model of this size early in training. Model state. By Table 1, the model state is 1,120 gigabytes. It must be divided at least fifteen-fold just to fit 80-gigabyte GPUs, and much further to leave room for activations. Option A: fully sharded data parallelism alone. Shard everything across all 1,024 GPUs. The model state per GPU is about 1.1 gigabytes. Activations, with selective recomputation and no tensor parallelism, are about 34·s·h bytes per layer per sequence, 34 × 8,192 × 8,192, about 2.3 gigabytes per layer, or 182 gigabytes for 80 layers. That does not fit. With full recomputation, storing only each layer's input, the activations drop to about 2·s·h per layer, roughly 11 gigabytes for the model, plus transient memory for recomputing one layer at a time. That fits. But check the batch. With 1,024 data-parallel replicas and 512 sequences, each GPU gets half a sequence per step, which is impossible without context parallelism; even with a batch of 1,024 sequences, each GPU processes 8,192 tokens. Chapter 4's threshold for fully sharded communication to be hidden was about 8,000 tokens per GPU, and full recomputation adds a third to the compute. Option A is at best marginal at a doubled batch and infeasible at the target batch. Fully sharded data parallelism alone is excellent for models of this size on a few hundred GPUs with a generous batch, but not here. Option B: tensor parallelism within each server, with sharded data parallelism across servers. Use eight-way tensor parallelism with sequence parallelism inside each server, so that each server holds one model replica, and 128-way data parallelism across servers with ZeRO stage 1. Per GPU, the 16-bit parameters and gradients cost 4 × 70 / 8 = 35 gigabytes; the optimizer state, sharded across both the tensor and data groups, costs 12 × 70 / (8 × 128), under one gigabyte. Activations with tensor and sequence parallelism and selective recomputation are 34·s·h/8 per layer, about 285 megabytes, or 23 gigabytes for 80 layers per sequence. The total is around 59 gigabytes before buffers, which fits in 80 with a micro-batch of one sequence. Each replica processes 512/128 = 4 sequences per step, about 33,000 tokens, comfortably above the data-parallel threshold because the replica's gradient is divided among its eight GPUs. Tensor-parallel communication is the main overhead; Chapter 5's estimate put it at up to a quarter of compute time for this hidden size if not overlapped, less with communication overlap and NVLink reductions. Option C: add pipeline parallelism. Use eight-way tensor parallelism, four-way pipeline parallelism, and 32-way data parallelism. Each GPU now holds a quarter of the layers, so the 16-bit parameters and gradients fall to under 9 gigabytes. Activations depend on the schedule: under 1F1B, the first stage holds up to four micro-batches' worth of activations for its 20 layers, again about 23 gigabytes. The memory freed can be spent on a larger micro-batch, on dropping selective recomputation, or on reducing tensor parallelism to four, which halves its relative cost. Each replica now processes 512/32 = 16 sequences per step. With one sequence per micro-batch, 16 micro-batches across 4 stages gives a 1F1B bubble of 3/16, about 19 per cent of useful compute, which is too high. With interleaving at five virtual stages per device, each holding four layers, the bubble falls to 3/(5 × 16), under 4 per cent. The choice. Option B is simpler and fits; its costs are tensor-parallel communication and tight memory. Option C uses memory more comfortably and opens the possibility of reducing tensor parallelism, at the cost of a small bubble and the complexity of pipeline scheduling. Both are reasonable starting points, and a team would profile both. If the same model were later trained with a 128,000-token context, sixteen-way context parallelism would be added, drawn from the data-parallel dimension exactly as Llama 3 did, to bring the per-GPU sequence back to 8,192 tokens. Scaling up. Now suppose the same model is to be trained on 8,192 GPUs at the same 4-million-token batch. Option B would require 1,024 data-parallel replicas for 512 sequences, impossible. Option C with eight-way tensor and eight-way pipeline parallelism would give 128 replicas of 64 GPUs, each processing four sequences: four micro-batches across eight stages, a bubble far exceeding the useful work. The only remedies are to raise the global batch, perhaps to 16 million tokens, giving each replica 16 sequences and an interleaved bubble of about 9 per cent, or to accept that this many GPUs cannot be usefully employed on this model at this batch size. This is the central lesson of the coupling between batch, pipeline, and data parallelism: adding GPUs to a job is not always possible without changing the training recipe, and beyond a point, more hardware buys less speed. Real configurations Table 6 collects the published layouts of several well-documented large training runs. They span four hardware generations and both dense and sparse models, and they illustrate the rules above. Table 6. Published parallel layouts of large training runs and their reported per-device throughput (from the Megatron-LM, Megatron-Turing NLG, PaLM, Llama 3 and DeepSeek-V3 papers and reports). Run Hardware Tensor Pipeline Other axes Reported throughput Megatron-LM 1T (2021) 3,072 A100 8 64 Data 6 163 TFLOP/s per GPU, 52% of peak Megatron-Turing NLG 530B (2021) 2,240 to 3,360 A100 8 35 Data 8 to 12 113 to 126 TFLOP/s per GPU PaLM 540B (2022) 6,144 TPU v4 12 (model) None Fully sharded data 256 per pod; 2 pods 46.2% model FLOPs utilisation Llama 3 405B, 8K context (2024) 16,384 H100 8 16 Data 128 400 TFLOP/s per GPU, 41% Llama 3 405B, 128K context (2024) 16,384 H100 8 16 Context 16; data 8 380 TFLOP/s per GPU, 38% DeepSeek-V3 671B MoE (2024) 2,048 H800 None 16 Expert 64; ZeRO-1 data Not reported per GPU Several patterns stand out. Every GPU-based dense run used tensor parallelism of exactly eight, the size of the server's NVLink domain. Pipeline parallelism, from 16 to 64 stages, spans servers. Data parallelism takes whatever remains, and its degree varies with cluster size, as Megatron-Turing NLG's range shows. Throughput per GPU falls modestly as the data-parallel degree rises at fixed batch, as both Megatron-Turing NLG and Llama 3 report. PaLM is the instructive exception among the dense runs. Google's TPU v4 pods connect thousands of chips with a high-bandwidth inter-chip interconnect, so a much larger fast domain is available, and PaLM used no pipeline parallelism at all, relying on 12-way model parallelism and fully sharded data parallelism within each pod and data parallelism across the two pods. Its 46.2 per cent model FLOPs utilisation was among the highest reported for its time. The lesson is not that pipeline parallelism is unnecessary, but that its necessity depends on the hardware hierarchy: where the fast domain is large enough, the costs of the bubble can be avoided. DeepSeek-V3 is the sparse exception. It used no tensor parallelism, relying on careful memory management, pipeline parallelism, and a large expert-parallel group to fit the model, and devoted its engineering to hiding the all-to-all communication of expert parallelism. The table also shows how the field has reported efficiency. Throughput per GPU in teraflops, and model FLOPs utilisation as a fraction of peak, are the standard measures. A dense run on a mature stack achieving 40 to 50 per cent of peak is considered good. The next chapter examines how these figures are defined and why they are harder to compare than they look. Automating the search The space of layouts is large. With five axes, each with several possible sizes, plus choices of micro-batch size, recomputation policy, pipeline schedule, and sharding stage, the number of combinations runs into thousands, and the best one depends on details of the model and hardware. Several research systems have attempted to search it automatically. Alpa, from Zheng and colleagues in 2022, formulated the choice of intra-operator parallelism, such as tensor splitting, and inter-operator parallelism, such as pipeline stages, as a hierarchical optimisation problem solved with integer programming and dynamic programming. Google's compiler-based approach, in which users annotate shardings and the compiler propagates them, automates the insertion of communication while leaving the high-level choices to the user. In practice, large production runs continue to rely mostly on hand-chosen layouts, guided by the rules in this chapter and validated by profiling. The search space is large, but the rules eliminate most of it quickly: tensor parallelism of the NVLink domain size or a divisor of it, pipeline parallelism just large enough to fit, context parallelism only for long sequences, and data parallelism for the rest. What remains is a handful of candidates that can be measured directly. Automated tools are most useful for unusual models, unusual hardware, or teams without deep experience, and they are improving. The layout as a negotiation A parallel layout is a negotiation among competing constraints. Memory pushes toward more model parallelism. Communication efficiency pushes toward less, and toward keeping chatty axes on fast links. The pipeline bubble pushes toward more micro-batches per replica, which pushes toward fewer replicas. The critical batch size pushes the other way, bounding the total. Hardware shapes the negotiation: a larger NVLink domain allows more tensor or expert parallelism without leaving the fast tier, faster networks loosen the data-parallel threshold, and more memory per device reduces the need for any model parallelism at all. The rules in this chapter turn this negotiation into a procedure. But a layout that works on paper must still survive months of operation on thousands of devices, where hardware fails, software hangs, and efficiency quietly erodes. That is the subject of the last chapter. Chapter 10. Running the Job: Failures, Checkpoints and Measured Efficiency A parallel layout is a design on paper. A training run is an operation that lasts weeks or months on thousands of devices, during which hardware fails, software hangs, networks congest, and efficiency erodes for reasons no one predicted. The difference between a well-designed layout and a successful training run is the operational engineering around it: measuring efficiency honestly, finding the bottleneck when the numbers disappoint, saving state often enough to survive failures without wasting the time saved, and detecting the failures that do not announce themselves. This chapter covers that engineering. It is less elegant than the theory of the preceding chapters, but at scale it determines as much of the final cost as the layout does. Measuring efficiency The standard measure of training efficiency is model FLOPs utilisation (MFU), introduced in Google's PaLM paper in 2022. It is the ratio of the throughput a job achieves, counted in the floating-point operations the model mathematically requires, to the hardware's theoretical peak. If a job processes a certain number of tokens per second, and the model requires a certain number of operations per token, roughly six times the parameter count for a dense transformer plus a term for attention, then the achieved operations per second divided by the peak of all the devices is the MFU. The definition deliberately counts only the operations the model requires, not those the system performs. If a job uses activation recomputation, the recomputed forward passes are real work for the hardware but are not counted in MFU, because a system that avoided recomputation would achieve the same training in less time. A related measure, hardware FLOPs utilisation (HFU), counts all operations performed, including recomputation. HFU is always at least as large as MFU and can be substantially larger; reporting HFU without saying so makes a system look better than it is. MFU is harder to compare across reports than it appears. The peak used in the denominator varies: some reports use dense BF16 peak, others the peak with structured sparsity, which on NVIDIA hardware is twice the dense figure and almost never relevant to training. FP8 training raises the hardware peak again, so the same achieved throughput yields a lower MFU against an FP8 peak than against a BF16 one. The attention term in the numerator is sometimes included and sometimes omitted, and for long sequences it is large. Clock speeds vary with power and thermal limits. A careful reader of any efficiency claim should ask which peak, which precision, whether attention was counted, and whether recomputation was excluded. With those caveats, the published numbers give a sense of what is achievable. Megatron-LM's 2021 trillion-parameter run reported 52 per cent of peak on A100s; PaLM reported 46.2 per cent MFU on TPU v4; Llama 3 405B reported 38 to 43 per cent BF16 MFU on H100s, depending on the configuration. A dense training job on a mature software stack achieving 40 to 50 per cent MFU on current hardware is performing well. Below 30 per cent, something is usually wrong. Above 60 per cent is rare for large models, because the parts of a transformer that are not large matrix multiplications, normalisation, activation functions, attention at modest lengths, and communication that cannot be hidden, consume a real share of time. A complementary measure is the fraction of wall-clock time spent making progress, which Google has called goodput. A job that runs at 45 per cent MFU when running but loses a fifth of its wall-clock time to failures, restarts, and recomputed work since the last checkpoint achieves an effective efficiency of 36 per cent. At large scale the second factor is as important as the first. Finding the bottleneck When a job's throughput disappoints, the cause is almost always one of a small number of things, and the tools for finding it are standard. The first tool is a timeline profiler. PyTorch's profiler and NVIDIA's Nsight Systems record every kernel launch, memory copy, and communication operation on every GPU, with timestamps, and display them as parallel tracks. A healthy timeline shows the compute stream almost continuously busy, with communication kernels running on separate streams underneath. An unhealthy one shows gaps: periods when the compute stream is idle. The location of the gaps identifies the cause. Gaps after each tensor-parallel block indicate exposed tensor-parallel communication. Gaps at the start and end of each step indicate pipeline bubbles. A long gap at the end of the backward pass indicates exposed data-parallel communication, usually the last gradient bucket or, with full sharding, a final reduce-scatter. Gaps between kernels throughout indicate the CPU failing to launch work fast enough, often fixed by CUDA graphs or by fusing small operations. The second tool is a direct test of the hardware's communication performance. NVIDIA's nccl-tests measure the bus bandwidth of each collective at each message size on a given set of GPUs. Run within a server, they verify NVLink; run across servers, they verify the network. A large cluster should be tested in subsets before a job is launched, because a single misconfigured switch port or degraded cable can slow every collective that crosses it. Comparing measured bus bandwidth with the link's rated speed identifies such problems quickly. The third is per-rank timing. In a synchronous job, every step takes as long as the slowest rank's step. Logging each rank's time for each phase, forward, backward, communication, and optimiser, and looking for outliers reveals stragglers: a GPU running slower because of thermal throttling, a host with a failing network card, or a node whose data loader falls behind. At thousands of ranks, there is almost always a straggler somewhere, and finding it is often worth more than any optimisation of the layout. The Llama 3 paper gives an unusual view of how subtle these effects can be. It reports a diurnal variation in throughput of one to two per cent, which it attributed to higher midday temperatures affecting GPU clock speeds. It also reports that when tens of thousands of GPUs simultaneously stop computing, for example while waiting for a checkpoint or a collective, and then simultaneously resume, the data centre's power consumption swings by tens of megawatts, stretching the limits of the power grid. At this scale, the training job is part of the electrical and thermal environment as much as it is software. Failure as the normal condition The most important operational fact about large-scale training is that failure is routine. If each device in a cluster fails independently, with some mean time between failures, then a cluster of N devices experiences failures N times as often as a single device. A component reliable enough to fail once in ten years, on average, will produce a failure somewhere in a 16,000-GPU cluster several times a week. And a synchronous training job typically cannot continue when any single participant fails: a collective operation that waits for a dead rank waits forever, until a timeout fires. The public record bears this out. The BigScience project, training the 176-billion-parameter BLOOM model on 384 A100 GPUs over about three and a half months in 2022, reported averaging one to two GPU failures per week, with checkpoints saved every three hours so that each crash cost about an hour and a half of training on average. Meta's OPT-175B paper, describing training on 992 A100 GPUs in 2022, reported at least 35 manual restarts and the cycling of over 100 hosts over about two months, and Meta published the team's day-by-day logbook, which remains one of the most candid accounts of what large-scale training is like in practice. At the next order of magnitude, the Llama 3 paper reported 466 job interruptions during a 54-day snapshot of 405B pre-training on 16,384 GPUs, of which 419 were unexpected, roughly one every three hours. The large majority of unexpected interruptions were attributed to hardware, with faulty GPUs and failures of their HBM3 memory the leading categories, followed by software bugs, network cables, and network adapters. Despite this, the team reported achieving more than 90 per cent effective training time, through automated detection, rapid restart, and careful checkpointing. These numbers make the design goal clear. Failures cannot be prevented at this scale; they must be survived cheaply. The cost of each failure is the time to detect it, plus the time to restart the job, plus the training progress lost since the last checkpoint. Each must be minimised. Checkpointing A checkpoint saves everything needed to resume training: the model parameters, the optimizer state, the data loader's position, the random number generator states, and the learning-rate schedule's progress. For a 70-billion-parameter model with Adam, the parameters and optimizer state alone are over a terabyte. For a 405-billion-parameter model, several terabytes. Writing this from one rank would be hopeless. In practice every rank writes its own shard: with tensor, pipeline, and sharded data parallelism, each GPU holds a distinct slice of the model state, and all of them write to a parallel file system or object store simultaneously. The aggregate bandwidth of the storage system, not of any single device, determines the time. PyTorch's Distributed Checkpoint and comparable facilities in other frameworks write sharded checkpoints with metadata that records how the shards map to the full tensors, allowing a checkpoint written with one layout to be loaded with another. This resharding matters because a job restarted after a failure may run on a different number of GPUs, and because later phases of training often change the layout, as Llama 3 did when it added context parallelism. How often to checkpoint is a classical optimisation, studied for supercomputing applications long before deep learning. Checkpointing too often wastes time writing checkpoints; checkpointing too rarely wastes time recomputing work lost to failures. If writing a checkpoint stalls training for C seconds and the cluster's mean time between failures is M seconds, the approximately optimal interval, given by a formula due to Young in 1974 and refined by Daly in 2006, is the square root of 2·C·M. At that interval, the time lost to writing checkpoints and the expected time lost to recomputation are roughly equal. The formula shows why making checkpoints cheap is so valuable. Suppose a cluster fails every three hours, 10,800 seconds, and a checkpoint stalls training for 60 seconds. The optimal interval is the square root of 2 × 60 × 10,800, about 1,140 seconds, or 19 minutes. The overhead from checkpoint writing is 60/1,140, about 5 per cent, and the expected lost work is half an interval per failure, another 5 per cent, for a total of roughly 10 per cent of all training time, before counting the time to restart. Now suppose the stall can be reduced to 5 seconds. The optimal interval becomes about 330 seconds, and the combined overhead falls to about 3 per cent. This is why large-scale training systems use asynchronous checkpointing. Training stalls only long enough to copy the model state from GPU memory to host memory, a few seconds over PCIe; the slow write to persistent storage then proceeds in the background while training continues. A further step keeps recent checkpoints in the host memory of other machines in the cluster, so that a job recovering from a single node's failure can reload from fast peer memory rather than from remote storage. Researchers at Rice University and Amazon described such a system, named Gemini (unrelated to Google's model of the same name), at the SOSP conference in 2023, reporting failure recovery more than an order of magnitude faster than reloading from remote storage. Restarting quickly The other half of failure cost is the time from failure to resumed training. That includes detecting the failure, which may take minutes if it manifests as a hang that only a collective-communication timeout catches; identifying and excluding the faulty node; allocating a replacement, usually from a pool of spares kept warm for this purpose; launching tens of thousands of processes; initialising the communication library, which builds rings and trees across the full job; loading the checkpoint; and restoring the data loader to the exact position. At small scale these steps take seconds. At 16,000 GPUs each can take minutes, and the sum can rival the training lost since the checkpoint. Large teams invest in every one: health checks that run continuously on idle and active nodes, timeouts tuned to detect hangs quickly without false alarms, faster communicator initialisation, and parallel checkpoint loading. Hangs deserve special mention because they are the hardest failures to diagnose. A collective that never completes gives no error message; it simply waits. The usual cause is that ranks disagree about which collective to perform next, because of a software bug, a data-dependent branch, or a rank that crashed silently. Tools such as PyTorch's NCCL flight recorder, which keeps a ring buffer of recent collective calls on each rank, let engineers compare what each rank was doing when the hang began; the Llama 3 paper reports using it for exactly this purpose. Failures that do not announce themselves The most dangerous failures produce no error at all. Silent data corruption occurs when a processor computes a wrong result without detecting it, because of a manufacturing defect, ageing, or marginal operating conditions that manifest only under particular workloads. Meta and Google have both published studies of silent data corruption in their CPU fleets, finding that a small fraction of machines produce wrong results under specific conditions. In a training job, a single corrupted computation can propagate through gradient reductions to every replica, leaving a subtle change in the model, a loss spike, or a divergence that appears to be a numerical instability. Defences include monitoring the loss and gradient norms for anomalies, checking that data-parallel replicas that should hold identical parameters actually do, and replaying suspicious steps deterministically on different hardware to see whether the result changes. Google's Gemini technical report described silent data corruption as a real concern at its scale and mentioned using deterministic replay to isolate faulty hardware. Deterministic training, in which the same inputs produce bitwise identical results, makes such diagnosis possible but costs some performance, because it rules out certain non-deterministic kernels and collective algorithms whose summation order depends on timing. Related to silent corruption, and often confused with it, are genuine numerical instabilities: loss spikes caused by the training dynamics rather than the hardware. These are common in large-model training. Standard responses include restarting from a checkpoint before the spike and skipping the data batches that preceded it, lowering the learning rate, and architectural changes such as normalising queries and keys in attention. Distinguishing hardware-caused anomalies from dynamics-caused ones is one of the more difficult judgements in large-scale training, and the ability to replay a step deterministically is the best tool for making it. Operations as part of the design The lessons of this chapter feed back into the layout. Asynchronous and in-memory checkpointing need host memory, which constrains offloading. Restart time grows with job size, which favours layouts that can be reloaded quickly at a different scale. Stragglers in a synchronous job argue for keeping each synchronisation group as small as possible, a consideration that favours hybrid sharding over full sharding at large scale. And the efficiency lost to failures can exceed the efficiency gained by a better layout; a team that improves MFU from 40 to 42 per cent but loses an additional 10 per cent of wall-clock time to slow recovery has gone backwards. The training of a large model is, in the end, a long-running distributed system, and the disciplines of reliable systems engineering, monitoring, fast failure detection, automated recovery, and careful measurement, are as much a part of it as the mathematics of parallelism. Conclusion The preceding chapters took apart a large training job axis by axis and then put it back together. Each technique was introduced with the same two questions: what does it divide, and what does that division cost in communication, on which links, and how much of it can be hidden? Having asked those questions of data, tensor, sequence, context, pipeline, and expert parallelism, and of the sharding that runs across them, it is possible to say several things that could not have been said at the start. What the arithmetic shows The first is that a handful of ratios decide nearly everything. Data-parallel communication can be hidden when each replica processes more than about (2/3)·F/β tokens per step, a threshold that does not depend on the model's size at all. Fully sharded data parallelism raises that threshold to about F/β. Tensor-parallel overhead scales as t·F/(h·β), so it rises with the parallel degree, falls with the model's width, and multiplies roughly ninefold when the links change from NVLink to the network. Pipeline communication is a fraction of a per cent of computation, but its bubble scales as (p−1)/(v·m). Ring attention hides its communication once each device's chunk exceeds about F/β times the key-value fraction of the hidden size. Fine-grained expert parallelism across servers moves more bytes than it computes on. In every one of these expressions the hardware enters through the same quantity: the ratio of a device's arithmetic speed to the bandwidth of the link it is using. That ratio has been rising for a decade, because accelerators have gained compute faster than networks have gained bandwidth per device. The direct consequence is that every threshold in this book has been moving in the same direction: more tokens per replica to hide data-parallel communication, longer chunks to hide context-parallel communication, more pressure to keep the chatty axes inside the fast domain. The engineering response has been to enlarge that fast domain, from eight GPUs in a server to 72 in an NVLink rack, and to put more of the communication work into the network itself, through in-switch reductions. The reader who knows the ratio for a new machine can predict how each part of a layout will fare on it, without waiting for the benchmarks. The constraint no hardware removes The second conclusion is less obvious and more important. The binding constraint on very large training runs is frequently not memory and not bandwidth but the batch size. Data parallelism needs enough tokens per replica to hide its communication. Pipeline parallelism needs enough micro-batches per replica to shrink its bubble. Both draw on a global batch that cannot grow without limit, because beyond the critical batch size additional examples per step stop buying proportional progress. Chapter 9's worked example showed what happens when these collide: a configuration that ran well on 1,024 GPUs could not use 8,192 at the same batch, and no choice of layout rescued it. Only changing the training recipe could. This has a practical meaning for anyone planning or buying compute. Doubling a cluster does not halve the time to train a given model unless the training recipe can absorb a larger batch, or the layout can absorb more model parallelism at acceptable cost. The relationship between cluster size and training time bends well before the hardware runs out, and where it bends depends on the model, the batch schedule, and the pipeline schedule as much as on the network. The steady improvement of pipeline schedules, from GPipe's bubble through interleaving to zero-bubble designs, matters precisely because each improvement moves that bend outward. So does context parallelism, which lets long sequences supply the tokens that more replicas would otherwise have needed. Models and systems are designed together The third conclusion is that the boundary between model architecture and systems engineering has largely dissolved. Grouped-query attention was introduced to make inference cheaper, but it also shrinks the key and value tensors that context parallelism must move, which is why Llama 3 could choose a simple all-gather over a ring. The number of key-value heads caps the tensor-parallel and Ulysses degrees. The width of the model sets the relative cost of tensor parallelism. The vocabulary size sets the imbalance at the ends of a pipeline. Fine-grained mixture-of-experts layers improve quality per unit of compute and simultaneously push expert parallelism into a communication-bound regime, which is why DeepSeek-V3's routing was constrained to four servers per token and why its pipeline schedule was built around hiding the all-to-all. The models that train most efficiently at scale are increasingly those whose architects understood the parallel layout before they fixed the architecture. What remains unsettled Several questions are open. Clusters of 100,000 GPUs and more were built in 2024 and 2025, and at that scale failures arrive every few minutes rather than every few hours; whether synchronous training, in which one failure stops everything, remains the right model at the next order of magnitude is a real question. Methods that synchronise less often, such as DiLoCo and its relatives, offer a way to train across loosely connected clusters or even separate data centres, but they change the optimisation itself and have not yet been proven at frontier scale in public. Lower-precision training, already at FP8 for matrix multiplications in some production runs and moving toward four-bit formats on the newest hardware, reduces both computation and communication but brings numerical risks that are still being mapped. Rack-scale NVLink domains reopen the question of how far tensor and expert parallelism should extend before the network takes over. And automated layout search, long a research topic, may become necessary as the number of axes and the variety of hardware grow past what experienced engineers can enumerate by hand. None of these developments changes the controlling idea of this book. Each new technique still divides some tensor along some axis, and each still pays for that division in communication over some tier of a hierarchy. What changes is the shape of the hierarchy and the price of each tier. What to do differently For the reader who must plan, operate, or evaluate a large training job, the argument of this book reduces to a few habits. Do the arithmetic before launching. Four numbers, the model state in bytes, the activation memory per sequence per layer, the tokens per replica, and the ratio of compute speed to the relevant link bandwidth, eliminate most bad configurations in minutes. The formulas in these chapters are approximations, but they are approximations of the right shape, and a configuration they rule out will not be rescued by tuning. Order the axes by bandwidth and size them by necessity. Put tensor parallelism inside the fastest domain, data parallelism outermost, and everything else in between according to how often and how urgently it must communicate. Use the least model parallelism that fits, and spend the remaining devices on data parallelism, subject to the batch. Measure what the hardware actually delivers. Test the fabric's bus bandwidth before trusting it, profile the timeline before optimising, and look for the straggler before blaming the layout. Count failures as part of efficiency. A job's real speed is its model FLOPs utilisation multiplied by its goodput. Asynchronous checkpointing, fast restart, and good diagnostic tools can recover more wall-clock time than any improvement in the layout. Read every efficiency claim, including your own, with care about which peak, which precision, and which operations were counted. Training a frontier model is one of the largest coordinated computations ever performed. It succeeds not because any device is powerful enough to do the work, but because the work has been divided so that thousands of devices, each holding a fragment, spend nearly all their time computing and almost none of it waiting for one another. That division is an engineering choice, made with arithmetic, and it can be made well by anyone who knows where every tensor lives and what it costs to move it. Glossary Activation recomputation (gradient checkpointing). Discarding intermediate activations during the forward pass and recomputing them during the backward pass, trading extra computation for reduced memory. Selective recomputation applies this only to cheap-to-recompute, memory-heavy parts of a layer such as the attention core. All-gather. A collective operation in which each rank contributes a shard and every rank receives the concatenation of all shards. All-reduce. A collective operation in which every rank contributes a buffer and every rank receives the elementwise sum of all buffers. Equivalent to a reduce-scatter followed by an all-gather. All-to-all. A collective operation in which every rank sends a distinct chunk of data to every other rank; a transpose of data across the group. Used in expert parallelism and Ulysses context parallelism. Bubble. Idle time in pipeline parallelism while the pipeline fills at the start of a step and drains at the end. Bus bandwidth. A measure of collective performance that adjusts the observed throughput for the algorithm's inherent traffic, so that it can be compared with link speed. For all-reduce, the algorithm bandwidth multiplied by 2(N−1)/N. Context parallelism. Splitting each sequence across devices for the entire transformer layer, with special handling of attention, to train on very long sequences. Critical batch size. The batch size beyond which increasing the batch gives diminishing reductions in the number of training steps needed. Data parallelism. Replicating the model on several devices, giving each different training examples, and combining gradients before each update. Device mesh. A multi-dimensional arrangement of devices in which each dimension is assigned to one form of parallelism. Expert parallelism. Distributing the experts of a mixture-of-experts layer across devices and routing tokens to them with all-to-all communication. FSDP (Fully Sharded Data Parallel). PyTorch's implementation of sharded data parallelism, equivalent in its default mode to ZeRO stage 3. Micro-batch. A subdivision of a training batch processed in one forward and backward pass; used for gradient accumulation and to fill pipelines. MFU (model FLOPs utilisation). The ratio of the floating-point operations a model requires, at the throughput achieved, to the hardware's theoretical peak. Excludes recomputation. NVLink and NVSwitch. NVIDIA's high-bandwidth GPU-to-GPU interconnect and the switch chips that connect all GPUs in a server or rack at full NVLink bandwidth. Pipeline parallelism. Dividing the layers of a model into sequential stages on different devices, passing activations forward and gradients backward between stages. Rank. One process, usually corresponding to one GPU, participating in distributed training. Reduce-scatter. A collective operation in which every rank contributes a full buffer and each rank receives one fully summed shard of the result. Sequence parallelism. In the Megatron sense, sharding the layer normalisation, dropout, and residual regions of a transformer layer along the sequence dimension within the tensor-parallel group, replacing each all-reduce with a reduce-scatter and an all-gather. Tensor parallelism. Splitting individual weight matrices across devices so that each computes part of every matrix multiplication, combining partial results with collectives inside each layer. ZeRO (Zero Redundancy Optimizer). A family of techniques that shard optimizer state (stage 1), gradients (stage 2), and parameters (stage 3) across data-parallel ranks to eliminate redundant copies. Further Reading The literature of distributed training lives mostly in papers and technical reports rather than books. The list below collects the primary sources on which the chapters draw, grouped by topic, with a note on what each contributes. Foundations Rajbhandari, Rasley, Ruwase and He, "ZeRO: Memory Optimizations Toward Training Trillion Parameter Models" (2019; SC 2020). The memory accounting of mixed-precision Adam and the three stages of sharded data parallelism. The clearest single statement of why data parallelism wastes memory and how to stop it. Shoeybi, Patwary, Puri, LeGresley, Casper and Catanzaro, "Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism" (2019). The column-and-row decomposition of the transformer layer that defines tensor parallelism. Short and readable. Narayanan et al., "Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM" (SC 2021). The paper that established 3D parallelism as standard practice, introduced the interleaved pipeline schedule, and analysed how the axes interact. Essential. Korthikanti et al., "Reducing Activation Recomputation in Large Transformer Models" (2022). The activation-memory formulas used throughout this book, sequence parallelism, and selective recomputation. Communication Patarasuk and Yuan, "Bandwidth Optimal All-reduce Algorithms for Clusters of Workstations" (Journal of Parallel and Distributed Computing, 2009). The proof that ring all-reduce is bandwidth optimal. Sergeev and Del Balso, "Horovod: Fast and Easy Distributed Deep Learning in TensorFlow" (2018). The ring all-reduce brought to mainstream deep learning. NVIDIA NCCL documentation and the nccl-tests repository. The practical reference for collective algorithms, tuning variables, and the definitions of algorithm and bus bandwidth. Data parallelism and batch size Li et al., "PyTorch Distributed: Experiences on Accelerating Data Parallel Training" (VLDB 2020). The design of DistributedDataParallel, including bucketing and overlap. Goyal et al., "Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour" (2017). The linear scaling rule and learning-rate warm-up. McCandlish, Kaplan, Amodei and the OpenAI Dota Team, "An Empirical Model of Large-Batch Training" (2018). The gradient noise scale and the critical batch size. Zhao et al., "PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel" (VLDB 2023). How ZeRO stage 3 became a core framework feature, with the engineering of prefetching, rate limiting, and hybrid sharding. Wang et al., "ZeRO++: Extremely Efficient Collective Communication for Giant Model Training" (2023). Quantised and hierarchical communication for sharded training. Pipelines Huang et al., "GPipe: Efficient Training of Giant Neural Networks Using Pipeline Parallelism" (NeurIPS 2019). Micro-batched pipelines and the bubble. Narayanan et al., "PipeDream: Generalized Pipeline Parallelism for DNN Training" (SOSP 2019). The one-forward-one-backward schedule. Qi, Wan, Huang and Lin, "Zero Bubble Pipeline Parallelism" (ICLR 2024). Splitting the backward pass into input and weight gradients to fill the bubble. Long sequences Dao et al., "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness" (NeurIPS 2022). The tiled attention kernel that removed the quadratic activation term and underlies ring attention. Liu, Zaharia and Abbeel, "Ring Attention with Blockwise Transformers for Near-Infinite Context" (2023). Context parallelism by circulating keys and values. Jacobs et al., "DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models" (2023). Context parallelism by all-to-all. Mixture of experts Lepikhin et al., "GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding" (2020) and Fedus, Zoph and Shazeer, "Switch Transformers" (2021). The foundations of large sparse transformers, routing, and load balancing. Gale, Narayanan, Young and Zaharia, "MegaBlocks: Efficient Sparse Training with Mixture-of-Experts" (2022). Dropless MoE via block-sparse computation. Case studies Meta, "The Llama 3 Herd of Models" (2024). The most detailed public account of a dense frontier-scale run: the four-dimensional layout, pipeline and context-parallel modifications, network design, and a frank record of failures. DeepSeek-AI, "DeepSeek-V3 Technical Report" (2024). The reference account of efficient fine-grained MoE training: DualPipe, node-limited routing, FP8 training, and custom communication kernels. Chowdhery et al., "PaLM: Scaling Language Modeling with Pathways" (2022). Training without pipeline parallelism on TPU pods, and the definition of model FLOPs utilisation. Smith et al., "Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B" (2022) and Zhang et al., "OPT: Open Pre-trained Transformer Language Models" (2022), with its published logbook. Earlier large runs documented with unusual candour about layout and operations. BigScience Workshop, "BLOOM: A 176B-Parameter Open-Access Multilingual Language Model" (2022), and the accompanying Hugging Face engineering posts on its Megatron-DeepSpeed training. Practical guides Hugging Face, "The Ultra-Scale Playbook: Training LLMs on GPU Clusters" (2025). A long, practical open guide with extensive benchmarks of the techniques covered in this book. The Megatron-Core, DeepSpeed, and TorchTitan repositories and documentation. The production implementations of the techniques described here, and the best place to see how the details are handled. Hashtags: #DistributedModelTraining #DistributedDeepLearning #TensorParallelism #PipelineParallelism #SequenceParallelism #ContextParallelism #DataParallelism #ModelParallelism #HybridParallelism #ExpertParallelism #ArtificialIntelligence #MachineLearning #DeepLearning #LargeLanguageModels #LLMTraining #LargeScaleTraining #MultiGPUTraining #GPUComputing #HighPerformanceComputing #ParallelComputing #DistributedComputing #ScalableAI #TrainingInfrastructure #GPUClusters #NVIDIAH100 #NVIDIABlackwell #NVLink #NVSwitch #InfiniBand #NCCL #CollectiveCommunication #AllReduce #ReduceScatter #AllGather #CommunicationOptimization #ZeROOptimizer #DeepSpeed #FullyShardedDataParallel #PyTorchFSDP #MegatronLM #GradientAccumulation #GradientCheckpointing #ActivationRecomputation #FlashAttention #MixedPrecisionTraining #MixtureOfExperts #PipelineScheduling #MemoryOptimization #TrainingEfficiency #AIInfrastructure
- Robotics and AI (Sensor Fusion and Path Planning)
Download the Book (PDF): Introduction: Machines That Must Admit What They Do Not Know A warehouse robot carrying a shelf of phone chargers does not know where it is. It has a very good guess. The guess comes from a laser scanner sweeping the aisles forty times a second, from encoders counting wheel revolutions, from an inertial unit sensing every lurch and turn, and perhaps from a camera reading a code on the floor. None of these sources is right. Each is wrong in its own characteristic way, and the robot's software spends most of its working life reconciling them into a position estimate that is good to a centimetre or two, together with an honest statement of how much that estimate should be trusted. Then, using that estimate and a map it may have built itself, it decides where to drive, how fast, and what to do about the person who has just stepped out from behind a rack. A delivery drone faces the same problem in three dimensions, with less margin. It cannot stop and think in mid-air. Its satellite position fix can drift by metres near buildings or vanish under a bridge. Its cameras are blinded by low sun and confused by blank walls. Its battery sets a hard limit on how long it can hesitate. And the consequence of a wrong guess is not a bumped shelf but an aircraft falling onto a street. This booklet is about the software that makes such machines work: the engineering that turns noisy streams of LiDAR returns, camera frames and inertial measurements into a model of the world, a position within it, and a safe path through it. Its subject is the three capabilities named in the title and description, sensor fusion, simultaneous localization and mapping, and path planning with collision avoidance, as they are actually built for the two kinds of autonomous system that have moved fastest from laboratory to deployment: mobile robots in warehouses and small uncrewed aircraft. The argument The controlling idea of the book is simple to state and harder to live by. An autonomous physical system is a machine for managing uncertainty, and it works only to the extent that every layer of its software represents what it does not know and passes that knowledge honestly to the next layer. Fusion is the business of combining uncertain measurements so that the result is less uncertain than any of them. Localization and mapping is the business of estimating two unknown things, the world and the robot's place in it, from evidence that constrains both at once. Planning and collision avoidance are the business of acting well when the map is incomplete, the robot's position is approximate, and other agents are unpredictable. The engineering consequence follows directly. The most damaging failures in these systems rarely come from a bad algorithm in isolation. They come from the seams: a camera timestamp that is twenty milliseconds late, so that fused points land on the wrong object; a localization module that reports a confident pose when it has quietly snapped to the wrong aisle; a planner that treats the map as truth and plans a path through a gap that only exists because a pallet was not scanned. Each module may be individually excellent. The system fails because one of them hid its uncertainty from the others. Readers who have met robotics only through demonstrations tend to imagine that the hard part is intelligence, the clever decision at the critical moment. In practice the hard part is epistemics: knowing how much to believe each sensor, knowing when the map has gone stale, knowing that the estimate is wrong before acting on it. The cleverness that matters most is the cleverness of a well-designed estimator that notices its own confusion. Who this is for The booklet is written for engineers, technical managers and informed readers who want to understand how these systems are put together without first reading a shelf of textbooks. It assumes comfort with the idea of a probability distribution and a matrix, but derives nothing at length. Where mathematics carries the idea, the idea is stated in words first; the symbols, where they appear at all, are a compact restatement rather than the argument. It is not a survey. The field produces thousands of papers a year, and a short book that tried to name them all would teach nothing. Instead the book follows the path of information through a working system, from photons and laser pulses arriving at a sensor to wheel commands leaving a controller, and stops at each stage long enough to explain what problem that stage solves, how the established methods solve it, and where they characteristically break. The methods chosen are the ones that working engineers actually deploy or that define the vocabulary of the field: the Kalman filter and its descendants, scan matching and the iterative closest point algorithm, factor-graph optimization, the A* search and its kinodynamic relatives, sampling-based planners, the dynamic window approach and velocity obstacles, and the safety-rated layers that sit beneath all of them. The shape of the book The first chapter sets out the probabilistic frame on which everything else rests: why a robot must carry a belief rather than a fact, how the recursive Bayes filter organizes the problem, and why coordinate frames and time are the hidden variables in every bug report. The second chapter looks closely at the sensors themselves, LiDAR, cameras, inertial units, wheel encoders and radar, and at the specific ways each one misleads. The third chapter covers calibration and time synchronization, the unglamorous engineering without which fusion is impossible and on which a surprising share of field failures can be blamed. With those foundations in place, the fourth chapter turns to fusing LiDAR and camera data: the choice between combining raw signals, intermediate features or final decisions, and what each choice costs. The fifth chapter treats odometry, the estimation of motion from one moment to the next, through scan registration, visual tracking and tightly coupled inertial fusion. The sixth chapter takes up simultaneous localization and mapping proper: graph optimization, loop closure, place recognition, and the peculiar difficulties of repetitive environments such as warehouse aisles. The last three chapters move from knowing to acting. The seventh covers path planning, from grid search to sampling methods to trajectory optimization, and the particular demands of wheeled bases and multirotors. The eighth covers collision avoidance among moving people and machines, and the layered safety architecture that separates the autonomy stack from the certified functions that must stop the robot no matter what the autonomy believes. The ninth looks at what changes when there are hundreds of robots in one building or many aircraft in one airspace: fleet coordination, traffic management, regulation, and the discipline of validating a system whose behaviour cannot be exhaustively tested. The conclusion draws out what follows from the whole: a set of design commitments for anyone building or buying an autonomous physical system, and an honest account of what remains unsolved. A note on currency The field moves quickly, and some of what it does has changed in the last few years in ways worth flagging at the outset. Learned models now sit inside many perception pipelines that a decade ago were entirely hand-engineered, and fusion of LiDAR and camera data increasingly happens in a shared bird's-eye-view representation rather than by projecting points onto images. Open-source navigation software, especially the ROS 2 navigation stack, has become a de facto starting point for commercial mobile robots. Warehouse automation has scaled to a degree that would have seemed fanciful in 2012, when Amazon bought Kiva Systems; in mid-2025 Amazon reported that it had deployed its millionth robot across more than three hundred facilities, alongside a fleet-coordination model it called DeepFleet. In the air, regulators in the United States and Europe are moving from case-by-case waivers toward standing rules for flights beyond the operator's line of sight, though at the time of writing the American rule had not yet been finalized. What has not changed is the underlying structure of the problem. Sensors still lie in characteristic ways. Estimates still drift. Maps still go stale. Plans still have to be made with imperfect knowledge and executed by machines with mass and momentum. The methods change; the shape of the problem endures, and it is the shape of the problem that this booklet is mainly about. CHAPTER 1 Belief, Not Truth Every autonomous robot runs on a model of the world that is wrong. The question that separates a reliable machine from a dangerous one is whether its software knows how wrong, and in which direction. This chapter sets out the framework that almost all modern robot state estimation is built on, the recursive Bayesian filter, and explains why the habit of carrying a probability distribution rather than a single best guess is not academic fastidiousness but the precondition for everything else in the book. The state and the belief Engineers describe a robot's situation with a state: a short list of numbers that, if known exactly, would be enough to predict what happens next. For a warehouse robot driving on a flat floor, the state might be three numbers, its position in two dimensions and its heading. Add its forward and turning speeds and the state has five. For a quadrotor drone the state is larger: three numbers for position, three for velocity, three or four for orientation, and often six more for the slowly wandering biases of its accelerometers and gyroscopes. When the robot is also building a map, the positions of landmarks or the occupancy of grid cells become part of the state as well, and the list may grow to millions of entries. The robot never observes its state directly. It observes measurements that depend on the state: the distance from its laser scanner to the nearest wall in each direction, the pixel coordinates of a corner in a camera image, the number of encoder ticks since the last reading. And it issues commands that change the state: a velocity setpoint to the motor drivers, a thrust command to four propellers. The task of state estimation is to work backwards from a history of commands and measurements to the state that most plausibly produced them. Because the measurements are noisy and the commands are executed imperfectly, there is never a single answer. There is a spread of possible states, some more plausible than others. Probabilistic robotics calls this spread the robot's belief: a probability distribution over states, conditioned on everything the robot has seen and done. The robot's best guess is a summary of the belief, perhaps its peak or its mean. But the belief itself contains much more: how confident the robot should be, in which directions it is unsure, and whether there might be two or three quite different places it could plausibly be. The distinction matters in practice. Consider a robot in a long, featureless corridor. Its laser scanner sees two parallel walls. That observation pins down the robot's distance from each wall and its heading relative to them very precisely. It says nothing at all about how far along the corridor the robot is. A belief represented properly will be narrow across the corridor and long along it, a thin cigar shape. A system that carried only a best-guess position would have no way to express this; it would report a point and leave every downstream module to assume the point was equally good in all directions. A planner relying on that point might happily schedule a turn into a side door that, in reality, the robot has not yet reached. The two-step rhythm The recursive Bayes filter organizes estimation into a repeating rhythm of two steps, and once the rhythm is understood, a large part of robotics software becomes legible. The first step is prediction. The robot takes its current belief and pushes it forward through a model of how the state evolves given the command just issued. If the belief said the robot was at a certain spot facing north, and the command was to drive forward at one metre per second for a tenth of a second, the prediction moves the belief ten centimetres north. Because the motion model is imperfect, wheels slip, floors are uneven, motors lag, prediction also spreads the belief out. Uncertainty grows when the robot moves without looking. The second step is correction, sometimes called the update. A measurement arrives. The filter asks, for every possible state in the predicted belief, how likely this particular measurement would have been if the robot were really in that state. States that would have produced something close to the actual measurement gain weight; states that would have produced something very different lose it. The belief is reshaped accordingly, and normally it contracts. Uncertainty shrinks when the robot looks. Prediction, correction, prediction, correction: the cycle runs dozens or hundreds of times per second. The two ingredients that define any specific filter are the motion model, which says how the state changes under a command, and the measurement model, which says what a sensor should report given a state. Most of the craft of state estimation is the craft of writing these two models well, including writing honest descriptions of their errors. The rhythm has a property worth stating plainly. The filter is recursive: it needs only the previous belief, the latest command and the latest measurement, not the whole history. This is what makes it fast enough to run on a robot. It is also what makes it fragile, because any error that enters the belief is carried forward indefinitely unless a later measurement corrects it. A filter that once becomes confidently wrong can stay confidently wrong. Kalman's filter and its descendants The most influential implementation of this rhythm dates from 1960, when Rudolf Kálmán published a recursive solution for the case in which the motion and measurement models are linear and all errors are Gaussian. Under those assumptions the belief stays Gaussian forever, and a Gaussian is completely described by a mean vector and a covariance matrix. The Kalman filter updates these two quantities in closed form at each step. The mean is the best guess; the covariance is the shape of the uncertainty around it, including the correlations between variables, for example the fact that an error in heading will, after some driving, turn into an error in sideways position. The correction step of a Kalman filter has an intuitive core. It compares the measurement that arrived with the measurement the predicted state would have produced. The difference, called the innovation, is the surprise. The filter then moves the estimate some fraction of the way toward explaining that surprise. The fraction, the Kalman gain, is set by the relative size of two uncertainties: how uncertain the prediction was and how noisy the sensor is. A precise sensor and a vague prediction yield a large gain, and the estimate jumps to agree with the measurement. A noisy sensor and a confident prediction yield a small gain, and the measurement nudges the estimate only slightly. This weighting is the essence of sensor fusion, and it reappears, in more elaborate dress, in every method later in this book. Robots are not linear. A wheeled robot's position update involves the sine and cosine of its heading; a camera's measurement of a point involves division by depth; a drone's orientation lives on a curved space of rotations where adding small angles is only approximately valid. The extended Kalman filter handles this by linearizing the models around the current estimate at each step, using their first derivatives. It works well when the uncertainty is small relative to the curvature of the models and fails, sometimes abruptly, when it is not. The unscented Kalman filter instead pushes a small, carefully chosen set of sample points through the true nonlinear model and fits a Gaussian to where they land, which often handles moderate nonlinearity better at similar cost. Both remain workhorses. The flight controllers on most small drones run some form of extended Kalman filter to combine inertial, barometric, magnetic and satellite measurements, and the widely used open-source autopilots publish theirs in detail. When the belief cannot reasonably be approximated by a single Gaussian, as in the corridor that might be any of several identical corridors, the usual alternative is the particle filter. Instead of a mean and covariance, it represents the belief by a large set of sample states, each with a weight. Prediction moves every particle according to the motion model plus random noise; correction reweights the particles by how well each explains the measurement; periodic resampling discards low-weight particles and duplicates high-weight ones. A particle filter can represent a belief with several separate peaks, which is exactly what a robot needs when it has been switched on with no idea where it is, or when it has been picked up and moved. Dellaert, Fox, Burgard and Thrun introduced Monte Carlo localization on this basis in 1999, and an adaptive variant of it remains the default two-dimensional localizer in the ROS navigation software used by a large share of commercial indoor robots. Uncertainty that is honest All of these filters share a vulnerability: they are only as good as the error models they are given. The covariance in a Kalman filter is not measured; it is computed from assumed noise levels. If the engineer tells the filter that the wheel encoders are accurate to one percent of distance travelled, and the robot then drives across a patch of spilled oil where the wheels slip by twenty percent, the filter will report a confident estimate that is wrong. It has no way of knowing that its assumptions were violated unless some measurement reveals the contradiction. This leads to one of the central practical concerns of the field, consistency. An estimator is consistent if the uncertainty it reports matches the errors it actually makes: if it says it is within five centimetres ninety-five percent of the time, then, tested against ground truth, it should be within five centimetres about ninety-five percent of the time. An overconfident estimator, one whose reported covariance is too small, is the dangerous kind. It rejects valid measurements as outliers because they disagree with its overconfident prediction, and it tells downstream modules that they may plan with tight margins when they may not. There are well-known structural causes of overconfidence. Treating two measurements as independent when they share an error source, for instance two features extracted from the same blurred image, double-counts information. Linearizing at a wrong estimate in an extended Kalman filter can make the filter believe it has observed something it has not; a classic result in visual-inertial estimation showed that naive linearization makes a filter think it can observe its absolute heading about the gravity axis, which it cannot, leading to falsely small uncertainty in yaw. Engineers counter these with careful modelling, with deliberate inflation of process noise, and with statistical tests on the innovations: if the surprises are consistently larger than the filter's own covariance says they should be, something is wrong. Practitioners learn to treat the innovation sequence as a diagnostic instrument. A healthy filter's surprises look like the noise it was told to expect, centred on zero and of the predicted size. A biased surprise, consistently positive in one axis, points to a calibration error or an unmodelled bias. Surprises that grow over time point to a model that is drifting out of validity. Logging and plotting the innovations is often the fastest way to find a fault that would otherwise surface only as a robot that occasionally drives into things. Frames and time: the hidden state Two further pieces of the state are rarely written down in textbooks and almost always appear in bug reports: coordinate frames and time. Every measurement is made in some frame of reference. A laser scanner reports distances relative to its own spinning head. A camera reports pixels relative to its own optical centre. The robot's wheels move it relative to its own chassis. The map is expressed in some fixed world frame chosen when it was built. To combine any two pieces of information, the software must transform them into a common frame, and it can only do so if it knows the rigid transform between every pair of frames involved: where exactly the scanner is mounted on the chassis, at what angle the camera points, where the chassis is in the world. Robotics middleware formalizes this as a tree of frames with transforms on every edge; in ROS the convention, documented in a standards proposal known as REP 105, distinguishes a map frame that is globally consistent but may jump when localization corrects itself, an odom frame that is smooth and continuous but drifts, and a base_link frame attached to the robot. Getting these relationships right is the subject of Chapter 3; the point here is that they are part of what the robot must know, and errors in them are indistinguishable from errors in the state. The second hidden variable is time. A measurement describes the world at the instant it was taken, not the instant it reached the software. A camera image may take thirty milliseconds to be exposed, read out, transferred and decoded. A spinning LiDAR collects its points over a tenth of a second, during which the robot itself may have moved several centimetres and turned a degree or two. If the filter treats a measurement as current when it is already stale, it corrects the wrong predicted state. On a slow robot the effect may be a small blur; on a drone turning at a hundred degrees per second, a ten-millisecond timing error turns into a one-degree attitude error, which at ten metres range shifts a fused point by seventeen centimetres. Good estimators carry timestamps on every measurement, buffer their state history, and apply each correction at the moment the measurement actually describes. From filtering to smoothing The recursive filter, for all its elegance, makes one decision that later chapters will revisit. Once it has incorporated a measurement and moved on, it never reconsiders. Linearization choices made with an early, rough estimate are frozen into the belief forever. If, a minute later, the robot recognises a place it has seen before and learns that its estimate has drifted by half a metre, the filter has no clean way to go back and correct the trajectory that led there. The alternative is smoothing: keeping some or all of the past states as unknowns and solving for all of them together, using every measurement that bears on any of them. This is computationally heavier but more accurate and, crucially, allows re-linearization when the estimate improves. For years smoothing was too expensive for real-time use on a robot. Advances in sparse linear algebra, and the recognition that the structure of the problem can be expressed as a graph of variables and constraints, have made it practical. Most modern simultaneous localization and mapping systems are smoothers, and many modern odometry systems use a compromise, a sliding window of recent states optimized together while older ones are summarized and set aside. Chapters 5 and 6 explain how. The vocabulary of this chapter, belief, prediction and correction, motion and measurement models, covariance and consistency, frames and timestamps, filtering and smoothing, is the vocabulary of the rest of the book. Sensor fusion, localization, mapping and even planning under uncertainty are all elaborations of the same underlying act: holding a belief about the world, testing it against evidence, and acting in a way that respects how much is still unknown. The rest of the booklet follows that act through each stage of a working system, beginning with the instruments that supply the evidence. CHAPTER 2 What Sensors Actually Report A sensor datasheet describes an instrument under ideal conditions. A robot meets its sensors under real ones: dust, glare, vibration, reflective shrink-wrap, cold mornings, low batteries and the electromagnetic noise of a dozen motors. Fusion is only as good as the measurement models it uses, and a measurement model is only as good as the engineer's understanding of how the instrument behaves when conditions are not ideal. This chapter looks at the main sensors on warehouse robots and drones with that question in mind: not what they measure, but how they fail. LiDAR: distance by the speed of light A LiDAR, for light detection and ranging, measures distance by timing light. The most common design emits a short pulse of infrared laser light and measures how long it takes for the reflection to return. Light travels about thirty centimetres per nanosecond, and the pulse makes a round trip, so each nanosecond of timing corresponds to fifteen centimetres of range. Centimetre-level precision therefore requires timing electronics resolving tens of picoseconds, or statistical tricks that average many returns. A single beam measures one distance. To build a picture, a LiDAR must sweep its beam or use many beams. The classic automotive and robotics LiDAR stacks a column of laser-detector pairs, sixteen, thirty-two, sixty-four or more, on a head that spins several times per second, producing a set of horizontal rings around the sensor. Planar two-dimensional LiDARs, with a single beam swept in a flat fan, are the standard sensor on indoor mobile robots; they are inexpensive, robust and exactly suited to a world of vertical walls and rack uprights. Solid-state designs replace the spinning head with steerable optics, micro-mirrors, optical phased arrays or flash illumination of the whole scene at once, trading field of view for fewer moving parts and lower cost. Some newer sensors use non-repetitive scan patterns that fill in the field of view progressively the longer the sensor dwells, which changes how odometry algorithms must accumulate points. A second family, frequency-modulated continuous-wave LiDAR, sends a continuous beam whose optical frequency is swept over time and measures the beat between outgoing and returning light. This yields range and, through the Doppler shift, the radial velocity of each point directly, which is valuable for separating moving objects from static ones. It is less common on robots and drones than pulsed time-of-flight at the time of writing but is arriving in automotive products. Wavelength matters. Many sensors operate near 905 nanometres, where silicon detectors are cheap, but eye-safety limits cap the power they can emit. Sensors at 1550 nanometres can emit much more power while remaining eye-safe, because the eye's fluid absorbs that wavelength before it reaches the retina, which extends range; they require more expensive detectors. How LiDAR misleads The strengths of LiDAR are direct geometric measurement, independence from ambient light and accuracy that does not degrade with distance the way stereo vision does. Its failure modes are specific and worth knowing by heart. Dark and absorbent surfaces return little light. Black rubber, matte black plastic and some dark fabrics may produce no return beyond a few metres, so an obstacle simply does not appear. A forklift tyre or a dark-clothed person's legs can be close to invisible to a low-power planar scanner at the edge of its range. Specular surfaces behave like mirrors. A polished floor, a glass door, a stainless-steel cabinet or a wet patch can reflect the beam away entirely, producing no return, or toward another object, producing a return that appears to come from behind the mirror. Robots in shopping centres and hospitals have been confused by phantom corridors extending through glass walls. Retroreflective material, the kind used in safety vests and some warehouse labels, does the opposite: it returns so much light that the detector saturates, which can bias the range and produce bloom, spurious points spreading around the true target. Transparent and semi-transparent surfaces, such as stretch-wrap around pallets, plastic strip curtains at cold-store doorways and glass, may return a weak echo from the surface, a stronger one from behind, or both. Many sensors report multiple returns per pulse for exactly this reason, and software must choose which to trust. Edges produce mixed pixels. When a beam's footprint straddles the edge of a near object and a far wall, part of the energy returns from each, and the sensor may report a distance somewhere in between, a point floating in empty space. Rows of such points appear as ghostly veils trailing off the edges of objects, and a careless obstacle detector will treat them as obstacles. Airborne particles scatter light. Dust in a distribution centre, fog, rain, snow and the propeller wash of a drone kicking up debris all produce returns that are real reflections from real particles but that do not correspond to solid obstacles. Some sensors filter these by return shape; many leave the problem to software. Motion during the sweep distorts the scan. A spinning LiDAR collecting points over a tenth of a second while its vehicle moves produces a scan in which the start and end of the sweep are measured from different positions. A straight wall appears bent; a full sweep may fail to close on itself. Correcting this, called deskewing or motion compensation, requires knowing the robot's motion during the sweep, usually from an inertial unit, and it is one of the reasons Chapter 3 treats timing so seriously. Mutual interference between LiDARs is a growing concern as fleets densify. One sensor can receive another's pulses and report false ranges. Manufacturers use coding and randomized timing to reject foreign pulses, with varying success, and a fleet engineer deploying hundreds of robots of the same model in one hall should test for it rather than assume. Cameras: rich, cheap and ambiguous A camera measures direction, not distance. Each pixel records the brightness and colour of light arriving from one narrow cone of directions. This makes cameras extraordinarily rich, able to read labels, recognise people, classify objects and detect the texture that visual odometry tracks, and extraordinarily ambiguous, since a single image cannot distinguish a small near object from a large far one. Depth can be recovered in several ways. A stereo pair of cameras a known distance apart sees each point from two viewpoints, and the shift between its two image positions, the disparity, gives its depth. Stereo depth error grows with the square of distance, so a pair with a ten-centimetre baseline may be accurate to millimetres at half a metre and to tens of centimetres at ten metres. Structured-light and time-of-flight depth cameras project their own infrared illumination and recover depth at short range with good density; they are common on manipulators and on the downward- or forward-looking obstacle sensors of indoor robots, and they struggle in sunlight, which swamps their projected light. A single moving camera can also recover depth by triangulating features across frames, though only up to an unknown scale unless other information, typically from an inertial unit, supplies it. And learned monocular depth networks now produce plausible dense depth from a single image, useful as a prior but not to be confused with a measurement. How cameras mislead Camera failures are dominated by lighting. A camera has a limited dynamic range: the ratio between the brightest and darkest scene it can capture at once. A warehouse aisle with a sunlit loading dock at one end, or a drone flying from shadow into sun, can exceed it, leaving parts of the image saturated white or crushed to black. Automatic exposure takes several frames to adapt, during which visual tracking may fail. High-dynamic-range sensors help but do not eliminate the problem. Motion blur arises when the exposure is long relative to the motion. In dim indoor lighting a camera may need an exposure of ten milliseconds or more, during which a turning robot's image smears, destroying the fine texture that feature trackers need. Engineers trade exposure time against noise, and drones in particular often run short exposures and accept grainy images rather than blurred ones. Most inexpensive cameras use a rolling shutter: the sensor exposes and reads its rows one after another rather than all at once, so the top of the image is captured several milliseconds before the bottom. On a vibrating drone or a fast-turning robot, vertical lines lean, and the geometry assumed by every standard vision algorithm, that the whole image was taken from one pose, is violated. Global-shutter sensors expose all rows simultaneously and are preferred for visual odometry, at some cost in price and low-light sensitivity. Where rolling-shutter cameras must be used, estimators can model the per-row timing explicitly. Low-texture and repetitive scenes defeat feature-based methods. A blank wall gives a tracker nothing to hold on to. A warehouse aisle of identical shelving, a field of identical crops or the regular pattern of a car-park ceiling gives it too much of the same thing, so a feature in one frame may be matched to the wrong instance of the pattern in the next. This perceptual aliasing is a theme of Chapter 6. Finally, lens geometry must be modelled. Wide-angle and fisheye lenses, popular on drones for their field of view, bend straight lines into curves; the camera's intrinsic calibration, its focal length, optical centre and distortion coefficients, must be known accurately for any geometric use of the image. An error of a pixel in the assumed optical centre becomes a systematic angular error in every measurement. Inertial measurement: fast, local, drifting An inertial measurement unit, or IMU, combines three accelerometers and three gyroscopes, and on drones usually a magnetometer as well. Modern robots and drones use micro-electromechanical units costing a few dollars, sampled hundreds or thousands of times per second. The IMU is the only sensor that measures the robot's own motion directly and continuously, with no dependence on the environment, which is why nearly every serious odometry system leans on it. Its weakness is integration. A gyroscope measures angular rate; integrating rate gives angle, so a small constant error in rate, a bias, becomes an angle error that grows linearly with time. An accelerometer measures specific force, acceleration minus gravity; to get position, the software must subtract gravity using its current attitude estimate and integrate twice, so a small bias becomes a position error growing with the square of time, and a small attitude error leaks a component of gravity's nearly ten metres per second squared into the horizontal acceleration. Consumer-grade units, left alone, drift by metres within seconds. Biases also change with temperature, with time since power-on and with mechanical shock. IMUs are therefore never used alone for position; they bridge the gaps between slower, absolute measurements and supply the high-rate motion information needed to deskew LiDAR scans and predict feature locations between camera frames. Magnetometers, which provide heading by sensing the Earth's field, are notoriously unreliable indoors and near steel structures, motors and power cables. A drone flying close to a steel bridge or a warehouse robot driving over reinforced concrete may see its magnetic heading swing by tens of degrees. Many indoor systems ignore the magnetometer entirely. Wheels, radar, satellites and beacons Wheel encoders count rotation and, with a model of wheel radius and spacing, give distance and turning. They are precise over short distances on good floors and degrade with slip, wear, uneven loading, which changes the effective radius of soft tyres, and, for differential-drive robots, any error in the assumed distance between the wheels, which turns directly into heading error. They remain the backbone of short-term motion estimation for ground robots. Radar uses radio waves rather than light. Its resolution is much coarser than LiDAR's, and its returns are cluttered by multipath reflections, but it sees through dust, smoke, fog and rain, measures relative velocity directly through the Doppler shift, and is cheap at automotive volumes. Imaging radars with many antenna elements are improving resolution sharply. On drones, compact radars are used for obstacle detection and altitude measurement; in outdoor yards and on ports, radar complements LiDAR in weather. Global navigation satellite systems give drones and outdoor robots absolute position. A standalone receiver is accurate to a few metres in open sky; corrections from a nearby base station, real-time kinematic positioning, bring this to centimetres. The failure modes are well known: loss of signal under bridges and inside buildings, multipath reflection from building faces that biases positions by metres in urban canyons, and deliberate jamming and spoofing, which has become a significant concern for drone operators in some regions. A drone navigation stack that trusts satellite position without checking it against inertial and visual evidence is one well-placed jammer away from a crash. Indoor robots frequently use artificial landmarks: fiducial codes on the floor, reflective markers on rack uprights, or ultra-wideband radio beacons giving range to fixed anchors. The original Kiva robots, which Amazon acquired in 2012 and developed into its drive fleet, navigated a grid of two-dimensional codes on the floor, reading one with a downward camera at each grid cell and dead-reckoning between them. Such infrastructure trades installation cost for an enormous simplification of the localization problem, and in structured facilities it is often the right trade. Comparing the instruments No single sensor is adequate, and the reason is visible in their complementary weaknesses. Table 1 summarizes the qualitative trade-offs discussed in this chapter. Table 1. Typical strengths and weaknesses of common robot sensors. Sensor Measures Main strength Typical failure Typical rate LiDAR Range and bearing Direct geometry, light-independent Dark, glossy or glass surfaces; dust 10–20 Hz sweeps Camera Bearing, colour Semantics, texture, low cost Glare, darkness, blur, no direct depth 20–60 Hz IMU Own motion High rate, environment-free Integrated drift, bias change 200–1000 Hz Encoders Wheel rotation Precise short-term motion Slip, wear, load changes 50–200 Hz Radar Range, Doppler Weather and dust penetration Coarse resolution, multipath 10–20 Hz GNSS Absolute position Global reference outdoors Blockage, multipath, jamming 1–20 Hz The design of a sensor suite follows from the operating environment's dominant failure modes. A robot in a clean, well-lit pharmaceutical warehouse with glass partitions needs something that is not LiDAR to see the glass; ultrasonic sensors and depth cameras are common supplements. A drone inspecting a dusty mine or a steel bridge cannot trust satellites or magnetometers and must lean on LiDAR and inertial fusion. A robot that must recognise a person and slow down for them needs a camera, because a LiDAR sees a person as a pair of leg-shaped clusters that look much like the uprights of a trolley. The deeper point is that each sensor's errors are structured, not random. LiDAR errors cluster at edges and on certain materials; camera errors cluster in certain lighting; IMU errors accumulate with time; encoder errors arrive in bursts with slip. Fusion works when the measurement model captures that structure well enough that the filter can tell which sensor to believe in which circumstance. That depends first on knowing exactly where each sensor is and exactly when each measurement was taken, the subject of the next chapter. CHAPTER 3 Calibration and Time Ask experienced robotics engineers where their worst field problems came from, and a striking number will describe something that was not an algorithm at all. A camera bracket that flexed when the robot braked. A LiDAR remounted after a repair, two degrees off from its original orientation. A clock on one computer that drifted against the clock on another until fused measurements were forty milliseconds apart. A firmware update that changed the default timestamping of a sensor driver from the end of a scan to its start. None of these is glamorous, and none appears in the headline figure of a research paper. Every one of them can make an excellent fusion algorithm produce nonsense. This chapter covers the two foundations on which all multi-sensor fusion rests: spatial calibration, knowing where every sensor is and how it maps the world into numbers, and temporal calibration, knowing when every measurement was taken on a common clock. Intrinsic calibration: how each sensor sees Each sensor has internal parameters that govern how a quantity in the world becomes a number in its output. For a camera these intrinsics are the focal lengths in each image direction, the location of the optical centre on the sensor, and a set of distortion coefficients describing how the lens bends rays away from the ideal pinhole model. They are estimated by photographing a known pattern, classically a checkerboard or a grid of uniquely identifiable markers, from many angles and distances, and solving for the parameters that best explain where the pattern's corners appear. The procedure popularized by Zhengyou Zhang around 2000 remains the core of most calibration toolkits. The quality of an intrinsic calibration depends on coverage more than on the number of images. Corners must be observed across the whole image, including the edges and corners where distortion is strongest, and the pattern must be tilted substantially relative to the camera so that focal length and distance can be separated. A calibration done with a board held face-on in the middle of the frame can report a low reprojection error and still be badly wrong at the image edges, which is exactly where a wide-angle drone camera sees the obstacles approaching from the side. LiDARs have intrinsics too, though they are usually calibrated at the factory: the precise angle of each laser in a multi-beam head, per-beam range offsets, and sometimes intensity response curves. Factory calibration is generally good, but users of multi-beam sensors occasionally find individual beams with range biases of a few centimetres, visible as a ring that sits slightly inside or outside its neighbours on a flat floor. IMUs have scale-factor errors, axis misalignments and biases; inexpensive units benefit from a calibration in which the unit is held still in several orientations so that gravity provides a known reference. Wheel odometry has its own intrinsics, the effective wheel radii and the track width, which are best estimated by driving known paths, for example a measured square in both directions, and fitting the parameters that close the loop. Extrinsic calibration: where each sensor sits Extrinsic calibration determines the rigid transform, three translations and three rotations, between each sensor and a common reference frame on the robot. It is easy to underestimate how much accuracy this requires. A rotation error of one degree between a LiDAR and a camera displaces a LiDAR point projected into the image by about seventeen millimetres per metre of range: at twenty metres, more than a third of a metre. A pedestrian's LiDAR points would be painted with the colour and class of the wall behind them. Measuring the transform with a ruler and a CAD model gets translation to perhaps a few millimetres and rotation to perhaps a degree, which, as the arithmetic shows, is not good enough for fusion at range. Engineers therefore estimate extrinsics from data. For camera to LiDAR calibration, the classic approach uses a planar target seen by both sensors. The camera locates the board's plane from its pattern; the LiDAR locates the same plane from the points that fall on it. Each pose of the board gives constraints relating the two frames, and several poses at different orientations determine the transform. Qilong Zhang and Robert Pless described this method for a camera and a laser rangefinder in 2004, and many variants followed, using board edges, holes cut in the board, or spheres that are easy to find in both modalities. Targetless methods instead align natural structure: edges in the image with depth discontinuities in the point cloud, or intensity in the LiDAR returns with brightness in the image, or the motion each sensor independently estimates over a drive, which must agree once the transform is right. Targetless methods are attractive for recalibrating in the field, where nobody wants to carry a calibration board to every robot. For camera to IMU calibration, the standard practice is to wave the sensor rig in front of a calibration pattern while exciting all rotational and translational axes, and to solve jointly for the transform, the time offset between the two sensors, and the IMU biases. The Kalibr toolbox, from work by Paul Furgale, Joern Rehder and Roland Siegwart published in 2013, popularized this joint spatial and temporal estimation using a continuous-time representation of the trajectory, and it remains widely used. The lesson it embodies is that space and time cannot be calibrated separately: a time offset between camera and IMU, during rotation, looks almost exactly like a rotation between them. For LiDAR to IMU calibration, similar motion-based methods apply, and several recent LiDAR-inertial odometry systems estimate the extrinsic transform online as part of their state, starting from a rough guess and refining it as the robot moves. Calibration does not stay calibrated The most important operational fact about extrinsic calibration is that it decays. Brackets loosen. Thermal expansion shifts mounts by fractions of a millimetre over the day. A robot that bumps a rack, or a drone that lands hard, may knock a sensor by a degree. Maintenance technicians replace sensors and reinstall them slightly differently. In a fleet of hundreds of robots, calibration drift is not an occasional event but a steady background process. Mature operations treat calibration as a monitored quantity rather than a one-time setup. Some approaches: • Online consistency checks. The fusion pipeline continuously compares what each sensor implies about the same thing, for example whether LiDAR depth edges align with image edges, or whether the motion estimated by the camera matches that from the LiDAR, and raises a flag when the agreement degrades. • Online estimation. For transforms that are only weakly constrained during normal operation, the estimator may include small corrections as slowly varying states, with the important caveat that unobservable parameters, those the robot's normal motion does not excite, will wander if estimated online, so they must be held fixed or tightly constrained. • Calibration stations. Some warehouse operators place a calibration target at a charging station or a maintenance bay so that each robot verifies its sensor alignment every time it docks. • Mechanical design. The cheapest calibration is one that does not change. Rigid, single-piece sensor mounts, stiff enough that braking and vibration do not flex them, and keyed so that a replaced sensor goes back exactly where the old one was, pay for themselves quickly. Time: the shared clock problem A robot is a distributed system. Its LiDAR, cameras, IMU and wheel controllers each contain their own processors and their own clocks, and they deliver data over different links with different latencies: Ethernet for the LiDAR, USB or a camera serial interface for the cameras, a serial line or SPI bus for the IMU, a CAN bus for the motor controllers. The main computer receives messages with some unknown and variable delay after the measurements they describe. Fusion needs to know the true time of each measurement on a single common clock. There are three levels of solution, and a robust system usually uses all of them. The first is hardware synchronization. A single source generates a trigger pulse that starts camera exposures, and the same pulse, or a pulse-per-second signal from a satellite receiver, is fed to the LiDAR and IMU so that they stamp their data against it. Many LiDARs accept a pulse-per-second input together with a time message precisely so that their point timestamps can be aligned to an external reference. Hardware triggering makes multi-camera rigs capture simultaneously, which is essential for stereo, and gives each image a known capture instant. The second is clock synchronization over the network. The Precision Time Protocol, standardized as IEEE 1588, synchronizes clocks over Ethernet to sub-microsecond accuracy when switches and network interfaces support hardware timestamping, and to tens of microseconds in software. Most modern LiDARs support it. A robot's main computer can act as the master clock, and each sensor then stamps its own measurements in the shared timebase. The third is estimation. Where neither hardware triggers nor network synchronization reaches a sensor, the software estimates the offset. If a camera's images arrive with a roughly constant latency, the offset can be estimated during calibration, as Kalibr does, or online, as several visual-inertial estimators do by adding the time offset to the state. The principle is that a time offset makes the camera's view of rotation lag the gyroscope's measurement of it, and the lag can be found by correlation. Software timestamping on arrival, the default when nothing else is done, is the trap. The arrival time includes driver buffering, operating-system scheduling jitter and transport delay. On a busy computer, two messages from the same sensor may be delayed by different amounts, so even relative timing is corrupted. A system that stamps messages on arrival and fuses them as though the stamps were true will work on the bench, where the computer is idle and the robot is slow, and fail intermittently in production, where the computer is loaded and the robot is fast. Motion compensation: making a sweep a snapshot Timing becomes geometry in the case of a scanning LiDAR. A spinning sensor that completes one revolution every hundred milliseconds measures its first point and its last point a tenth of a second apart. During that interval a robot moving at two metres per second travels twenty centimetres, and a drone yawing at ninety degrees per second turns nine degrees. Treating the sweep as a single snapshot taken from one pose smears every surface. Deskewing corrects this. Each point carries its own timestamp, either explicitly or implicitly through its position in the sweep. The software uses a high-rate motion estimate, typically IMU integration over the sweep or the previous odometry estimate extrapolated forward, to compute where the sensor was when each point was measured, and transforms every point into the frame of a single reference instant, usually the end of the sweep. The corrected cloud is what a sensor with an infinitely fast shutter would have seen. The quality of the correction depends on the quality of the motion estimate, which creates a pleasing circularity: odometry needs deskewed scans, and deskewing needs odometry. Modern LiDAR-inertial systems resolve this by iterating, deskewing with the IMU prediction, estimating the motion from the deskewed scan, and deskewing again if needed, or by building the per-point motion directly into the estimation problem. Either way, the per-point timestamps must be correct, and the IMU and LiDAR clocks must agree. Rolling-shutter cameras raise the same issue at a smaller scale: each row of the image has its own capture time, and on a vibrating drone the difference matters. Estimators that model rolling shutter treat each row's pose as an interpolation along the trajectory. Anatomy of a timing fault It helps to walk through how a timing fault typically presents, because it rarely looks like one. Imagine a warehouse robot whose people-detection pipeline paints LiDAR points with the class labels from a camera segmentation network. On the bench, with the robot stationary, the painted points line up perfectly with the people in the images. In the facility, the operations team reports that the robot occasionally fails to slow for pickers who approach from the side, and that it sometimes slows for no apparent reason as it passes rack uprights. A first instinct is to blame the segmentation network, and retraining it would consume weeks without fixing anything. The pattern of the symptoms points elsewhere. Both failures occur while the robot is turning, and both have the same geometry: the labels appear displaced sideways from the objects they belong to, in the direction opposite to the turn. A person approaching from the side has their label painted onto the empty floor beside them, so their LiDAR points are classified as generic obstacles; an upright has a person's label from a picker a metre away painted onto it. The arithmetic of the displacement gives the diagnosis. If a robot turns at forty-five degrees per second and the labels are shifted by about one and a half degrees, the camera and LiDAR data are misaligned by roughly thirty-three milliseconds, which is suspiciously close to one frame interval of a thirty-frames-per-second camera. Inspection then finds that the camera driver stamps each image when the frame has been fully received over USB, not when it was exposed, and that a recent operating-system update changed the buffering so that one frame is held back. The images are one frame late. The fix is to stamp at exposure, using the camera's own hardware timestamp translated to the robot's clock, or, failing that, to measure and subtract the latency. Two lessons generalize. First, a timing error behaves like a rotation error that grows with angular speed, so any symptom that appears only during turns should prompt a check of timestamps before anything else. Second, the bench is the worst place to find timing faults, because a stationary robot makes stale data indistinguishable from fresh. Calibration and timing should be validated with the robot moving, ideally by driving past a distinctive target while turning in both directions and checking that the fused data stay aligned. The transform tree as a contract All of this calibration and timing information has to be made available to every module that needs it, consistently. Robotics middleware does this with a transform tree: a data structure, maintained by a dedicated library in ROS and similar systems, that stores the transform along every edge of the tree of frames, each stamped with the time it applies to, and answers queries of the form "where was frame A relative to frame B at time t?" by chaining and interpolating transforms. It is useful to think of the transform tree as a contract between modules. Static edges, from base to each sensor, carry the extrinsic calibration. Dynamic edges carry the odometry and localization estimates. A perception module that wants to place a camera detection in the map asks the tree for the camera-to-map transform at the image's capture time, and gets an answer composed from the camera's extrinsic calibration, the odometry estimate at that instant and the latest localization correction. If any edge is wrong or any timestamp is mislabelled, the answer is wrong, and the perception module has no way of knowing. Several defensive practices follow. Treat calibration files as version-controlled configuration tied to each physical robot's serial number, never as defaults copied between units. Log raw sensor data with hardware timestamps alongside the transform tree, so that any incident can be replayed and the calibration questioned after the fact. Test timing under load, with the main computer saturated, not only on the bench. And build automated checks that will notice when a sensor's data stops agreeing with its neighbours, because a calibration fault rarely announces itself; it shows up as a robot that is slightly worse at everything. With sensors characterized and their geometry and timing established, the system can begin to combine them. The most visible and most discussed combination in modern autonomy is between the two richest sensors, the LiDAR and the camera, and it is the subject of the next chapter. CHAPTER 4 Fusing LiDAR and Camera LiDAR and cameras are complementary in an almost textbook way. LiDAR gives accurate geometry with poor semantics: it knows precisely that there is a vertical surface 4.31 metres away, but not whether it is a person, a cardboard cutout or a stack of cartons. A camera gives rich semantics with poor geometry: it can tell a person from a carton at a glance, but not reliably how far away either is. Combining them promises the best of both, and in practice delivers it only when the designer has made a clear choice about where in the processing chain the combination happens, and has understood what that choice costs. This chapter explains the main options, from fusing raw measurements to fusing final decisions, with attention to what each assumes about calibration, timing and the failure of one sensor. The geometry of combining them All LiDAR-camera fusion rests on a single geometric operation: projecting a three-dimensional LiDAR point into the camera image. Given a point in the LiDAR's frame, the extrinsic calibration transforms it into the camera's frame; the camera's intrinsic model then maps it to a pixel location. Points behind the camera are discarded, as are points that project outside the image. The result associates each LiDAR point with a pixel, and so with whatever the camera knows about that pixel: its colour, its gradient, the class a segmentation network assigned it. The reverse direction is harder. A pixel corresponds not to a point but to a ray, and finding where that ray hits the world requires depth, which the camera does not have. Fusion methods that go from image to three dimensions must either borrow depth from the LiDAR, estimate it from the image, or keep a distribution over possible depths along the ray. Two complications attend projection. The first is parallax and occlusion: the LiDAR and camera are mounted at different positions, so they see slightly different things. A LiDAR point on a wall behind a post may be visible to the LiDAR but hidden from the camera by the post; projected naively, it lands on the post's pixels and inherits the post's label. Careful pipelines detect such cases by checking depth ordering among nearby projected points and discard the occluded ones. The second is density mismatch: a camera may have two million pixels, while a sixteen-beam LiDAR scan may put only a few hundred points on a pedestrian at twenty metres. Most pixels have no LiDAR point, and the few that do are arranged in sparse horizontal lines. Three places to fuse The literature divides fusion approaches according to how processed each signal is when the two are combined. The labels vary, but three levels capture the main design space, and Table 2 sets out what each typically involves. Table 2. Levels at which LiDAR and camera data can be fused. Level What is combined Main advantage Main cost Early Raw points and pixels Preserves all information Most sensitive to calibration and timing Middle Learned or engineered features Flexible, strong accuracy Needs training data; opaque failure Late Per-sensor detections or tracks Modular, degrades gracefully Discards cross-sensor evidence Early fusion In early fusion, the raw data are combined before any interpretation. The simplest form is colourising a point cloud: project each LiDAR point into the image and attach the pixel's colour. A richer form, sometimes called depth completion, goes the other way: project the sparse LiDAR points into the image to create a sparse depth map, then use the image to fill in the gaps, on the assumption that depth tends to be smooth within regions of similar colour and to change at image edges. The result is a dense depth image registered to the camera, which downstream modules can treat as a high-resolution depth sensor. Early fusion keeps the most information, which is why it can, in principle, be the most accurate. It is also the most demanding on calibration and synchronization, because any misalignment is baked into the fused data from the start. A one-degree extrinsic error or a twenty-millisecond timing error on a turning robot produces fused points whose colour or depth belongs to a neighbouring object, and nothing downstream can untangle it. Middle fusion Middle fusion combines intermediate representations, features computed separately from each sensor, before final decisions are made. This is where much of the progress of the last decade has been, driven by deep learning. An influential early idea in this family is point painting, introduced by Vora and colleagues in 2020. A semantic segmentation network runs on the image and assigns each pixel class scores. Each LiDAR point is projected into the image and painted with the scores of its pixel. The painted point cloud then goes to an ordinary LiDAR object detector, which now sees not just geometry but a semantic hint attached to every point. The approach is attractive because it can be bolted onto existing detectors. A second idea uses the camera to propose and the LiDAR to confirm. A two-dimensional object detector finds a box in the image; the box defines a frustum, a pyramid of space extending from the camera through the box; only the LiDAR points inside the frustum are passed to a three-dimensional detector that estimates the object's position and extent. Frustum PointNets, published by Charles Qi and colleagues in 2018, is the best-known example. It is efficient, because the expensive three-dimensional processing runs only on small regions, but it inherits the camera detector's misses: an object the camera fails to see is never examined. The dominant contemporary approach fuses in a bird's-eye-view representation. The idea is to convert both sensors' features into a common top-down grid around the robot, where geometry is natural and where both can be combined by simple concatenation or attention. LiDAR features are easy to place in such a grid. Camera features are harder, because placing them requires depth; methods following the lift-splat approach predict a distribution over depth for each pixel, spread the pixel's features along its ray accordingly, and pool the results into grid cells. BEVFusion, from a group at MIT and presented at ICRA in 2023, demonstrated that this unified representation could serve multiple tasks, detection and map segmentation, at once, efficiently. Variants of bird's-eye-view fusion now underpin much of the autonomous-driving perception literature and are increasingly appearing in outdoor robots. Middle fusion's strengths are accuracy and flexibility: the network learns which sensor to trust for which kind of object, in which conditions. Its costs are equally clear. It requires large quantities of labelled training data that match the deployment environment; a model trained on street scenes knows nothing about pallet jacks. Its failures are opaque, so that when it misses an object it is hard to say whether the camera branch, the LiDAR branch or the fusion layer was at fault. And many such networks degrade badly when one sensor fails: a model that always saw both modalities in training may produce garbage when the camera is blinded by sun, unless it was deliberately trained with one modality randomly dropped. Late fusion In late fusion, each sensor runs its own complete perception pipeline, producing detections or tracks, and the outputs are combined at the end. A LiDAR pipeline clusters points into objects and estimates their positions and sizes; a camera pipeline detects objects and classifies them; a fusion module associates detections that refer to the same physical object and merges their attributes, taking position from the LiDAR and class from the camera, for example. Late fusion is the oldest and, in industrial robotics, still the most common architecture. Its appeal is engineering, not accuracy. Each pipeline can be developed, tested and certified independently. When the camera fails, the LiDAR pipeline continues, and the system degrades gracefully to a known, reduced capability. Failures are easy to attribute. The cost is that each pipeline must make decisions on its own evidence alone. An object that is ambiguous to the LiDAR and ambiguous to the camera, but unambiguous when both are considered together, will be missed by both. Association and tracking Whatever level of fusion is chosen, a robot operating among moving objects must track them over time, and tracking introduces a fusion problem of its own: deciding which detection at time t corresponds to which object at time t minus one, and which detection from the LiDAR corresponds to which from the camera. The classic machinery is again the Kalman filter, one per tracked object, predicting each object's position forward with a motion model, typically constant velocity, and correcting it with associated detections. Association is posed as an assignment problem: compute a cost for each pairing of predicted track and new detection, usually based on distance normalized by the track's uncertainty, the so-called Mahalanobis distance, and find the assignment of minimum total cost, often with the Hungarian algorithm. A gate rules out pairings that are statistically implausible, so that a detection twenty metres from any existing track starts a new track rather than being forced onto an old one. Gating is where the honest uncertainty of Chapter 1 pays off directly. A track whose covariance has grown because the object has not been seen for a while has a large gate and can absorb a detection that appears some distance from its predicted position, which is exactly right for a person who stepped behind a pillar and emerged. A track whose covariance is overconfident has a small gate, rejects the reappearing person, and starts a duplicate track, so that the planner suddenly sees two people where there is one, and may lose the one it was carefully avoiding. More sophisticated trackers maintain multiple association hypotheses until the evidence disambiguates them, or treat the whole set of objects as a random finite set. Learned trackers now associate objects using appearance embeddings computed by networks, which helps when objects cross paths. For warehouse robots, where the moving objects are mainly people, forklifts and other robots, a well-tuned Kalman tracker with Mahalanobis gating remains the norm. Fusion for robots, not just cars Most published LiDAR-camera fusion research targets road vehicles, with large, labelled public datasets and a defined set of object classes. Warehouse robots and drones differ in ways that change the design. Warehouse robots operate in a closed, mostly known environment. The static structure, racks, walls, columns, is in the map. What perception mainly needs to find is what is not in the map: people, forklifts, dropped cartons, misplaced pallets, and the forks of a stationary forklift protruding into an aisle at shin height. This favours a simple, robust geometric pipeline, LiDAR or depth points that are not explained by the map become obstacles, with camera-based classification added mainly to identify people, whose presence may trigger different speed limits or keep-out distances. The forks are a well-known hazard because a planar LiDAR mounted at one height may pass entirely over or under them; robots that share space with forklifts often add downward-angled depth cameras precisely to catch thin horizontal obstacles. Drones have tighter constraints on weight and power, so their sensor suites are lighter: often stereo or depth cameras for obstacle detection, sometimes a small solid-state LiDAR, and almost always an IMU. Fusion on a drone is frequently geometric and fast, aiming to build a local occupancy map around the vehicle at high rate rather than to classify objects. Thin obstacles, wires, branches and guy lines, are the classic drone hazard, and they are hard for every sensor: too thin for most LiDAR beams to hit reliably, too low in contrast for stereo matching against the sky. Specialized wire-detection methods and conservative flight envelopes around known infrastructure are the usual responses. Seeing what is not there Most discussion of fusion concerns detecting objects. For a mobile robot, an equally important question is the opposite one: where is it certain that there is nothing? A planner needs free space, not just obstacles, and the two are not symmetric. An obstacle is evidenced by a return; free space is evidenced by the absence of one along a ray that would have hit something if anything had been there. LiDAR provides this evidence naturally. Each beam that returns from a wall twenty metres away testifies that the space along its path is empty, at least at that height and at that instant. Mapping systems exploit this by ray casting: every cell a beam passes through has its probability of occupancy reduced, and the cell where it ends has its probability increased. A beam that returns nothing, however, is ambiguous. It may have travelled to the sensor's maximum range through empty space, or it may have struck a black tyre or a sheet of glass that returned too little light. Treating no-return beams as free space up to maximum range is the more permissive choice and is often wrong in warehouses; treating them as unknown is safer but leaves holes in the free-space map. Cameras contribute free-space evidence in a different way. A segmentation network trained to recognize drivable floor can label every pixel that shows clear floor, and with a known camera height and a flat-floor assumption each such pixel maps to a point on the ground. Fusing this with LiDAR free space helps exactly where LiDAR is weak: the camera sees that the patch of floor ahead is clear even if the LiDAR had no return from it. The hardest case is the negative obstacle, a hole rather than a bump. Loading-dock edges, open floor hatches, the top of a staircase and the edge of a mezzanine are all places where the floor ends. A horizontal planar LiDAR sees nothing unusual: there is no surface to return a beam. A downward-angled depth camera or LiDAR sees the floor stop and then either nothing or a surface far below. Detecting negative obstacles therefore requires deliberately looking at the floor ahead, and treating the absence of expected floor returns as a hazard rather than as free space. Warehouse operators also address the risk structurally, by marking dock edges as keep-out zones in the map and fitting physical barriers; but a robot that relies solely on the map to know where the floor ends is one map error away from a fall. For drones, the equivalent of free-space reasoning is knowing that the volume ahead along the planned trajectory has actually been observed. A drone flying fast toward a region it has not yet seen must slow enough that, if its sensors reveal an obstacle at the edge of their range, it can still stop. Fusion that tracks not only occupied and free but also unobserved space is what allows a planner to enforce this. Designing for sensor failure The final question about any fusion design is what happens when one sensor stops working. Cameras are blinded by sun, darkness, condensation or a splash of mud; LiDARs by dust, fog or a misbehaving neighbour; any sensor by a loose cable. A fusion design is only as safe as its behaviour in these states. Three principles help. First, detect degradation explicitly. Each sensor's health should be monitored: image brightness histograms and feature counts for cameras, return counts and range statistics for LiDARs, message rates and timestamps for everything. A blinded camera that keeps publishing images is more dangerous than one that stops, because downstream modules continue to trust it. Second, define degraded modes in advance. A robot whose camera has failed may continue in LiDAR-only mode at reduced speed, without the people-specific behaviours that depended on the camera, and with its fleet manager informed. Third, test the degraded modes, by injecting failures in simulation and on real robots, rather than assuming that a learned fusion model or a carefully written fusion module will behave sensibly in conditions it has never seen. Fusion, in other words, is not only the art of combining sensors when they work; it is also the art of knowing when they do not. Both depend on the system knowing precisely how much each sensor's evidence is worth at each moment. The next chapter applies the same principle to a different question: not what is around the robot, but how the robot itself has moved. CHAPTER 5 Odometry: Knowing How Far You Have Moved Before a robot can know where it is in the world, it needs to know how it has moved since a moment ago. This is odometry: the estimation of incremental motion from sensor data, integrated over time to give a trajectory. Odometry is always relative, and it always drifts, because each small error in each increment is carried forward into everything that follows. Yet it is the backbone of every mobile robot's state estimate, the smooth, continuous, high-rate signal that controllers steer by and that localization and mapping correct only occasionally. The quality of a robot's odometry largely determines how hard everything downstream has to work. The name comes from wheel odometry, counting wheel turns, and the principles are visible there in their simplest form. A differential-drive robot with encoders on its two wheels computes, at each step, how far each wheel moved; the average gives forward distance and the difference, divided by the track width, gives the change in heading. Integrated, these give a pose. Errors in wheel radius cause scale drift; errors in track width cause heading drift, which over a long straight drive turns into large lateral error; slip on wet or dusty floors injects sudden errors that no model anticipates. Wheel odometry alone is good for a few metres. The methods in this chapter, scan-based, visual and inertial, aim for drift of a fraction of a percent of distance travelled, so that a robot can go hundreds of metres between external corrections, or, in a drone's case, fly without satellite positioning at all. Registering scans The core operation of LiDAR odometry is registration: finding the rigid transform that best aligns a new scan with a previous scan or with a local map. If the robot has moved forward thirty centimetres and turned two degrees, the transform that aligns the two scans is exactly that motion. The classic method is the iterative closest point algorithm, introduced in its widely cited form by Paul Besl and Neil McKay in 1992, with a closely related formulation by Yang Chen and Gérard Medioni around the same time. Its logic is a loop. Start with a guess of the transform, perhaps from wheel odometry or an IMU prediction. Apply it to the new scan. For each point in the transformed new scan, find the nearest point in the reference. Compute the transform that minimizes the total distance between these paired points. Apply it, and repeat until the change becomes negligible. ICP works when the initial guess is close enough that most nearest neighbours are true correspondences. When the guess is poor, it converges to the wrong answer, a local minimum, and reports success. Much of the engineering of practical LiDAR odometry is about providing a good initial guess, usually from the IMU, and about making the matching robust to points that have no true correspondence: moving people, newly placed pallets, the far edge of a scan that overlaps nothing in the reference. Two refinements matter in practice. Point-to-plane ICP measures the distance from each point not to its nearest neighbour but to the local surface around that neighbour. Because LiDAR samples a surface at different spots on each sweep, point-to-point distances never shrink to zero even at the perfect alignment; point-to-plane distances do, and convergence is faster and more accurate on structured scenes. Robust loss functions reduce the influence of pairs with large residuals, so that a few outliers from a passing forklift do not drag the solution. Feature-based LiDAR odometry and its successors Registering every point is expensive with a dense three-dimensional LiDAR. In 2014 Ji Zhang and Sanjiv Singh published LOAM, LiDAR odometry and mapping, which became the template for a generation of systems. LOAM extracts two kinds of feature from each sweep by examining the local curvature along each scan line: edge points, where curvature is high, such as corners and poles, and planar points, where it is low, such as walls and floors. Edge points are matched to lines and planar points to planes in the reference. LOAM also split the problem into two threads running at different rates: a fast odometry thread that registers each sweep to the previous one, and a slower mapping thread that registers to an accumulated local map for accuracy. The split, a quick estimate for control and a careful one for the map, is a pattern that recurs throughout the field. Many descendants refined the formula: versions tuned for ground vehicles that exploit the floor plane, versions for solid-state sensors with irregular scan patterns, and versions that fuse IMU data more tightly. A counter-current argued that much of this machinery was unnecessary. KISS-ICP, published by Ignacio Vizzo and colleagues in 2023 under the subtitle "in defense of point-to-point ICP", showed that a carefully engineered point-to-point registration with a constant-velocity motion prior, adaptive outlier thresholds and voxel downsampling could match feature-based systems across a wide range of sensors and platforms with almost no parameter tuning. The lesson for practitioners is that the details around the core algorithm, how points are downsampled, how outliers are thresholded, how the motion prior is formed, often matter more than the choice of algorithm itself. Where LiDAR odometry fails Scan registration fails in geometrically degenerate environments, places where the structure does not constrain all six degrees of motion. The corridor of Chapter 1 is the canonical case: two parallel walls constrain sideways position and heading but not forward motion. Long tunnels, open fields with only a ground plane, a featureless warehouse aisle between identical racks with little structure along its length, and the open sky above a drone are all partly degenerate. In such places the registration problem has a direction along which the cost is almost flat, and the optimizer may slide along it, producing odometry that underestimates or overestimates motion along the corridor while remaining confident. Detecting degeneracy is possible: the curvature of the cost surface in each direction, computed from the eigenvalues of the matrix the optimizer already builds, reveals which directions are poorly constrained. A well-designed system then refuses to update the estimate in those directions from the LiDAR, leaving them to wheels, IMU or other sensors. This is a direct application of honest uncertainty: the LiDAR is not wrong in a corridor, it is uninformative along one axis, and saying so prevents its silence from being mistaken for a confident claim of no motion. Visual odometry Visual odometry estimates motion from a sequence of camera images. The two main families differ in what they track. Feature-based methods detect distinctive points, corners and blobs, in each image and describe each with a compact descriptor of its surrounding appearance. Matching descriptors across frames gives correspondences, from which the camera motion and the three-dimensional positions of the points can be estimated by the geometry of multiple views. Outliers among the matches, inevitable with repetitive textures, are rejected by robust estimation such as RANSAC, which repeatedly fits a motion to small random subsets and keeps the one most matches agree with. Feature-based methods are fast and handle large motions between frames well. Direct methods skip features and instead adjust the estimated motion and depth to minimize the difference in pixel brightness between images, using every pixel with useful gradient. They can exploit weak texture that feature detectors ignore, such as the shading on a smooth wall, but they assume brightness constancy, which automatic exposure and lighting changes violate, and they need a good initial guess. A single camera recovers motion only up to an unknown scale: the same image sequence could come from a small robot moving slowly through a model or a large one moving quickly through a real building. Stereo cameras resolve scale through their known baseline; a monocular camera needs another source, and the usual one is an IMU. Tightly coupled inertial fusion The IMU and the exteroceptive sensors are almost perfectly complementary. The IMU is fast, environment-independent and accurate over fractions of a second, but drifts within seconds. Cameras and LiDARs are slower and environment-dependent, but do not drift between frames. Fused well, the IMU carries the estimate through blur, darkness and degenerate geometry for the short periods they last, supplies the initial guess that registration needs, provides metric scale and the direction of gravity, and in return has its biases continuously estimated and corrected. Loosely coupled fusion runs a complete visual or LiDAR odometry pipeline and feeds its pose outputs into a filter alongside the IMU. It is simple and modular, but it discards information: when the camera sees only two good features, too few for the visual pipeline to produce a pose, a loosely coupled system gets nothing from them. Tightly coupled fusion feeds the raw measurements, individual feature observations or individual point-to-plane residuals, into a single estimator with the IMU, so that every measurement contributes whatever constraint it can. Two features are not enough to estimate a pose alone, but combined with an IMU prediction they still reduce uncertainty. Tightly coupled systems are more accurate and more robust, and they are now the norm. On the visual side, two lineages dominate. The multi-state constraint Kalman filter, published by Anastasios Mourikis and Stergios Roumeliotis in 2007, is a filter that keeps a window of recent camera poses in its state and uses each feature's observations across that window as a constraint on the poses, without ever adding the feature's position to the state. It is efficient enough to run on phones and small drones and has been the basis of several commercial systems. Optimization-based visual-inertial odometry instead solves a small nonlinear least-squares problem over a sliding window of recent states; VINS-Mono, published by Tong Qin, Peiliang Li and Shaojie Shen in 2018, is a widely used open-source example, and the same group's work has been extended to stereo and multi-sensor configurations. An enabling technique for optimization-based methods is IMU preintegration. Between two camera frames the IMU produces dozens or hundreds of readings. Naively, each change in the estimate of the starting state would require re-integrating all of them. Preintegration, given a rigorous treatment on the manifold of rotations by Christian Forster and colleagues in 2017, summarizes the readings between two frames as a single relative-motion measurement, with a first-order correction for changes in the bias estimate, so that the optimizer can treat it as one constraint between two states. On the LiDAR side, the same pattern emerged. LIO-SAM, from Tixiao Shan and colleagues in 2020, built LiDAR-inertial odometry on a factor graph, with IMU preintegration, LiDAR registration and optional satellite measurements as factors. FAST-LIO2, from Wei Xu and colleagues in 2022, used an iterated extended Kalman filter, registering raw points directly to a map stored in an incremental tree structure designed for fast insertion and nearest-neighbour search, and achieved high rates on modest embedded computers. These systems, and their many descendants, now provide the motion backbone of a large share of research and commercial drones and outdoor robots. Starting up: the initialization problem An estimator that fuses inertial data must begin somewhere, and the starting point is less trivial than it appears. A visual-inertial system needs, before it can run, an estimate of the direction of gravity, the IMU biases, the initial velocity and, for a monocular camera, the metric scale of the scene. None of these is known at power-on. If the system starts with a poor guess, the nonlinear optimization may converge to a wrong solution and stay there, producing a trajectory that is tilted, mis-scaled or both. Ground robots usually avoid the problem by starting at rest. While stationary, the accelerometers measure gravity alone, which fixes roll and pitch, and the average gyroscope output is the gyroscope bias, since the true rotation rate is zero. Velocity is zero by assumption. Many systems detect stationary periods throughout operation, not just at start-up, and apply a zero-velocity update whenever the robot is stopped at a pick station or waiting at an intersection, which re-anchors velocity and refines bias estimates for free. Drones and handheld devices often cannot start at rest in a convenient way, and monocular visual-inertial systems need motion to observe scale at all. Initialization procedures therefore run a short visual-only reconstruction over the first second or two of motion, which recovers the trajectory up to scale, then align it with the integrated IMU measurements to solve for scale, gravity direction, velocity and gyroscope bias together. This alignment requires sufficient acceleration in more than one direction; a drone that takes off and climbs straight up at constant speed gives the estimator almost nothing to work with. Practical systems check the conditioning of the alignment and refuse to declare initialization complete until the motion has been informative enough, which is another instance of an estimator declining to be confident without evidence. The same issue reappears whenever an estimator must be restarted in flight or on the move after a failure, for example when a drone's visual tracking is lost in darkness and recovers. Reinitialization while moving is harder than initialization at rest, and drones that depend on visual-inertial odometry for position hold commonly fall back to attitude-only stabilization, holding level but drifting with the wind, until the estimator has reinitialized and passed its own consistency checks. Drift in numbers Some rough arithmetic shows why the choice of sensors matters so much for a drone without satellite positioning. Suppose a small inspection drone flies a two-kilometre route inside a large tunnel. Good LiDAR-inertial odometry in a well-structured environment might achieve drift on the order of a few tenths of a percent of distance travelled, which would leave the drone a few metres from its estimated position at the end of the route, enough for inspection imagery to be tagged to the wrong segment but not enough for the drone to lose its way between walls. In a long, smooth, featureless section of the same tunnel, the forward direction becomes degenerate, and the effective drift along the tunnel axis may grow many times larger. Pure inertial integration with a consumer-grade IMU over the same route would be useless within seconds. The figures are illustrative rather than measured, but the structure is general: drift depends less on the algorithm than on whether the environment and the sensor suite together constrain every direction of motion. Wheels still matter For ground robots, wheel odometry remains a valuable input to the fusion, and warehouse robots in particular rarely run without it. On a flat floor, wheel odometry constrains exactly the motions a planar LiDAR can be weak on, forward progress along a uniform aisle, and the robot's nonholonomic constraint, that a wheeled base cannot slide sideways, is a powerful extra piece of information. The engineering task is to know when to stop believing the wheels. A slip detector compares wheel odometry with the IMU and the LiDAR; when they diverge sharply, as when a robot's drive wheel spins on a patch of shrink-wrap or the robot is pushed by hand, the wheels' weight in the fusion is temporarily reduced. Measuring drift honestly Odometry systems are compared by their drift: how far the estimated trajectory diverges from ground truth per unit distance. Public benchmarks report relative errors, the error over fixed-length segments, and absolute trajectory errors after aligning the whole trajectory to ground truth. Both are useful, and both can mislead. A system tuned to a benchmark's sensor and environment may not transfer to a warehouse with a different LiDAR and a different floor. A low average error can hide rare, catastrophic failures, the moments when registration converged to a local minimum and the estimate jumped by a metre, which matter far more to a deployed robot than a slightly lower average. The appropriate question for an engineer is not "how accurate is this odometry on average?" but "how does it fail, how often, and does it know when it has failed?" An odometry module that reports a covariance consistent with its errors, inflates that covariance in degenerate geometry and flags registration that converged poorly is worth more than a slightly more accurate module that never doubts itself. No matter how good it is, odometry drifts. Over a long run, small errors accumulate into a trajectory that no longer lines up with the world. Correcting that drift requires recognizing places the robot has been before and relating its estimate to a map. That is the problem of simultaneous localization and mapping. CHAPTER 6 SLAM: Building the Map While Standing on It A robot dropped into an unfamiliar building faces a chicken-and-egg problem. To know where it is, it needs a map; to build a map, it needs to know where it is when it observes each wall and shelf. Simultaneous localization and mapping, universally abbreviated SLAM, is the name for solving both at once. The problem was formulated in the late 1980s and early 1990s, given its acronym in the mid-1990s, and surveyed in a widely read two-part tutorial by Hugh Durrant-Whyte and Tim Bailey in 2006. A decade later, Cesar Cadena and colleagues surveyed the field again under the title "Past, Present, and Future of Simultaneous Localization and Mapping", arguing that SLAM had entered an era of robustness, where the open questions were less about the core mathematics than about making systems that keep working for months in changing environments. That is the right emphasis for the engineer. The core of SLAM is now well understood. What separates a demonstration from a deployment is everything around the core: how the system recognises places, how it rejects false recognitions, how it copes when the world changes, and how it decides that its map is wrong. Why odometry is not enough Odometry, as the previous chapter showed, integrates small relative motions and therefore drifts. A robot driving a large rectangular loop around a warehouse will, on returning to its starting point, find that its odometry places it somewhere else: a few centimetres away with excellent LiDAR-inertial odometry, perhaps a metre with poor wheel-based odometry. If the robot has been building a map from its odometry along the way, that map is distorted by the same drift: the loop does not close, and the same wall appears twice, slightly offset. The essential insight of SLAM is that errors in the robot's trajectory and errors in the map are correlated. If the robot's heading drifted by one degree halfway round the loop, every landmark it observed after that point is displaced in a consistent way. When the robot recognises its starting point and learns how far off its estimate is, that one piece of information can correct not just its current position but its entire past trajectory and every map feature observed along it. This is why a SLAM system must maintain the joint uncertainty of robot and map, and why the correction at loop closure is so powerful. Filters, particles and graphs The first practical SLAM systems used the extended Kalman filter, with a state vector containing the robot's pose and the positions of all landmarks, and a covariance matrix capturing every correlation between them. EKF-SLAM is conceptually clean, but its cost grows with the square of the number of landmarks, because every update touches the whole covariance matrix, and its linearization errors accumulate into inconsistency over long runs. It is now mainly of historical and pedagogical interest, though filter-based SLAM persists in small-scale and resource-constrained settings. Particle-filter SLAM, in the Rao-Blackwellized form, exploits the structure of the problem: if the robot's trajectory were known, mapping would be easy, because each landmark or grid cell could be estimated independently. So each particle carries a hypothesized trajectory and its own map conditioned on that trajectory. FastSLAM, developed by Michael Montemerlo, Sebastian Thrun and colleagues in the early 2000s, applied this to landmarks, and GMapping, from Giorgio Grisetti, Cyrill Stachniss and Wolfram Burgard, published in 2007, applied it to occupancy grids built from planar laser scans. GMapping became the default two-dimensional mapper in early ROS and built many of the first maps of commercial indoor robot deployments. Its weakness is that a finite number of particles cannot represent the uncertainty of a very long loop, so that by the time the loop closes, no particle may have the right trajectory. The modern approach is graph-based SLAM. The robot's poses at successive times become nodes in a graph. Each odometry measurement becomes an edge, a constraint, between consecutive poses, saying "these two poses are related by approximately this transform, with this uncertainty". Observations of landmarks become edges between poses and landmark nodes; registering a scan to a scan taken at an earlier pose becomes an edge between those two poses. A loop closure, the recognition that the robot has returned to a place seen before, becomes an edge between a recent pose and an old one. SLAM then reduces to finding the configuration of all nodes that best satisfies all edges simultaneously, weighted by their uncertainties: a nonlinear least-squares problem. The graph view is now usually expressed as a factor graph, a bipartite graph of variables and the probabilistic factors that constrain them, a formulation developed for robotics notably by Frank Dellaert and Michael Kaess, whose monograph Factor Graphs for Robot Perception (2017) is a standard reference. The key computational fact is sparsity. Each factor touches only a few variables, so the matrices involved in solving the problem are overwhelmingly zero, and sparse linear algebra solves them quickly. Incremental solvers such as iSAM2, which update the solution efficiently as new factors arrive rather than re-solving from scratch, made graph SLAM practical in real time. Open-source libraries, GTSAM, g2o and the Ceres solver among them, made it accessible, and most modern SLAM systems are built on one of them. Table 3 summarizes how the three families compare. Table 3. Families of SLAM back end. Family Represents Strength Weakness EKF One Gaussian over pose and map Simple, recursive Quadratic cost; inconsistent over long runs Particle Many trajectories, each with a map Handles ambiguity; good for small 2D maps Particle depletion on large loops Graph Poses and landmarks as variables, measurements as factors Accurate, re-linearizes, scales via sparsity Needs good loop closures; batch cost Front end and back end Practitioners divide a SLAM system into a front end and a back end. The front end turns raw sensor data into factors: it runs odometry, decides when to create a new pose node, often called a keyframe, extracts or registers features, and proposes loop closures. The back end takes the factors and solves the optimization. The division matters because most failures originate in the front end, while most of the mathematics lives in the back end. A back end given one wrong loop closure, a factor saying that two poses on opposite sides of the building are at the same place, will dutifully warp the entire map to satisfy it, folding aisles onto one another. Two systems illustrate mature designs. Google's Cartographer, described by Wolfgang Hess and colleagues at ICRA in 2016 and released as open source, builds small local submaps by scan matching, each an occupancy grid assembled from a few seconds of scans, and connects them in a pose graph. Loop closure searches match new scans against old submaps using a branch-and-bound method that makes exhaustive search over a wide window affordable. It was widely adopted for two-dimensional and three-dimensional LiDAR mapping in indoor robotics. On the visual side, ORB-SLAM, developed at the University of Zaragoza and culminating in ORB-SLAM3, published by Carlos Campos and colleagues in 2021, combines feature-based tracking, local bundle adjustment, a place-recognition database and support for monocular, stereo, RGB-D and visual-inertial configurations. It maintains multiple disconnected maps when tracking is lost and merges them when it recognises a shared place, a design that tolerates the inevitable tracking failures of real deployments. Recognising places, and the danger of recognising them wrongly Loop closure depends on place recognition: given the robot's current observation, which previously visited places could this be? Visual systems commonly use a bag of words approach, quantizing image features into a vocabulary of visual words and describing each image by which words it contains, so that candidate matches can be retrieved quickly from thousands of images; learned global descriptors now often replace or supplement it. LiDAR systems use compact descriptors of a scan's geometry, for example ones that summarize the maximum height of points in polar bins around the sensor, and learned descriptors as well. Candidates retrieved by the descriptor are then verified geometrically: the system attempts a full registration and accepts the loop closure only if the alignment is good. The danger is perceptual aliasing: two different places that look the same. Warehouses are among the most aliased environments in robotics. Aisle 14 looks exactly like aisle 15, the same racks at the same spacing, the same beam heights, the same uprights. Bay 30 of an aisle looks like bay 31. A robot that sees a scan matching aisle 15 perfectly may in fact be in aisle 14, and a SLAM system that accepts that loop closure will slide part of its map sideways by one aisle pitch. Defences operate at several levels. Geometric verification requires that the registered alignment is not only good but uniquely good; if a scan aligns equally well at several offsets, the match is ambiguous and should be rejected. Temporal consistency requires that a candidate loop closure be supported by several consecutive observations, not one. Robust back ends treat loop closures as possibly wrong: techniques such as switchable constraints, dynamic covariance scaling and graduated non-convexity let the optimizer down-weight or reject constraints that conflict with the rest of the graph, rather than bending everything to satisfy them. And in warehouses, the most practical defence is often to add distinctiveness: unique markers or codes at aisle ends, which turn an aliased problem into an unambiguous one at trivial cost. The broader principle is that a false loop closure is far more damaging than a missed one. A missed closure leaves drift uncorrected for a while longer; a false one corrupts the map, possibly permanently. Well-designed systems are therefore conservative, accepting only loop closures they are highly confident in, and they monitor the residuals of the optimized graph for signs that a constraint has been forced. Mapping once, localizing many times In many warehouse deployments, full SLAM runs only during commissioning. A technician drives a robot, or a dedicated mapping rig, through the facility once, the SLAM system builds a map, an engineer inspects and edits it, adding keep-out zones, marking doorways and removing transient clutter, and the cleaned map is distributed to the fleet. In daily operation, each robot runs localization only: estimating its pose against the fixed map, typically with a particle-filter localizer or by scan matching against the map, fused with odometry. This separation has real advantages. Localization against a fixed map is cheaper and more predictable than full SLAM. The map can be validated by a person before any robot relies on it. All robots share the same coordinate frame, which the fleet manager needs in order to coordinate them. And a robot that is merely localizing cannot corrupt the shared map with a bad loop closure. The price is that the world changes and the map does not. Warehouses are rearranged: racks are added, pallets are staged in aisles, seasonal displays appear. If enough of what the robot sees no longer matches the map, its localization degrades. Here the honest uncertainty of the localizer is again the key. A particle-filter localizer whose particles are spreading, or a scan matcher whose alignment score is falling, is telling the system that it no longer recognises its surroundings. A good system reacts by slowing down, by relying more heavily on odometry for a while, by alerting the fleet manager, and, in mature deployments, by feeding the discrepancies back into a map update process. Inside the localizer Because localization against a fixed map is what most deployed warehouse robots actually run all day, its mechanics deserve a closer look. The most widely used implementation in the ROS ecosystem is AMCL, adaptive Monte Carlo localization, the particle filter of Chapter 1 specialized to planar laser scans and occupancy-grid maps. Each particle is a hypothesis of the robot's pose. At each step, the particles are moved according to the odometry, with noise added in proportion to how far and how sharply the robot moved. Each particle is then scored by how well the current laser scan agrees with the map if the robot were at that particle's pose. A common and efficient scoring model, the likelihood field, precomputes for every cell of the map the distance to the nearest obstacle; a scan endpoint that falls close to a mapped obstacle scores well, one that falls in open space scores poorly. Particles are resampled according to their scores. The adaptive part, based on a technique known as KLD sampling introduced by Dieter Fox, varies the number of particles with the spread of the belief: a few hundred suffice when the robot is well localized, while thousands may be needed when it is uncertain. Several practical behaviours follow from this design. Because each endpoint is scored independently, a scan dominated by unmapped clutter, a row of pallets staged in an aisle, produces low scores everywhere, and the filter's estimate may be pulled toward wherever the few mapped structures still visible happen to fit best. Mixing a small amount of a uniform random component into the measurement model makes the filter tolerant of unmapped returns. The filter can also add random particles when the average score drops, which lets it recover if it has become lost, the kidnapped robot problem in its classic form, though in a highly aliased warehouse a randomly placed particle may find a convincing fit in the wrong aisle. Many commercial systems supplement or replace the particle filter with direct scan-to-map registration, refining the pose by optimization each cycle, which is more precise when the robot is already well localized. Whatever the method, the operational practice is the same. Initial localization at start-up is done at known places, a charging dock with a precisely surveyed pose or a marker the robot can read, rather than by global search. The localizer's confidence is monitored continuously. And a robot that loses confidence is stopped or slowed and asked to relocalize, rather than allowed to continue on a guess. Drones that operate repeatedly in the same structure, inspecting a warehouse's high bays or a tunnel, face the same choice in three dimensions. They may localize against a prior point-cloud map using LiDAR registration, gaining a consistent frame across flights at the cost of keeping the map current, or run full SLAM on every flight and align the results afterward. The former is increasingly common where repeat inspections must be compared over time. Lifelong mapping The aspiration of current research is lifelong SLAM: systems that keep their maps current indefinitely, distinguishing permanent changes, a new rack, from transient ones, a parked forklift, and from semi-static ones, a pallet that will be gone tomorrow. Approaches include maintaining multiple map layers with different expected lifetimes, tracking how often each part of the map is confirmed or contradicted, and pruning old keyframes so that the graph does not grow without bound. Some warehouse platforms now detect map changes automatically across the fleet and propose map edits for human approval, which is a sensible middle ground between a frozen map and one that any robot can change on its own. The field has also moved beyond geometric maps. Semantic maps label regions and objects, aisle, dock door, charging station, pallet location, so that robots and their fleet managers can reason in the vocabulary of the operation. Dense reconstructions, from truncated signed distance fields to neural and Gaussian-splat representations, produce detailed three-dimensional models valuable for inspection drones and digital twins, though their role in real-time navigation is still being settled. What has not changed is the structure of the problem. A SLAM system is an estimator that must decide, at every moment, how much to trust each piece of evidence, and it is only as good as its ability to say when it is unsure. With the robot localized, even approximately, against a map, even an imperfect one, the next question is where it should go and how it should get there. CHAPTER 7 Path Planning: From Map to Motion Knowing where the robot is and what surrounds it is only half the job. The robot must then decide how to move: which route to take across the building, and which precise sequence of velocities and turns will carry it along that route without striking anything, within the limits of its motors and its momentum. Path planning is the discipline that answers these questions, and it is best understood not as a single algorithm but as a stack of planners operating at different scales of space and time, each handing a problem to the next. The layered planner Almost every practical mobile-robot navigation system divides planning into at least two layers. A global planner computes a route from the robot's current position to its goal over the whole map, typically a few times per second or less, ignoring the robot's detailed dynamics and the finer details of moving obstacles. A local planner, often called a controller, runs much faster, tens of times per second, and computes the actual velocity commands that make the robot follow the global route while respecting its dynamics and avoiding obstacles that have just appeared. Some systems add a third, behavioural layer above both, deciding what the robot should be trying to do: proceed, wait, reroute, call for help. The ROS 2 navigation stack, Nav2, described by Steve Macenski and colleagues in 2020 and now used widely by commercial mobile-robot makers, makes this layering explicit and configurable. Planners and controllers are plugins; a behaviour tree orchestrates them, deciding when to replan, when to attempt recovery behaviours such as backing up or clearing the local map, and when to give up. The value of the layering is that each layer can be good at one thing. A global planner need not worry about the robot's acceleration limits; a local planner need not search the whole building. Representing free space Before searching for a path, the planner needs a representation of where the robot can and cannot go. The dominant representation for ground robots is the costmap: a grid laid over the map in which each cell holds a cost, from free through increasingly undesirable to lethal. Costmaps are usually built in layers. A static layer comes from the map. An obstacle layer is populated in real time from LiDAR and depth sensors, marking cells where obstacles are seen and clearing cells along sensor rays that pass through them unobstructed. Further layers can encode keep-out zones, preferred lanes, speed restrictions or the proximity of people. An inflation layer then spreads cost outward from every obstacle by the robot's radius, so that the planner can treat the robot as a point: any cell within one robot radius of an obstacle is lethal, and cells a little further out are expensive, encouraging paths that keep a margin. This trick, configuration-space inflation, is what allows a simple point-based search to plan for a robot of real size. It is exact for a circular robot. For a rectangular robot, or one carrying a shelf that overhangs its base, a circle large enough to contain the footprint is overly conservative in narrow aisles, and planners must instead check the actual footprint at each candidate orientation. The clearing behaviour of costmaps is a subtle source of trouble. An obstacle that is seen and marked, and that then moves out of the sensors' field of view, remains in the costmap until something clears it. A robot can thus become boxed in by ghosts: people who walked away, a forklift that left. Conversely, an obstacle that is not seen, a glass panel or a black tyre, never appears. Costmap tuning, how long marks persist, which sensors may clear which cells, how far rays clear, is a significant fraction of the practical work of deploying a mobile robot. Drones need three-dimensional representations. Occupancy grids extended to three dimensions are memory-hungry, so drone systems typically use octrees, which subdivide space only where there is detail, or signed distance fields, which store at each point the distance to the nearest obstacle and therefore give a planner both collision information and a gradient pointing away from obstacles. Several fast mapping systems build such fields incrementally from depth data at the rates a drone needs. Searching graphs The simplest global planners treat the costmap grid as a graph, with cells as nodes and edges to neighbours, and search for the lowest-cost path. Dijkstra's algorithm, published in 1959, finds it by expanding outward from the start in order of accumulated cost. The A* algorithm, published by Peter Hart, Nils Nilsson and Bertram Raphael in 1968, adds a heuristic: an estimate of the remaining cost to the goal, such as the straight-line distance. By expanding nodes in order of cost-so-far plus estimated cost-to-go, A* focuses its search toward the goal and, provided the heuristic never overestimates, still finds the optimal path. On a warehouse map of a few million cells, A* with a good heuristic finds a route in milliseconds. Grid paths have characteristic flaws. Restricted to eight directions, they zigzag; they hug obstacles at the minimum allowed margin; and they ignore the robot's heading, so that a path may demand an instantaneous ninety-degree turn that a car-like robot cannot make. Variants address each. Any-angle planners such as Theta* allow straight segments between non-adjacent cells. Post-processing smooths the path while respecting costs. And for robots with real kinematic constraints, the search itself can be changed. Hybrid A, developed by Dmitri Dolgov, Sebastian Thrun and colleagues for Stanford's entry in the 2007 DARPA Urban Challenge and published in 2010, searches over continuous position and heading but associates each search node with a grid cell to keep the search finite. From each node it expands by simulating short arcs the vehicle can actually drive, forward and reverse at several steering angles, so that every path it finds is drivable. State lattice* planners precompute a library of feasible motion primitives connecting discrete states and search over the graph they form. Both approaches are used in Nav2 for robots that cannot turn on the spot, such as tuggers, forklifts and Ackermann-steered vehicles. Sampling the space Grid search becomes impractical in high-dimensional spaces: a drone's position and velocity in three dimensions, a manipulator's joint angles, or a car's position, heading, speed and steering. The number of grid cells grows exponentially with dimension. Sampling-based planners avoid building the grid. The probabilistic roadmap, introduced in the mid-1990s by Lydia Kavraki, Jean-Claude Latombe and colleagues, samples random collision-free configurations and connects nearby ones with simple collision-free paths, building a graph that is then searched. It suits repeated queries in a static environment. The rapidly-exploring random tree, introduced by Steven LaValle in 1998, grows a tree from the start: it samples a random point, finds the nearest node in the tree, and extends toward the sample by a short feasible motion. The tree is biased toward unexplored space and spreads through it quickly. RRT is simple, handles kinodynamic constraints naturally because extensions can be simulated motions, and finds a path quickly, but the path is usually ugly and far from optimal. In 2011 Sertac Karaman and Emilio Frazzoli published a careful analysis showing that RRT almost surely does not converge to an optimal path, and introduced RRT, which rewires the tree as it grows so that each node is reached by its cheapest known route. RRT converges toward the optimum as samples accumulate. A large family of refinements followed, informed sampling that concentrates samples where a better path could lie, batch methods that combine sampling with graph search, and many others. Sampling-based planners dominate manipulation and are used for drones in cluttered three-dimensional environments, though for ground robots on two-dimensional maps graph search usually suffices. From path to trajectory A path is a geometric curve. A robot needs a trajectory: a path with timing, specifying where the robot should be at each instant and therefore its velocity and acceleration. The step from one to the other is where dynamics enter. For a wheeled warehouse robot, the dynamics are modest but real. A loaded robot carrying a full shelf may weigh well over a tonne and must accelerate and brake gently enough that the load does not shift or topple. Its turning rate may be limited when loaded. The trajectory must respect these limits and must slow before turns and near intersections. For a multirotor drone, dynamics dominate. A quadrotor's thrust acts along its body axis, so to accelerate sideways it must first tilt, and to tilt it must change the relative speeds of its rotors. Daniel Mellinger and Vijay Kumar showed in 2011 that a quadrotor is differentially flat: its full state and control inputs can be computed algebraically from its position and yaw and their time derivatives. This means a trajectory can be planned simply as a smooth curve in position and yaw, and any sufficiently smooth curve can be flown. Their method planned piecewise polynomial trajectories minimizing snap, the fourth derivative of position, which corresponds roughly to minimizing the rate of change of the rotor commands, through a sequence of waypoints. Minimum-snap trajectories, and their many descendants that add collision constraints by confining the trajectory to a chain of free-space corridors, are standard practice for agile drone flight. Optimizing locally The local planner faces a different problem: given the global path and the current costmap, choose velocity commands for the next fraction of a second. Three broad approaches are common. Pure pursuit and its regulated variants steer the robot toward a point a fixed distance ahead on the path, slowing near obstacles and on tight curves. They are simple, predictable and effective for following a clean path in a mostly static environment, which describes much warehouse driving. Sampling controllers simulate a set of candidate commands forward over a short horizon and score each resulting trajectory by its clearance from obstacles, progress toward the goal and closeness to the path. The dynamic window approach, discussed in the next chapter, is the classic member. A more recent and powerful member is model predictive path integral control, introduced by Grady Williams, Evangelos Theodorou and colleagues in 2016 for aggressive autonomous driving. MPPI samples thousands of noisy control sequences, simulates each over the horizon, and computes the next command as a weighted average in which low-cost sequences dominate. It handles nonlinear dynamics and arbitrary, even non-differentiable, cost functions, and it parallelizes naturally on modern processors. Nav2 includes an MPPI controller, and it has become a popular default for robots that must navigate dynamic spaces smoothly. Optimization-based model predictive control formulates the next few seconds of motion as a constrained optimization: minimize a cost combining path-tracking error, control effort and obstacle proximity, subject to the robot's dynamics and limits, solve it, apply the first command, and repeat at the next step with fresh sensor data. Timed elastic band planners, which deform a time-parameterized path under constraints, are one widely used variant for ground robots. Gradient-based optimization of this kind is the dominant approach for drone local planning, where the planner continually replans a smooth trajectory through a local signed distance field. One trip, every layer It is worth following a single trip through the layers to see how they interact. A mobile robot is sent from a pick station to a shelf location sixty metres away on the far side of a mixed floor shared with people. The behaviour layer receives the task and checks that it is feasible: the destination is reachable, the robot's battery will last, the route does not require entering a zone currently closed for maintenance. The global planner runs A* or a lattice search over the costmap and produces a route along the main aisle, then down a cross-aisle to the shelf. Because a preferred-lane layer adds cost to the left half of each aisle, the route keeps right, which makes the robot predictable to the pickers walking the other way. The route is published as a sequence of poses. The controller begins tracking it, generating velocity commands twenty or more times a second. As the robot accelerates down the main aisle, its obstacle layer marks a cart left at the side of the aisle that is not in the map. The global route passes close to the cart; the controller, scoring candidate trajectories against the inflated costmap, bends the robot's actual path a few tens of centimetres away from it without the global planner being involved. Half-way along, a picker steps into the aisle. The tracker picks them up, the controller's cost function penalizes trajectories that pass close to their predicted position, and the robot slows and edges further right. Had the picker stepped directly in front, the warning field of the safety scanner, discussed in the next chapter, would have forced a slowdown regardless. At the cross-aisle, the costmap shows that a pallet has been staged across it, blocking the route. The controller can no longer make progress. After a short wait, in case the obstruction is a person who will move, the behaviour layer asks the global planner to replan with the current costmap, and it finds a longer route via the next cross-aisle. The robot follows it, arrives at the shelf, and the controller hands over to a specialized docking behaviour that uses a short-range sensor or a fiducial marker for the final centimetres of alignment, where the costmap and the global localization are not precise enough. At no point did any single layer solve the whole problem. The global planner never considered the picker; the controller never considered the far cross-aisle; the docking behaviour ignored the map almost entirely. Each acted on the information appropriate to its scale, and the failure of one, the blocked route, was handled by escalation to another. The design of these hand-offs, when to wait, when to replan, when to call for help, is in practice as important as the quality of any individual planner, and it is where much of the tuning effort on a real deployment goes. Planning under uncertainty Everything above treats the map and the robot's pose as known. They are not, and planners handle this in mostly pragmatic ways. Inflating obstacles by a margin that includes localization uncertainty, not just the robot's radius, is the most common. Planning more conservatively, slower and with wider margins, when the localizer reports higher uncertainty is a natural extension: a robot that is unsure of its position to within ten centimetres should not try to thread a gap with five centimetres to spare. Treating unknown space differently from free space matters especially for drones exploring unmapped areas: a planner that treats unseen space as free will happily plan through the inside of an unseen wall, so most exploration planners treat unknown as occupied for commitment purposes while still planning toward it to see more. Formal treatments of planning under uncertainty exist, in particular the partially observable Markov decision process, which plans over beliefs rather than states and can reason about actions taken to reduce uncertainty, such as turning to look before entering an intersection. They are computationally demanding and appear in deployed robots mainly in simplified forms. The practical lesson is the one this book keeps returning to: the planner must receive the estimator's uncertainty, not just its best guess, and must spend that uncertainty as margin. Planning finds a route through the world as it is mapped and sensed. But the world contains things that move, and some of them are people. Avoiding collisions with moving agents, and doing so safely enough to satisfy regulators and the people who share the space, is the subject of the next chapter. CHAPTER 8 Collision Avoidance in Shared Spaces A robot that never moved among people would need no more than the planners of the last chapter. Real warehouse robots share aisles with pickers, forklifts and other robots; delivery drones share airspace with helicopters, light aircraft and other drones. Avoiding collisions with agents that move, decide and sometimes behave unpredictably is a harder and more consequential problem than avoiding walls, and the engineering response has two parts that must not be confused. One part is intelligent avoidance: perceiving, predicting and smoothly steering around other agents so that the robot is efficient and pleasant to work alongside. The other is safety: guaranteeing, to a standard that a regulator and an insurer will accept, that the robot stops or yields before it hurts someone, regardless of what the intelligent layer believes. This chapter covers both and argues that the separation between them is the single most important architectural decision in any robot that works near people. Reactive avoidance in velocity space The classic reactive methods work not in the space of positions but in the space of velocities the robot could choose. The dynamic window approach, published by Dieter Fox, Wolfram Burgard and Sebastian Thrun in 1997, observes that a robot with finite acceleration can only reach a small set of velocities in the next control interval: a window around its current forward and turning speeds. Within that window, it discards velocities that would bring the robot into collision before it could stop, because the distance to the nearest obstacle along the resulting arc is shorter than the braking distance. Among the remaining admissible velocities, it picks the one maximizing a weighted combination of heading toward the goal, clearance from obstacles and speed. DWA is simple, fast and respects dynamics by construction, and versions of it remain in wide use as local controllers. Its limitation is that it treats obstacles as static over the short arcs it evaluates, so a person walking across the robot's path appears only as an obstacle where the person is now, not where they will be. Velocity obstacles, introduced by Paolo Fiorini and Zvi Shiller in 1998, address moving obstacles directly. For each moving obstacle, with known position and velocity, the velocity obstacle is the set of robot velocities that would result in a collision at some time in the future if both continued at constant velocity. Geometrically it is a cone in velocity space. Any velocity outside the union of all cones is collision-free, at least for the assumed horizon and under the constant-velocity assumption, and the robot picks the one closest to its preferred velocity. When two robots each apply velocity obstacles to the other, a problem appears: each swerves fully to avoid the other, both overcorrect, and they oscillate. Reciprocal velocity obstacles, and their refinement optimal reciprocal collision avoidance (ORCA), published by Jur van den Berg, Stephen Guy, Ming Lin and Dinesh Manocha, assume that each agent will take half of the responsibility for avoidance. ORCA converts each pairwise constraint into a half-plane in velocity space and solves for the best velocity with a small linear program, which scales to thousands of agents. It is widely used in crowd simulation and for coordinating fleets of identical robots, where the reciprocity assumption holds because the other agents really are running the same algorithm. It is shakier with people, who do not run ORCA and who may not take their half of the responsibility, especially if they are looking at a handheld scanner. Predicting people Avoiding people well requires predicting where they will go, and constant velocity is a crude prediction. A person walking down an aisle toward a cross-aisle may turn, stop to pick an item, or step aside for the robot. Much research addresses human trajectory prediction: physics-inspired models such as the social force model of Dirk Helbing and Péter Molnár, which treats pedestrians as particles attracted to goals and repelled by obstacles and each other; planning-based models that infer a person's goal and predict a path toward it; and learned models trained on large datasets of pedestrian tracks, which predict a distribution over future paths conditioned on the scene and on other people. For a warehouse robot, the benefit of good prediction is efficiency and comfort more than safety. A robot that anticipates that a picker is about to step into the aisle can slow early and smoothly instead of braking hard, and a robot that predicts a person will continue past an intersection can wait a moment rather than forcing an awkward standoff. People rate robots as better co-workers when their motion is predictable and legible, when it is obvious what the robot intends. Lights, projected floor markings showing the robot's intended path, and consistent rules such as always yielding at intersections and keeping to one side of an aisle often do more for safety in practice than sophisticated prediction, because they make the robot predictable to people, which is the other half of the problem. There is also a known failure of naive avoidance in crowds, sometimes called the freezing robot problem. If a robot predicts each person independently, with uncertainty that grows over time, then in a dense crowd the union of all predicted positions can cover the entire space ahead, and the robot concludes that no safe path exists and stops. People navigate crowds by cooperating, by expecting others to make room, and approaches that model this interaction, rather than treating people as independent moving obstacles, avoid freezing at the cost of assuming that people will in fact cooperate. Safety is a separate function None of the above is safety in the engineering sense. A learned trajectory predictor, a tuned MPPI controller and a ROS navigation stack running on a general-purpose computer are not designed, verified or certified to the level of reliability required for a function whose failure could injure someone. They run on operating systems that may stall, use software too complex to verify exhaustively, and depend on perception that, as earlier chapters showed, can fail in characteristic ways. Industrial practice therefore separates the safety function from the autonomy stack. The autonomy stack plans, avoids and optimizes. The safety function independently monitors the space immediately around the robot and stops it, through a certified path to the motor drives, if anything enters a defined protective zone, regardless of what the autonomy stack is commanding. The safety function is deliberately simple, so that it can be verified, and it runs on safety-rated hardware: safety laser scanners certified to the relevant electro-sensitive protective equipment standard, IEC 61496-3, and safety controllers designed to a specified performance level under ISO 13849-1. The primary standard for driverless industrial vehicles, including warehouse robots, is ISO 3691-4, Industrial trucks — Safety requirements and verification — Part 4: Driverless industrial trucks and their systems, first published in 2020 and revised in 2023. In North America, ANSI/A3 R15.08, published in parts, covers industrial mobile robots. These standards require, among much else, that a manufacturer identify the hazards of the robot in its intended operating environment and provide protective functions with an appropriate level of integrity, including personnel detection in the direction of travel, speed limits appropriate to zones, and verified stopping distances. They also place obligations on the user, who must, for example, define and mark operating zones and ensure that the environment matches what the robot was designed for. Protective fields The central mechanism of the safety function on most warehouse robots is the protective field of a safety laser scanner. The scanner is configured with a set of fields, regions on the floor plane around the robot, and a safety-rated output that trips whenever an object is detected inside the active field. When it trips, the safety controller removes power from the drives or commands a safety-rated stop. Because a robot moving faster needs more distance to stop, fields are switched by speed. The safety controller reads wheel speeds through safety-rated encoders and selects a larger protective field at higher speed and a smaller one at lower speed, so that the robot can approach a shelf slowly to pick it up without tripping, while at full speed it stops for any person well ahead. Fields may also switch by direction, extending to the side the robot is turning toward, and by load, since a loaded robot brakes more slowly. Outside the protective field, a larger warning field typically does not stop the robot but signals the autonomy stack to slow down, which, if the autonomy stack is working, means the protective field is rarely tripped at all. This arrangement is the industrial form of a principle known in collaborative robotics as speed and separation monitoring: the robot may move as fast as it likes, so long as the distance to any person always exceeds the distance it would need to stop. The calculation of the required distance includes the robot's speed, its worst-case braking performance, the reaction time of the scanner and controller, the possible approach speed of a person and a margin for measurement error. Every term is verified by testing, and the result is the field geometry. Table 4 sets out the layers of protection that typically coexist on a warehouse robot. Table 4. Layers of collision protection on a typical warehouse robot. Layer Implemented by Purpose Certified? Planning Global planner, costmaps Avoid known obstacles and congested routes No Avoidance Local controller, tracking, prediction Steer smoothly around people and robots No Warning Scanner warning field Slow before a stop is needed Usually Protective Safety scanner, safety controller Stop before contact, whatever autonomy does Yes Physical Bumpers, emergency stops Last resort on contact or manual stop Yes The layering works only if the upper layers make the lower ones rare. A robot that trips its protective field every few minutes is safe but useless, and the operations team will be tempted to shrink the fields, which is exactly the wrong response. Equally, a robot whose autonomy stack is so conservative that it never reaches a warning field is safe but slow. The craft is to tune the intelligent layers so that the robot slows and steers early and naturally, leaving the certified layers as a backstop that people never notice. The logic of sizing a protective field can be followed with illustrative numbers. Suppose a robot travels at up to two metres per second. The scanner and safety controller together take, say, a tenth of a second to detect an intrusion and issue a stop; during that time the robot covers twenty centimetres. The drives and brakes then take time to bring the robot to rest; if the verified stopping distance from two metres per second, under the worst-case load and floor condition, is eighty centimetres, the robot needs at least a metre from detection to standstill. A person may be walking toward the robot during the same interval, which adds the distance they could cover in the total stopping time. Add an allowance for the scanner's measurement tolerance and for the fact that a person's leg may be detected only once it is fully inside the field, and the protective field at full speed may need to extend well over a metre ahead. At half a metre per second, the same terms shrink sharply, because braking distance falls roughly with the square of speed, and the field can be a fraction of the size. The numbers are invented for illustration, but the structure is not: it follows the general pattern used in machinery-safety standards for calculating minimum safety distances, a reaction-time term, a stopping term, an approach-speed term and a tolerance. Each term has an engineering consequence. Faster scanners and controllers shrink the first. Better brakes and lighter loads shrink the second. Lower top speeds in zones where people work shrink everything. The result is the familiar operating pattern of mixed warehouses: robots that move briskly in robot-only zones and slow markedly where people are present, not because the autonomy stack is timid but because the safety function's geometry requires it. There is a known gap in this architecture. Safety laser scanners on most robots see a single horizontal plane, often a few centimetres above the floor. Objects that do not intersect that plane, forklift forks raised a little, a table, the overhang of a shelf, are invisible to the safety function. Standards require the manufacturer to assess such hazards; in practice they are addressed by additional sensors, by operating rules that separate robots from forklifts, and by designing the operating zone so that the hazard cannot arise. This is a reminder that safety is a property of the robot in its environment, not of the robot alone. Avoiding collisions in the air Drones face collision hazards at two scales. The near scale is obstacles: buildings, trees, wires, terrain. Here the techniques are those of earlier chapters, local mapping from depth sensors, reactive and optimizing planners, conservative margins, with the twist that a drone cannot simply stop; it can hover, but hovering in wind and turbulence still requires space, and a drone flying at fifteen metres per second needs many metres to decelerate. Its planner must therefore ensure that it always has an escape trajectory available, a concept sometimes formalized as maintaining a known-safe stopping trajectory into observed free space. The far scale is other aircraft. A small drone flying beyond its operator's line of sight must be able to detect and avoid crewed aircraft, which may be travelling at fifty metres per second or more and which cannot be expected to see the drone. Approaches combine cooperative surveillance, receiving the position broadcasts that many crewed aircraft transmit through ADS-B, with non-cooperative detection by ground-based or onboard radar, acoustic or optical sensors, and with strategic separation through flight planning and airspace management. Industry standards set performance requirements for such detect-and-avoid systems, and the FAA has sponsored the development of an airborne collision avoidance logic tailored to small uncrewed aircraft, known as ACAS sXu. The proposed American rule for beyond-visual-line-of-sight operations, Part 108, published as a notice of proposed rulemaking in August 2025, would require drones to yield right of way to crewed aircraft broadcasting their position and to use detection systems to do so; at the time of writing it had been sent for final review but not yet published as a final rule. At both scales the same architectural lesson holds as on the warehouse floor. The intelligent planner is the primary means of avoiding collisions. Beneath it sits a simpler, more conservative and more verifiable layer, a geofence enforced by the flight controller, a return-to-home or controlled-landing behaviour triggered by loss of link or of navigation integrity, a parachute on larger delivery drones, which acts when the planner's assumptions have failed. The separation of an intelligent upper layer from a simple, verifiable lower one recurs whenever autonomous systems meet people. It is the engineering expression of the book's argument: the upper layers can be clever because the lower layers do not need to trust them. And it becomes even more important when there are not one but hundreds of robots in the space, which is the subject of the final chapter. CHAPTER 9 Fleets, Airspace and Operating at Scale A single robot that localizes well, plans sensibly and avoids people safely is a solved engineering problem in many settings. Hundreds of them in one building, or many drones in one airspace, raise problems that no individual robot can solve. Robots that each plan the shortest path to their goal converge on the same aisles and intersections and jam. Two robots meeting head-on in a one-wide aisle can deadlock, each waiting for the other to move. A drone that plans a perfect route may cross the route of another operator's drone that it cannot see. And a system of many machines running for months will encounter rare combinations of circumstances that no test plan anticipated. This chapter covers coordination at scale and the discipline of validating systems whose full behaviour cannot be tested exhaustively. The warehouse as a traffic system The modern goods-to-person warehouse inverted the traditional model of picking. Rather than people walking aisles to fetch items, robots bring storage units to people at fixed stations. Kiva Systems pioneered this at scale in the 2000s; Peter Wurman, Raffaello D'Andrea and Mick Mountz described the architecture in 2008 in a paper whose title, "Coordinating Hundreds of Cooperative, Autonomous Vehicles in Warehouses", names the central difficulty. Amazon bought Kiva in 2012 and, as its own announcements record, had deployed its millionth robot by mid-2025, alongside a generative foundation model for fleet coordination it called DeepFleet, which it said would improve robot travel efficiency by about ten percent. That last figure is worth pausing on. In a large facility, a ten percent improvement in travel efficiency is valuable precisely because travel and congestion, not individual robot capability, dominate throughput. The individual robots' localization and control are already good. What limits the system is how they share space. The structure of a goods-to-person floor is designed around this. The floor is typically laid out as a grid of storage locations with travel lanes between them, often with one-way traffic in designated lanes, highways around the perimeter and buffer zones in front of pick stations. The robots' navigation is correspondingly simplified: in grid-based systems a robot moves from cell to cell along the lanes, localizing at each cell from a floor marker or from LiDAR, and the hard problem moves from navigating the space to scheduling it. Multi-agent path finding The abstract version of this scheduling problem is multi-agent path finding: given a graph and a set of agents each with a start and goal, find paths for all agents such that no two occupy the same node at the same time or swap along the same edge, minimizing total travel time or the time until the last agent arrives. Finding optimal solutions is computationally hard in general, but the problem has been intensively studied and good algorithms exist for practical sizes. Prioritized planning is the simplest and still widely used approach. Agents are ordered by priority, each plans its path in turn with A* over space and time, treating the paths of all higher-priority agents as moving obstacles recorded in a reservation table: a record of which cell is occupied by whom at each time step. Prioritized planning is fast and usually finds good solutions, but it is incomplete; a low-priority agent may find no path because higher-priority agents have reserved every route. Conflict-based search, published by Guni Sharon, Roni Stern, Ariel Felner and Nathan Sturtevant in 2015, finds optimal solutions with a two-level search. At the low level, each agent plans independently. At the high level, the algorithm checks for conflicts between agents' paths, and when it finds one, it branches: in one branch, the first agent is forbidden from being at the conflicting place at the conflicting time; in the other, the second is. Each branch replans only the constrained agent. The search proceeds until a conflict-free set of paths is found. Conflict-based search and its bounded-suboptimal variants scale to hundreds of agents in many settings, and its ideas underpin many research and commercial fleet planners. Real warehouses add complications the abstract problem ignores. Goals arrive continuously, so the problem is lifelong: agents receive new tasks as they finish old ones, and the planner must continually replan a rolling horizon. Robots do not move in perfectly synchronized steps; they accelerate, brake, turn and are delayed by people, so a plan computed in discrete time must be executed robustly in continuous time. A common technique is to convert the plan into an action dependency graph that records only the ordering constraints between robots, which robot must pass a cell before which other robot may enter it, so that a robot delayed by a few seconds delays only those robots that depend on it, rather than invalidating the whole plan. Who goes where: task allocation and energy Path finding assumes each robot already knows its goal. Deciding which robot serves which task is a separate problem that strongly shapes congestion. The simplest rule, assigning each new task to the nearest idle robot, is greedy and myopic: it can send a robot across the floor for a task that a robot finishing nearby would have reached sooner, and it tends to concentrate robots wherever demand is momentarily highest. Batch assignment, collecting tasks over a short window and solving an assignment problem that minimizes total travel, does better, and the classical Hungarian algorithm or its faster approximations handle hundreds of robots and tasks in well under a second. In goods-to-person systems the allocation problem is entangled with storage. Which shelf a robot fetches depends on where the required items are stored, and where shelves are returned after picking determines where they will be next time. Systems that return popular shelves to locations near the pick stations, and less popular ones further away, reduce travel for the whole fleet; the floor slowly rearranges itself around demand. This is one reason why fleet-level improvements, of the kind Amazon claimed for DeepFleet, come from coordinating many decisions together rather than from improving any individual robot's navigation. Energy adds a further constraint. Battery-powered robots must charge, and a fleet in which every robot heads for a charger when its battery reaches the same threshold will produce queues at the chargers and a sudden loss of capacity. Fleet managers therefore schedule charging opportunistically, sending robots to charge during lulls and staggering thresholds so that the fleet's available capacity stays steady. The same reasoning applies to drone delivery operations, where battery swap or recharge time at the base sets the effective size of the fleet, and where reserve energy for diversions, holding and contingency landing must be planned into every flight before it departs. Deadlock, livelock and the art of traffic rules Even with good planning, fleets can lock up. A deadlock occurs when a cycle of robots each waits for the next to move. A livelock occurs when robots keep moving, each repeatedly yielding to the others, without anyone making progress. Both are well understood in principle and stubbornly common in practice, particularly in facilities where robots share space with people whose movements the planner cannot control, or where robots from different vendors, running different software, must share aisles. Practical fleet systems combine planning with traffic rules that make deadlock impossible by construction in at least part of the space: one-way aisles, right-of-way rules at intersections, mutex zones that only one robot may occupy at a time, such as a narrow doorway or an elevator, and dedicated parking spots where robots wait out of the way. Detection and recovery remain essential: the fleet manager monitors for robots that have made no progress for some time, identifies the cycle and directs one robot to back out. The rise of mixed fleets, robots from several vendors in one facility, has created demand for common interfaces between robots and fleet managers. The German automotive industry association's VDA 5050 specification defines a standard message protocol by which a master control system sends orders to automated vehicles and receives their state, and it has been adopted by a number of vendors outside the automotive sector. MassRobotics has published an interoperability standard aimed at letting robots from different vendors at least share their positions and intentions. Neither yet solves the full problem of coordinated planning across vendors, but both reduce the risk that two fleets in one building treat each other as unpredictable obstacles. Sharing the sky Drones have no building walls to constrain them and no single operator coordinating all traffic. The emerging solution, under various names, is a layer of digital traffic management for low-altitude airspace: in the United States, uncrewed aircraft system traffic management, or UTM, and in the European Union, U-space. The foundation is identification. The FAA's Remote ID rule requires most drones flying in American airspace to broadcast their identity, position and the location of their control station; operators were required to comply from September 2023, with enforcement discretion extended until March 2024. The European U-space regulation, Commission Implementing Regulation (EU) 2021/664, applicable from January 2023, establishes a framework in which designated airspace is served by certified U-space service providers offering network identification, geo-awareness, flight authorization and traffic information. On top of identification, UTM architectures provide strategic deconfliction: operators submit intended flight volumes, four-dimensional regions of space and time, before flight, and a network of service providers checks them against each other and against airspace restrictions, so that conflicts are resolved before take-off. During flight, conformance monitoring checks that each drone remains within its declared volume, and services can notify operators of conflicts or of crewed aircraft nearby. This is the airspace analogue of the warehouse reservation table, with the important difference that it is distributed across competing service providers and operators, not run by a single fleet manager. The proposed Part 108 rule would make beyond-visual-line-of-sight operations routine under a framework of operating permits for lower-risk operations and operating certificates, with organizational safety management requirements, for more complex ones. It reflects a shift from approving individual flights by waiver toward approving organizations whose processes, including the software practices described in this book, are trusted to produce safe operations. Whatever the final rule's details, the direction is clear: the regulator is increasingly interested in how an operator validates its autonomy, not merely in whether a given flight was safe. Validating what cannot be exhaustively tested That brings the book to its last engineering problem. A warehouse fleet that runs twenty hours a day, seven days a week, over hundreds of robots, accumulates millions of robot-hours a year. A failure that occurs once in a hundred thousand hours will occur many times. No test campaign can exercise every combination of floor condition, lighting, clutter, human behaviour and software state. How, then, does an organization gain confidence that its system is safe and reliable enough? The answer in mature organizations is a layered validation strategy, mirroring the layered architecture of the robot itself. Simulation is the broadest layer. Physics simulators with sensor models generate LiDAR scans and camera images of virtual warehouses and cities, and the full autonomy stack runs against them. Simulation is cheap, repeatable and can be scaled to thousands of runs overnight; it allows systematic variation of conditions and deliberate injection of faults, a blinded camera, a slipping wheel, a delayed timestamp. Its weakness is fidelity: simulated sensors rarely reproduce the exact failure modes of real ones, glass, dust and retroreflective tape among them, and a system tuned in simulation alone will be surprised by reality. Log replay is the most valuable layer for perception and estimation. Every deployed robot records its raw sensor data, with hardware timestamps, along with its estimates and decisions. When a robot behaves oddly, the log is pulled and replayed through the current and candidate software, and the failure is reproduced, diagnosed and fixed. The replayed logs then become regression tests: every future software release must handle them correctly. Over time an organization accumulates a library of its own hardest cases, drawn from reality rather than imagination. This is only possible if the robots log rich, correctly timestamped data, which is one more reason the timing discipline of Chapter 3 matters. Scenario testing on real robots in controlled facilities covers the safety-critical cases: people stepping out from behind racks at specified distances and speeds, dropped objects, blocked aisles, localization failures induced by moving racks. Standards such as ISO 3691-4 require verification of the safety functions by test, and the protective-field calculations of the previous chapter must be confirmed by measured stopping distances. Field monitoring closes the loop. Fleet operators track metrics that reveal the health of the autonomy stack: localization confidence over time and by location, the frequency of protective stops and of warning-field slowdowns, interventions by human operators, mean distance or time between failures, and the frequency of each recovery behaviour. A rising rate of localization uncertainty in one zone points to a map that needs updating. A rising rate of protective stops at one intersection points to a layout or traffic rule that needs changing. A cluster of recoveries after a software release points to a regression. A concept borrowed from automotive safety, the safety of the intended functionality, codified for road vehicles in ISO 21448, names the category of hazard this validation targets: not hardware faults or software bugs in the traditional sense, but situations where the system works exactly as designed and the design is inadequate for the situation, the perception that does not see glass, the predictor that assumes the person will keep walking. Such hazards can only be found by exposure to the real world, which is why field data, log replay and continuous monitoring are not optional extras but the core of a credible safety argument. Scale changes what matters The general lesson of this chapter is that scale changes the relative importance of the problems in the earlier chapters. For a single robot in a laboratory, accuracy is what matters: the best odometry, the best map. For a fleet of hundreds over years, what matters is consistency of behaviour, honest reporting of uncertainty and degradation, recoverability from failure and the ability to learn from every incident. A localizer that is a few centimetres less accurate but reliably reports when it is lost is worth more, at scale, than one that is more accurate but occasionally and silently wrong. A planner that produces slightly longer routes but never deadlocks is worth more than one that is optimal until it jams the floor. At scale, in other words, the argument of the book becomes operational. The systems that work are those in which every layer says what it knows and what it does not, so that the layers above can act with appropriate caution and the people running the operation can see, in the data, where the uncertainty is growing. Conclusion: Engineering Honest Machines The path of information through an autonomous robot runs from photons and laser echoes at the sensors, through calibration and timing, fusion, odometry, mapping and localization, to planning, avoidance and the certified safety layer that can override them all. At each stage the preceding chapters found the same pattern. The methods that work best are the ones that represent what they do not know. The failures that hurt most are the ones in which a module was confidently wrong and nothing downstream could tell. That observation, followed through the whole stack, leads to a set of design commitments that are more useful than any recommendation of a particular algorithm, because algorithms change and these do not. What follows for the people who build these systems Treat uncertainty as an interface, not an internal detail. Every module that produces an estimate should publish how much to trust it, in a form that consumers actually use. A localizer that outputs a pose without a covariance, or with a covariance that nobody reads, has thrown away the most important thing it knows. Planners should spend reported uncertainty as margin; trackers should size their gates by it; fleet managers should map it across the facility. The test of a well-designed stack is whether a degraded sensor produces a slower, more cautious robot rather than a confused one. Test consistency, not only accuracy. The accuracy figures in research papers measure how close an estimate is to the truth on average. The figure that matters operationally is whether the estimator's stated uncertainty matches its actual errors, and especially whether it knows when it has failed. An organization that logs its estimators' innovations and residuals, checks their statistics against ground truth where it can obtain it, and treats overconfidence as a defect will catch a large class of failures before they become incidents. Make calibration and time first-class engineering. The unglamorous foundations of Chapter 3 deserve the same rigour as the algorithms built on them: version-controlled calibration tied to each physical unit, hardware timestamps and clock synchronization, monitoring for drift and tests of timing under load. A large fraction of field problems are calibration and timing problems in disguise, and they disguise themselves well, as a robot that is a little worse at everything. Prefer conservative recognition to clever recognition. In perception and in SLAM, a false positive is usually more dangerous than a false negative. A missed loop closure leaves a little drift; a false one corrupts the map. A tracker that briefly loses a person and re-acquires them is safer than one that confidently merges two people into one. Where the environment is aliased, as warehouses are, add distinctiveness to the environment rather than hoping the algorithm will resolve the ambiguity. Separate the clever from the certified. The architecture of Chapter 8, an intelligent autonomy stack that avoids collisions efficiently above a simple, verified safety layer that prevents them absolutely, is not merely a regulatory convenience. It is what allows the upper layers to use learned models, complex optimization and fast-changing software at all, because the lower layer does not depend on them being right. The same separation applies in the air, where geofences, link-loss behaviours and parachutes stand beneath the planners. Anyone tempted to let a learned model take over a safety function should first ask how its failure modes would be found, bounded and certified. Design for degraded operation from the start. Every sensor will fail, and every map will go stale. A system whose behaviour with a blinded camera, a slipping wheel or a rearranged aisle has been defined in advance and tested will degrade gracefully. One whose degraded behaviour was left to emerge will degrade in ways nobody predicted. Learn from the field systematically. At fleet scale, the hardest cases come from reality, not imagination. Rich, correctly timestamped logs, a replay infrastructure and a growing library of regression cases drawn from real incidents are the means by which a system improves after deployment rather than merely accumulating workarounds. What follows for the people who buy and regulate them Those commitments also suggest questions that a buyer, an operations manager or a regulator can ask of any autonomous system, without needing to understand its algorithms. How does the robot report that it is uncertain or lost, and what does it do then? How is sensor calibration verified in service, and how often? What happens when each sensor fails, and has that been tested? Which functions are safety-rated, to what standard, and what hazards in this particular facility fall outside them? How are incidents logged, replayed and turned into regression tests? An organization that answers these crisply has probably built something trustworthy. One that answers with accuracy figures alone probably has not thought about them. The regulatory trend points the same way. Standards for driverless industrial trucks require hazard analysis of the robot in its specific environment, not merely of the robot. The emerging frameworks for beyond-line-of-sight drone flight shift attention from individual flights to the organizations and processes that conduct them. Both are, in effect, asking whether an operator knows what its machines do not know. What remains unsettled Some problems remain genuinely open. Learned perception is now indispensable for recognizing people and objects, yet its uncertainty estimates are often poorly calibrated, and its failures in conditions outside its training data are hard to predict. Bringing learned components into the honest-uncertainty discipline of classical estimation, so that a network can say reliably when it does not know, is one of the most important open problems in the field. Lifelong mapping in changing environments is improving but not solved. Telling a permanent change from a transient one, and updating a shared map without letting one robot's mistake corrupt the fleet's model, still requires human judgment in most deployments. Prediction of people remains limited, especially in the close, cooperative interactions that make shared spaces work. Robots that are legible and predictable to people may matter as much as robots that predict people well, and the design of that legibility, through motion, lights and conventions, is still more craft than science. Coordination across organizations, between fleets from different vendors in one building or drones from different operators in one airspace, is in its early stages. The interfaces are being standardized, but the deeper problem of planning jointly across systems whose owners do not share software or incentives is unresolved. And validation of systems that cannot be exhaustively tested remains as much an organizational discipline as a technical method. Simulation, replay, scenario testing and field monitoring together give a credible argument; none gives a proof. The enduring shape The specific methods described in this book will change. Bird's-eye-view fusion will be superseded; new odometry systems will top the benchmarks; planners will increasingly be learned. What will endure is the shape of the problem. A machine that moves through the physical world among people must estimate what it cannot directly observe, act on estimates it knows to be imperfect, and fail safely when its estimates are wrong. The engineering that makes this possible is, at every level, the engineering of honest uncertainty. Robots that admit what they do not know, and systems built so that every layer listens to that admission, are the ones that can be trusted to share a warehouse floor or a patch of sky. Notes Introduction 1. Amazon, announcement of its one-millionth robot deployment and the DeepFleet foundation model, 1 July 2025; reported in Automated Warehouse, "Amazon deploys 1 millionth robot, launches new AI foundation model" (2025). 2. Federal Aviation Administration, "Normalizing Unmanned Aircraft Systems Beyond Visual Line of Sight Operations" (proposed Part 108), notice of proposed rulemaking, Federal Register, 7 August 2025. Chapter 1 1. R. E. Kalman, "A New Approach to Linear Filtering and Prediction Problems", Journal of Basic Engineering 82 (1960): 35–45. 2. F. Dellaert, D. Fox, W. Burgard and S. Thrun, "Monte Carlo Localization for Mobile Robots", Proceedings of the IEEE International Conference on Robotics and Automation (1999). 3. S. Thrun, W. Burgard and D. Fox, Probabilistic Robotics (MIT Press, 2005). 4. ROS Enhancement Proposal 105, "Coordinate Frames for Mobile Platforms". Chapter 3 1. Z. Zhang, "A Flexible New Technique for Camera Calibration", IEEE Transactions on Pattern Analysis and Machine Intelligence 22, no. 11 (2000): 1330–1334. 2. Q. Zhang and R. Pless, "Extrinsic Calibration of a Camera and Laser Range Finder (Improves Camera Calibration)", Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2004). 3. P. Furgale, J. Rehder and R. Siegwart, "Unified Temporal and Spatial Calibration for Multi-Sensor Systems", Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2013). 4. IEEE 1588, Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems. Chapter 4 1. S. Vora, A. H. Lang, B. Helou and O. Beijbom, "PointPainting: Sequential Fusion for 3D Object Detection", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020). 2. C. R. Qi, W. Liu, C. Wu, H. Su and L. J. Guibas, "Frustum PointNets for 3D Object Detection from RGB-D Data", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018). 3. J. Philion and S. Fidler, "Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D", European Conference on Computer Vision (2020). 4. Z. Liu et al., "BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation", Proceedings of the IEEE International Conference on Robotics and Automation (2023). Chapter 5 1. P. J. Besl and N. D. McKay, "A Method for Registration of 3-D Shapes", IEEE Transactions on Pattern Analysis and Machine Intelligence 14, no. 2 (1992): 239–256. 2. Y. Chen and G. Medioni, "Object Modelling by Registration of Multiple Range Images", Image and Vision Computing 10, no. 3 (1992): 145–155. 3. J. Zhang and S. Singh, "LOAM: Lidar Odometry and Mapping in Real-time", Robotics: Science and Systems (2014). 4. I. Vizzo et al., "KISS-ICP: In Defense of Point-to-Point ICP — Simple, Accurate, and Robust Registration If Done the Right Way", IEEE Robotics and Automation Letters 8, no. 2 (2023). 5. A. I. Mourikis and S. I. Roumeliotis, "A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation", Proceedings of the IEEE International Conference on Robotics and Automation (2007). 6. T. Qin, P. Li and S. Shen, "VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator", IEEE Transactions on Robotics 34, no. 4 (2018). 7. C. Forster, L. Carlone, F. Dellaert and D. Scaramuzza, "On-Manifold Preintegration for Real-Time Visual-Inertial Odometry", IEEE Transactions on Robotics 33, no. 1 (2017). 8. T. Shan et al., "LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping", Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2020). 9. W. Xu, Y. Cai, D. He, J. Lin and F. Zhang, "FAST-LIO2: Fast Direct LiDAR-Inertial Odometry", IEEE Transactions on Robotics 38, no. 4 (2022). Chapter 6 1. H. Durrant-Whyte and T. Bailey, "Simultaneous Localization and Mapping: Part I", IEEE Robotics and Automation Magazine 13, no. 2 (2006); T. Bailey and H. Durrant-Whyte, "Part II", 13, no. 3 (2006). 2. C. Cadena et al., "Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age", IEEE Transactions on Robotics 32, no. 6 (2016). 3. G. Grisetti, C. Stachniss and W. Burgard, "Improved Techniques for Grid Mapping with Rao-Blackwellized Particle Filters", IEEE Transactions on Robotics 23, no. 1 (2007). 4. F. Dellaert and M. Kaess, "Factor Graphs for Robot Perception", Foundations and Trends in Robotics 6, no. 1–2 (2017). 5. M. Kaess et al., "iSAM2: Incremental Smoothing and Mapping Using the Bayes Tree", International Journal of Robotics Research 31, no. 2 (2012). 6. W. Hess, D. Kohler, H. Rapp and D. Andor, "Real-Time Loop Closure in 2D LIDAR SLAM", Proceedings of the IEEE International Conference on Robotics and Automation (2016). 7. D. Fox, "Adapting the Sample Size in Particle Filters through KLD-Sampling", International Journal of Robotics Research 22, no. 12 (2003). 8. C. Campos, R. Elvira, J. J. Gómez Rodríguez, J. M. M. Montiel and J. D. Tardós, "ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial, and Multimap SLAM", IEEE Transactions on Robotics 37, no. 6 (2021). Chapter 7 1. S. Macenski, F. Martín, R. White and J. Ginés Clavero, "The Marathon 2: A Navigation System", Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2020). 2. P. E. Hart, N. J. Nilsson and B. Raphael, "A Formal Basis for the Heuristic Determination of Minimum Cost Paths", IEEE Transactions on Systems Science and Cybernetics 4, no. 2 (1968). 3. D. Dolgov, S. Thrun, M. Montemerlo and J. Diebel, "Path Planning for Autonomous Vehicles in Unknown Semi-structured Environments", International Journal of Robotics Research 29, no. 5 (2010). 4. L. E. Kavraki, P. Švestka, J.-C. Latombe and M. H. Overmars, "Probabilistic Roadmaps for Path Planning in High-Dimensional Configuration Spaces", IEEE Transactions on Robotics and Automation 12, no. 4 (1996). 5. S. M. LaValle, "Rapidly-Exploring Random Trees: A New Tool for Path Planning", Technical Report, Iowa State University (1998). 6. S. Karaman and E. Frazzoli, "Sampling-based Algorithms for Optimal Motion Planning", International Journal of Robotics Research 30, no. 7 (2011). 7. D. Mellinger and V. Kumar, "Minimum Snap Trajectory Generation and Control for Quadrotors", Proceedings of the IEEE International Conference on Robotics and Automation (2011). 8. G. Williams, P. Drews, B. Goldfain, J. M. Rehg and E. A. Theodorou, "Aggressive Driving with Model Predictive Path Integral Control", Proceedings of the IEEE International Conference on Robotics and Automation (2016). Chapter 8 1. D. Fox, W. Burgard and S. Thrun, "The Dynamic Window Approach to Collision Avoidance", IEEE Robotics and Automation Magazine 4, no. 1 (1997). 2. P. Fiorini and Z. Shiller, "Motion Planning in Dynamic Environments Using Velocity Obstacles", International Journal of Robotics Research 17, no. 7 (1998). 3. J. van den Berg, S. J. Guy, M. Lin and D. Manocha, "Reciprocal n-Body Collision Avoidance", in Robotics Research: The 14th International Symposium ISRR (Springer, 2011). 4. D. Helbing and P. Molnár, "Social Force Model for Pedestrian Dynamics", Physical Review E 51 (1995). 5. ISO 3691-4:2023, Industrial trucks — Safety requirements and verification — Part 4: Driverless industrial trucks and their systems. 6. ANSI/A3 R15.08-1 (2020) and R15.08-2 (2023), Industrial Mobile Robots — Safety Requirements. Chapter 9 1. P. R. Wurman, R. D'Andrea and M. Mountz, "Coordinating Hundreds of Cooperative, Autonomous Vehicles in Warehouses", AI Magazine 29, no. 1 (2008). 2. G. Sharon, R. Stern, A. Felner and N. R. Sturtevant, "Conflict-Based Search for Optimal Multi-Agent Pathfinding", Artificial Intelligence 219 (2015). 3. Commission Implementing Regulation (EU) 2021/664 of 22 April 2021 on a regulatory framework for the U-space. 4. Federal Aviation Administration, Remote Identification of Unmanned Aircraft, 14 CFR Part 89. 5. ISO 21448:2022, Road vehicles — Safety of the intended functionality. Further Reading Sebastian Thrun, Wolfram Burgard and Dieter Fox, Probabilistic Robotics (MIT Press, 2005).** The standard text on the probabilistic view of robotics, covering Bayes filters, Kalman and particle filters, localization and SLAM. Its age shows in places, but no book better explains why a robot should carry a belief rather than a guess. Timothy D. Barfoot, State Estimation for Robotics, 2nd edition (Cambridge University Press, 2024). A rigorous, modern treatment of estimation, including the handling of rotations and poses on matrix Lie groups, batch and recursive estimation, and continuous-time trajectories. Demanding but rewarding for anyone who will write estimators. Frank Dellaert and Michael Kaess, Factor Graphs for Robot Perception (Now Publishers, 2017).** A compact monograph explaining the factor-graph formulation that underlies modern SLAM and odometry, from the authors of the iSAM family and the GTSAM library. Steven M. LaValle, Planning Algorithms (Cambridge University Press, 2006).** The comprehensive reference on motion planning, from discrete search to sampling-based and kinodynamic planning, also freely available online from the author. Roland Siegwart, Illah R. Nourbakhsh and Davide Scaramuzza, Introduction to Autonomous Mobile Robots, 2nd edition (MIT Press, 2011). A readable introduction to the whole mobile-robot stack, including locomotion, sensors, perception, localization and planning, suited to readers coming from outside robotics. Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, 2nd edition (Cambridge University Press, 2004). The reference for the geometry of cameras, calibration and reconstruction behind visual odometry and visual SLAM. Cesar Cadena et al., "Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age", IEEE Transactions on Robotics (2016).** A widely cited survey that sets out what SLAM had solved and what remained open, with a useful emphasis on robustness and long-term operation. Steve Macenski et al., "The Marathon 2: A Navigation System" (IROS, 2020), and the Nav2 documentation. The description of the ROS 2 navigation stack's architecture; the project's online documentation is an unusually practical guide to costmaps, planners, controllers and their tuning for real robots. Hashtags: #Robotics #ArtificialIntelligence #RoboticsAndAI #IntelligentRobotics #AutonomousRobots #AutonomousSystems #MobileRobotics #SensorFusion #MultisensorFusion #RobotPerception #RoboticNavigation #PathPlanning #MotionPlanning #CollisionAvoidance #ObstacleDetection #ObstacleAvoidance #SLAM #SimultaneousLocalizationAndMapping #RobotLocalization #ProbabilisticRobotics #BayesianFiltering #KalmanFilter #ExtendedKalmanFilter #ParticleFilter #StateEstimation #LiDAR #LiDARCameraFusion #ComputerVision #VisualOdometry #LiDAROdometry #InertialNavigation #IMU #SensorCalibration #TimeSynchronization #PointCloudProcessing #ScanMatching #IterativeClosestPoint #FactorGraphOptimization #LoopClosure #OccupancyGridMapping #AStarAlgorithm #SamplingBasedPlanning #TrajectoryOptimization #DynamicWindowApproach #VelocityObstacles #ROS2 #AutonomousDrones #WarehouseRobotics #FleetCoordination #RobotSafety
- High-Performance Computing (HPC): Supercomputing Architecture and MPI
Download the Book (PDF): Introduction Every few hours, somewhere in the world, a weather service starts a forecast. Observations from satellites, radiosondes, aircraft, ships and ground stations have been gathered and blended into a best estimate of the current state of the atmosphere, and that estimate is handed to a model that will step it forward in time, minute by simulated minute, until it has produced a picture of the next ten days. The forecast has to be finished in about an hour, because a forecast that arrives late is a forecast about the past. The model divides the globe into tens of millions of columns of air, each stacked with a hundred or so vertical levels, and for every cell it must update wind, temperature, pressure, moisture and a dozen other quantities thousands of times. No single computer could do this in the time allowed. The work is spread across thousands of processors, each responsible for a patch of the planet, each needing to know, at every step, what is happening at the edges of its neighbours' patches. That last clause is the whole subject of this book in miniature. High-performance computing is often described as the business of doing arithmetic very fast, and the headline numbers encourage the description. The fastest machines in the world are now ranked in exaflops, billions of billions of floating-point operations per second; on the TOP500 list published in June 2026, five systems had crossed that threshold on the standard benchmark. But anyone who has tried to make a real scientific code run well on such a machine learns quickly that arithmetic is the cheap part. The expensive part is getting the right numbers to the right place at the right time: from main memory into a processor's caches, from one processor's memory into another's, from one node of the machine across the network to a node in a different cabinet, and from the whole machine out to storage and back. The controlling idea of this book is that performance on a modern supercomputer is governed by data movement, not by computation, and that the main tools of the field (the Message Passing Interface, GPU clustering, and memory-hierarchy optimisation) are best understood as three ways of managing where data lives and how it travels. An engineer who internalises that idea can reason about a parallel program they have never seen. One who does not will spend months tuning the wrong thing. Why data movement wins The reason is physical, and it has been getting steadily more pronounced for thirty years. Transistors kept shrinking long after the clock frequency of processors stopped rising in the mid-2000s, and the industry spent those transistors on parallelism: more cores per chip, wider vector units in each core, and eventually graphics processors with thousands of simple arithmetic lanes. The capacity to compute grew enormously. The capacity to feed those lanes with data grew far more slowly, because moving a bit across a wire costs energy in proportion to the length of the wire, and memory chips sit a long way, in electrical terms, from the processor. Fetching an operand from main memory costs far more energy than performing a floating-point operation on it, and the gap widens again for an operand on another node. The consequence is that most scientific codes, left to themselves, spend most of their time waiting. A processor capable of trillions of operations per second sits idle while a cache line arrives from memory. A thousand processors wait at a synchronisation point for the slowest of them to finish exchanging boundary data. A GPU with enormous arithmetic throughput stalls because its input is still being copied across a bus. Making these machines perform is mostly a matter of eliminating that waiting: reorganising data so that each byte fetched is used many times, arranging communication so that it overlaps with computation, and dividing the problem so that the amount of data each processor must exchange is small compared with the work it does on its own. Two applications that keep the argument honest Abstract principles about parallelism are easy to state and hard to apply, so this book anchors them in two application domains that have shaped the field: computational fluid dynamics and numerical weather prediction. They are closely related, since a weather model is a fluid-dynamics code for a thin, rotating, moist shell of gas, but they stress machines in different ways. Computational fluid dynamics, used to design aircraft wings, turbine blades, cars, ships, heart valves and ventilation systems, typically works on meshes that conform to complicated geometry. The cells may be tetrahedra, hexahedra or polyhedra of varying size, refined near surfaces where the flow changes rapidly. Dividing such a mesh fairly among thousands of processors is a genuine problem in graph theory, and the solvers involve both local stencil operations and global linear-algebra steps that require every processor to agree on a single number. Weather and climate models work on the whole sphere, with relatively regular grids but enormous numbers of variables per cell, physics packages for clouds and radiation that behave very differently from the fluid dynamics, and a hard deadline. They have pushed the field towards some of its most interesting recent developments: the move of whole atmospheric models onto GPUs, the use of reduced numerical precision to save memory bandwidth, and the arrival of machine-learned forecasting systems that change what the supercomputer is asked to do. Between them, these applications exercise every layer of the hierarchy that this book describes. They also have the virtue of being consequential. A faster CFD code lets an engineer explore more designs; a faster weather code lets a forecaster run a larger ensemble, resolve storms more finely, or issue a warning sooner. What the book covers and what it leaves out The chapters build from the machine upwards. The first describes what a supercomputer physically is today: nodes, accelerators, memory, interconnect and storage, and the distances between them measured in time and energy. The second introduces the single most useful mental model for single-node performance, the roofline, and the concept of arithmetic intensity, using the stencil computations at the heart of fluid codes as the running example. Chapters three to five are about MPI, the Message Passing Interface, which has been the dominant way to program distributed-memory machines since the 1990s and remains so. Chapter three explains the model and its semantics: processes, communicators, point-to-point messages, and the subtle ways programs deadlock. Chapter four covers collective operations and the theory of scaling, including the laws named after Amdahl and Gustafson and what they do and do not predict. Chapter five puts these together in domain decomposition and halo exchange, the pattern that underlies almost every grid-based simulation. Chapter six turns inward to the node, where the memory hierarchy of caches, NUMA domains and vector registers determines whether each process does its share of work efficiently, and where hybrid programming with MPI plus threads comes in. Chapter seven covers GPU clusters: how accelerators change the balance of the machine, how data moves between GPUs within and across nodes, and how codes are written to be portable among vendors. Chapter eight is a sustained case study of weather and climate modelling, drawing on the preceding chapters to show how a real community has navigated these trade-offs. Chapter nine covers the parts of the job that are easy to neglect and expensive to get wrong: measurement, parallel input and output, and resilience. The conclusion asks what follows for people who write, buy or depend on these codes. Some things are deliberately left out. The book does not teach a programming language, though short code fragments in C appear where the exact form of an MPI call matters. It does not survey every parallel programming model; partitioned global address space languages, task-based runtimes and the large ecosystem of machine-learning frameworks are mentioned where they bear on the argument and otherwise set aside. It does not attempt to rank vendors or predict which architecture will win. Hardware generations change every few years; the principles of locality, communication cost and balance have been stable for decades, and they are what the reader should take away. Who this is for The intended reader is technically literate but not necessarily a specialist in parallel computing: a scientist or engineer whose simulations have outgrown a workstation, a graduate student about to inherit a large legacy code, a software developer moving into scientific computing, a manager who has to decide whether to port an application to GPUs, or a curious reader who wants to know what actually happens inside the machines that produce the weather forecast. Some familiarity with programming helps; calculus is not required, and where an equation appears, its meaning is explained in words. A reader who finishes the book should be able to look at a parallel simulation and ask the right first questions. How many bytes does it move for every operation it performs? Where do those bytes come from? How much of each processor's time is spent talking to other processors, and does that fraction grow as the machine gets bigger? Is the code waiting on memory, on the network, or on the slowest member of the team? Those questions, asked early, separate the codes that scale from the ones that merely run. Chapter 1: The Machine as a Hierarchy of Distances A supercomputer looks, from the floor of the machine room, like rows of tall black cabinets connected by thick bundles of cable, with a low roar of pumps and fans and the occasional glimpse of coolant pipes. It is tempting to think of it as one enormous computer. It is better to think of it as a city of ordinary-looking computers, each quite powerful on its own, wired together by a very fast and very expensive network, and sharing a filesystem that behaves like a vast warehouse at the edge of town. Almost every decision in parallel programming follows from the distances in that city: how far each piece of data is from the arithmetic unit that needs it, and how long it takes to bring it closer. Nodes, the unit of the city The basic building block is the node. A node is a computer with its own processors, its own memory and its own copy of the operating system. In a traditional CPU-only system, a node typically holds one or two multi-core processors, each with dozens of cores, and a few hundred gigabytes of DRAM. In the accelerated systems that now dominate the top of the rankings, a node also holds several GPUs, each with its own stack of high-bandwidth memory. The nodes of the large American exascale systems illustrate the variety. A node of Frontier, at Oak Ridge National Laboratory, pairs one AMD EPYC CPU with four AMD Instinct MI250X accelerators; each MI250X package contains two graphics compute dies, so software sees eight GPUs per node, each with its own high-bandwidth memory. El Capitan, at Lawrence Livermore National Laboratory, uses the AMD Instinct MI300A, an accelerated processing unit that places CPU cores and GPU compute units in the same package and lets them share a single pool of high-bandwidth memory, removing the separate CPU memory that earlier designs had to copy to and from. Aurora, at Argonne, combines Intel Xeon CPU Max processors, which themselves carry high-bandwidth memory on the package, with six Intel Data Center GPU Max accelerators per node. JUPITER, the first European exascale system at Forschungszentrum Jülich, builds its booster module from NVIDIA Grace Hopper superchips, which couple an Arm-based CPU to a Hopper GPU through a fast coherent link. The node is important because it marks the boundary of shared memory. Within a node, any core can in principle load any byte of the node's memory with an ordinary instruction, even if some bytes are closer than others. Between nodes, there is no shared memory at all. If a process on node 17 needs a value computed on node 4,312, someone has to package that value into a message and send it across the network. This boundary is the reason the Message Passing Interface exists, and it is why the first question about any parallel program is how its data is divided among nodes. The memory hierarchy inside a node Inside a node, memory is not one thing but a sequence of layers, each larger and slower than the one before. Closest to the arithmetic are the registers, a few hundred bytes to a few kilobytes per core, which can be read in the same cycle as an operation is issued. Next come the caches. A modern CPU core has a small first-level cache, typically tens of kilobytes, a larger second-level cache of a megabyte or two, and shares a third-level cache of tens or hundreds of megabytes with the other cores on the chip or on its chiplet. Beyond the caches lies main memory: DRAM on most CPUs, or high-bandwidth memory stacked beside the processor on GPUs and some CPUs. Each step outwards costs time. A load that hits in first-level cache may complete in a handful of cycles; a load that goes all the way to DRAM may take a few hundred. Bandwidth falls too: a core can draw data from its own caches many times faster than the whole chip can draw it from main memory, and the main-memory bandwidth is shared among all the cores. A useful rule of thumb, rather than a precise figure, is that each level of the hierarchy is roughly an order of magnitude slower in latency than the level above it, and that the energy cost of moving a word grows similarly. Processors hide much of this latency with tricks. Hardware prefetchers notice regular access patterns and request cache lines before they are needed. Out-of-order execution lets a core keep working on independent instructions while a load is outstanding. Caches deliver data not one word at a time but in lines of 64 or 128 bytes, so a program that walks through memory sequentially gets the neighbours of each value for free. All of these mechanisms reward the same behaviour: touching memory in predictable, contiguous patterns and reusing data while it is still close by. A program that leaps around memory at random defeats them all. On multi-socket CPU nodes, and on chiplet-based processors, there is a further layer: non-uniform memory access, or NUMA. Each socket, or each group of chiplets, has its own memory controllers attached to its own memory. A core can read memory attached to another socket, but it does so over an inter-socket link, with higher latency and lower bandwidth. To the programmer, all memory on the node looks the same; to the hardware, it is several neighbourhoods with toll roads between them. Chapter six returns to how codes respect these neighbourhoods. GPUs have a hierarchy of their own. A GPU contains many streaming multiprocessors (NVIDIA's term) or compute units (AMD's), each with a large register file, a small fast scratchpad memory that the programmer can manage explicitly, and a first-level cache. These share a second-level cache, and beyond that lies the high-bandwidth memory, which on current accelerators delivers several terabytes per second, far more than a CPU socket's DRAM. Between a discrete GPU and its host CPU there is traditionally a PCI Express link, much slower than either side's local memory, and the cost of crossing it has shaped GPU programming for fifteen years. The newer designs, such as MI300A and Grace Hopper, exist largely to shorten or remove that crossing. The interconnect, the city's roads Nodes talk to each other over the interconnect, a dedicated high-performance network quite unlike the Ethernet in an office. Its job is to deliver small messages with very low latency, on the order of a microsecond or two between nodes, and large messages at high bandwidth, while thousands of nodes are all communicating at once. Current systems use technologies such as HPE Slingshot, used on Frontier, El Capitan and Aurora, and NVIDIA's InfiniBand, used on many others including JUPITER. The shape of the network matters as much as its speed. The nodes cannot all be wired directly to one another, so they are connected through switches arranged in a topology. Older systems often used multidimensional tori, in which each node connects to its nearest neighbours in a three- or more-dimensional grid; IBM's Blue Gene machines and Japan's K computer and Fugaku used variants of this idea. Many current systems use a fat tree, in which switches are arranged in layers with more links near the top so that bandwidth does not shrink as traffic moves towards the root, or a dragonfly, in which groups of switches are densely connected internally and the groups are connected to one another by long global links, which keeps the number of switch hops between any two nodes small. Topology determines which communication patterns are cheap. On a torus, nearest-neighbour exchanges are almost free, but traffic between distant nodes must cross many links and competes with other traffic on the way. On a dragonfly, any two nodes are only a few hops apart, but the global links between groups are a shared resource, and when many jobs use them at once, one job's traffic can slow another's. That interference between jobs is one of the reasons that the same code, run on the same number of nodes, can take noticeably different times on different days. Modern interconnects also offload work from the processors. Network interface cards can perform remote direct memory access, writing data straight into another node's memory without involving its CPU, and some can perform parts of collective operations, such as summing values from many nodes, in the switches themselves. These features matter to MPI libraries, which exploit them, and to programmers, who benefit when they write communication in a form that the library can hand to the hardware. Storage, the warehouse Beyond the compute nodes lies the parallel filesystem: many storage servers, each managing many disks or flash drives, presented to the nodes as a single directory tree. Systems such as Lustre, IBM Storage Scale (formerly GPFS) and DAOS spread each large file across many servers so that thousands of nodes can read and write at once. Their aggregate bandwidth is impressive, but per node it is far below the bandwidth of memory, and the latency of metadata operations, such as creating a file or listing a directory, is high. A program that has each of ten thousand processes create its own small file at every time step can bring a filesystem to its knees while moving almost no data. Chapter nine considers how to avoid being that program. Increasingly, systems also include a layer between memory and the filesystem: node-local flash storage, or burst buffers, which absorb bursts of output quickly and drain them to the main filesystem in the background. They add another rung to the hierarchy and another decision about where data should live. The current landscape in numbers The TOP500 list, published twice a year since 1993, ranks systems by their performance on the High-Performance Linpack benchmark, which solves a large dense system of linear equations. Linpack is a demanding test of floating-point throughput, and it is valuable precisely because it has been run consistently for three decades. It is also a poor proxy for most scientific workloads, for reasons this book will make clear: dense linear algebra reuses each byte of data many times, whereas most simulations do not. A companion benchmark, HPCG, which solves a sparse system with a conjugate gradient method, behaves much more like real codes and typically achieves only a small percentage of a machine's peak. The top of the June 2026 list shows how the landscape has moved, as Table 1 sets out. Five systems exceeded one exaflop per second on Linpack. The new leader, a Chinese system called LineShine at the National Supercomputing Centre in Shenzhen, is a CPU-only design built from many-core processors with 304 cores each and a proprietary interconnect. It is a reminder that accelerated designs are not the only route to the top. The others are accelerated machines from the United States and Europe. Table 1. The five exascale systems on the June 2026 TOP500 list (source: TOP500, June 2026). Rank System and site Node architecture Rmax (Pflop/s) 1 LineShine, NSC Shenzhen, China LX2 304-core CPUs, no accelerators 2,198.4 2 El Capitan, LLNL, USA AMD EPYC with Instinct MI300A 1,809.0 3 Frontier, ORNL, USA AMD EPYC with Instinct MI250X 1,353.0 4 Aurora, ANL, USA Intel Xeon CPU Max with Data Center GPU Max 1,012.0 5 JUPITER Booster, FZJ, Germany NVIDIA Grace Hopper superchips 1,000.0 The power these machines draw is as instructive as their speed. LineShine is listed at about 42 megawatts. El Capitan, on the same list, achieved roughly 61 gigaflops per watt on Linpack, among the best of the large systems. Power is now a first-order design constraint: a facility that can supply and cool twenty or thirty megawatts is a significant piece of infrastructure, and the electricity bill over a machine's life can rival its purchase price. Since moving data costs more energy than computing on it, the energy budget and the data-movement budget are, to a large degree, the same budget. How a job meets the machine None of this hardware is visible to a user in the way a laptop's is. A scientist does not log into El Capitan's compute nodes; they log into a login node, compile their code there, and submit a job script to a batch scheduler, most commonly Slurm on current systems. The script asks for a number of nodes for a period of time, and the scheduler queues the job until those nodes are free, then launches the program on them. The user's allocation is measured in node-hours or GPU-hours, and on the largest machines it is awarded through competitive proposals, such as the US Department of Energy's INCITE programme or the European PRACE and EuroHPC calls, in which applicants must demonstrate that their code can use a large share of the machine efficiently. Scaling studies are, among other things, a currency in this market. The scheduler's choice of nodes matters more than most users realise. Two jobs requesting the same number of nodes may receive them packed into a few neighbouring cabinets or scattered across the whole machine, and the scattered allocation pays more for every message that crosses a global link. Some schedulers try to allocate compact sets of nodes for large jobs, and some systems let users request topology-aware placement, at the cost of waiting longer in the queue. On a busy machine, the job's neighbours in the network are other people's jobs, whose traffic shares the same links. It is worth following a single value through the city to make the distances concrete. Imagine a pressure value in one cell of a fluid simulation, computed by a GPU on one node at the edge of its patch of the domain. To be useful to the neighbouring patch, it must leave the GPU's registers, where it was computed, and be written to the GPU's high-bandwidth memory. A packing kernel then reads it, along with thousands of its neighbours along the same boundary face, and writes them contiguously into a send buffer. The network interface reads that buffer, either directly from GPU memory or after a copy to host memory, and injects it into the network as packets. The packets pass through a switch, perhaps through a global link to another group of switches, and arrive at another node's network interface, which writes them into a receive buffer. An unpacking kernel on the destination GPU copies the values into the halo region of its own field array, and only then can the next time step's stencil read them. At every stage the value waits behind others and costs energy to move, and the total time for the journey is many thousands of times longer than the time it took to compute the value in the first place. A well-designed simulation arranges things so that very few values need to make that journey, that those which do travel together in large groups, and that while they are in transit the processors have other useful work to do. That sentence, repeated in different forms, is most of what the rest of this book has to say. Balance, the number that matters The single most useful number for characterising a node is not its peak speed but its balance: the ratio between the rate at which it can perform floating-point operations and the rate at which it can bring data in from memory. A machine that can do a hundred operations in the time it takes to fetch one eight-byte number from main memory needs every number fetched to take part in a hundred operations before it can run at full speed. Most scientific kernels do nothing like that much work per byte. Over the history of the TOP500, this ratio has drifted steadily in one direction. Peak arithmetic has grown faster than memory bandwidth, which in turn has grown faster than network bandwidth, which has grown faster than storage bandwidth. Each layer of the city has been getting relatively farther from the arithmetic at its centre. Accelerators, with their high-bandwidth memory, improved the memory side substantially, but they improved the arithmetic side even more. This drift explains why the same code, unchanged, often gets a smaller fraction of peak on each new generation of machine, and why the skills described in the rest of this book have become more important over time rather than less. It also suggests the right frame for everything that follows. Parallel programming is often taught as a matter of dividing work. It is more accurately a matter of dividing data, so that the work each processor does is close to the data it needs, and the data it must fetch from far away is as little, and as early, as possible. Chapter 2: Counting Bytes: The Roofline and Arithmetic Intensity Before parallelism enters the picture at all, a single processor running a single kernel has a performance ceiling set by two quantities: how fast it can compute and how fast it can be fed. The first is advertised on every datasheet. The second is quieter, and more often the one that binds. Learning to estimate which ceiling a piece of code is pressing against, from a few minutes of arithmetic with a pencil, is the most valuable single skill in performance engineering. It tells you what to optimise, what not to bother with, and what speed-up is even possible. Arithmetic intensity The central quantity is arithmetic intensity: the number of floating-point operations a kernel performs for each byte it moves between the processor and main memory. It is a property of the algorithm and its implementation together, not of the machine, and it can usually be estimated by inspecting a loop. Take the simplest possible example, a loop that adds two arrays and stores the result in a third: for each index, the program reads one element of each input and writes one element of the output, and performs one addition. In double precision each element is eight bytes, so the loop moves at least twenty-four bytes per addition, and on most CPUs a little more, because writing to a cache line that is not already in cache typically forces the processor to read it first, a behaviour called write-allocate. The arithmetic intensity is therefore about one twenty-fourth of a flop per byte, or lower. This is the pattern measured by the STREAM benchmark, devised by John McCalpin in the 1990s and still the standard way to measure sustainable memory bandwidth. At the other extreme is dense matrix multiplication. Multiplying two n-by-n matrices takes about 2n³ operations but touches only about 3n² numbers. With careful blocking, each number brought into cache can take part in many operations, and the intensity grows with the block size. This is why Linpack, which spends nearly all its time in dense matrix multiplication, can run close to a machine's peak, and why it says little about codes that cannot reuse data in that way. Most of scientific computing lives between these extremes, and much of it lives uncomfortably close to the bottom. The roofline model In 2009 Samuel Williams, Andrew Waterman and David Patterson published a paper in Communications of the ACM proposing a simple visual model for this relationship, which they called the roofline. The idea can be stated entirely in words. A kernel's attainable performance is the smaller of two numbers: the machine's peak floating-point rate, and the product of the machine's memory bandwidth and the kernel's arithmetic intensity. If the intensity is low, the second number is smaller, and the kernel is memory-bound: doubling the speed of the arithmetic units would make no difference, while doubling the memory bandwidth would nearly double performance. If the intensity is high, the first number is smaller, and the kernel is compute-bound: only faster arithmetic helps. The point at which the two limits are equal, where the intensity equals the machine's peak rate divided by its bandwidth, is called the ridge point. It is the machine balance from the previous chapter expressed in flops per byte. To make this concrete with round, illustrative numbers rather than any particular product's specification, imagine a GPU that can perform 50 trillion double-precision operations per second and stream data from its high-bandwidth memory at 3 terabytes per second. Its ridge point is about 17 flops per byte. A kernel with an intensity of 0.5 flops per byte can achieve at most 1.5 trillion operations per second on this device, three per cent of peak, no matter how cleverly its arithmetic is arranged. A CPU node with, say, 5 trillion operations per second of peak and 400 gigabytes per second of DRAM bandwidth has a ridge point of about 12.5, which is lower but still far above what most simulation kernels achieve. For both, the conclusion is the same: the kernel is memory-bound, and the job of optimisation is to move fewer bytes. The model is extended in practice with additional ceilings. There is a separate bandwidth ceiling for each level of cache, so a kernel whose working set fits in the second-level cache can run against a higher roof than one that streams from main memory. There are lower compute ceilings for code that does not use the vector units, or that cannot issue fused multiply-add instructions, or that is limited by some other resource such as the number of loads per cycle. Profiling tools from Intel, NVIDIA and AMD now draw rooflines automatically, placing each measured kernel as a point beneath the roof. A kernel far below every roof is suffering from something the model does not capture, often latency, poor parallelism or excessive synchronisation, and that is itself useful diagnostic information. Stencils, the workhorse of fluid codes The kernels that dominate computational fluid dynamics and weather prediction are stencils. A stencil updates each point of a grid using the values at that point and a fixed pattern of neighbours. The canonical example is the seven-point stencil on a three-dimensional structured grid, which approximates the Laplacian of a field by taking each point's six face neighbours, summing them, subtracting six times the centre value, and scaling. Explicit time-stepping schemes for diffusion, the smoothing steps of multigrid solvers, and the inner loops of many pressure solvers all look like this. Weather models use wider stencils for advection, and finite-volume CFD codes on unstructured meshes perform the analogous operation over each cell's list of faces. Counting for the seven-point stencil is instructive. Updating one point takes roughly eight floating-point operations: six additions to gather the neighbours and the centre, and a multiplication or two to apply coefficients, depending on how the expression is written. How many bytes it needs depends entirely on what is in cache. In the worst case, if nothing is reused, each update reads seven doubles and writes one: sixty-four bytes, plus write-allocate traffic, for eight operations, giving an intensity around one-eighth of a flop per byte. In the best case, if every neighbour value read for one point is still in cache when it is needed for the next points, each grid value is read from main memory only once per sweep and the result written once, so the traffic is sixteen bytes, or twenty-four with write-allocate, per point. The intensity then rises to between a third and a half of a flop per byte. That best case is the realistic ceiling for this kernel, and it is still far below the ridge point of any modern processor. Even with perfect cache reuse, a seven-point stencil sweep is memory-bound by a factor of twenty or more. The only way to escape further is to change the algorithm so that it does more work per byte fetched, for example by applying several time steps to a block of the grid while it sits in cache, a technique called temporal blocking, or by fusing several kernels that read the same fields into one pass. The gap between the worst and best cases is where much practical optimisation happens. Whether the neighbours stay in cache depends on the grid dimensions, the loop order, and the cache size. For a grid stored with the x index varying fastest, a sweep that goes along x, then y, then z needs to keep roughly three planes of the grid in cache to reuse the z-neighbours: the plane below, the current plane, and the plane above. If those three planes fit in the last-level cache, the sweep runs near its best-case intensity; if they do not, the z-neighbours are fetched from memory again and the intensity drops. This is why the same stencil code can run at very different speeds on grids of slightly different sizes, and why cache blocking, which sweeps the grid in tiles small enough that the relevant planes fit, can produce large speed-ups without changing a single arithmetic operation. Table 2 sets out estimated intensities for a few representative kernels, derived by counting operations and bytes as in the examples above. The values are approximate and illustrative, not measurements of any particular code, but their order of magnitude is the point. Table 2. Approximate arithmetic intensity of common kernels, in double precision (illustrative estimates derived by counting). Kernel Operations per element Minimum bytes per element Intensity (flop/byte) Vector add (STREAM-like) 1 24–32 about 0.04 7-point stencil, perfect reuse about 8 16–24 about 0.3–0.5 7-point stencil, no reuse about 8 64–72 about 0.1 Sparse matrix-vector product (CSR) 2 per nonzero about 12 per nonzero under 0.2 Dense matrix multiply, blocked grows with block size shrinks with block size 10 to well over 100 The sparse matrix-vector product deserves its row in that table because it is the inner kernel of the iterative solvers that CFD codes use for pressure and implicit time-stepping. Each nonzero in a matrix stored in the common compressed sparse row format requires an eight-byte value and a four-byte column index, plus an irregular read of an element of the input vector, for one multiply and one add. Its intensity is low and its memory access pattern is irregular, so it suffers from both bandwidth and latency limits. That combination is exactly why the HPCG benchmark, built around this kernel, reaches only a small fraction of the peak that Linpack reaches on the same machines. The unstructured case: loops over faces Engineering CFD codes built on the finite-volume method on unstructured meshes have a characteristic kernel of their own, and it is worth counting it, because it behaves differently from a structured stencil. The mesh is stored as a list of cells and a list of faces; each face records the two cells it separates. The flux computation loops over faces. For each face it reads the state of the cell on each side, together with the face's geometry, such as its area and normal vector, computes a numerical flux, which for a compressible flow solver might take a few dozen to a few hundred floating-point operations depending on the scheme, and then adds the flux to one cell's residual and subtracts it from the other's. Two features distinguish this from the structured stencil. First, the reads of cell data are indirect: the code must look up which cells a face touches and then fetch their data from wherever it happens to sit in memory. If the mesh has been numbered so that neighbouring cells are close in memory, many of these fetches hit in cache; if not, each one may be a separate trip to main memory, and the latency limit rather than the bandwidth limit dominates. Second, the writes are scattered: two different faces may need to update the same cell's residual, so the loop cannot be naively parallelised across threads without risking two threads updating the same value at the same time. Codes handle this by colouring the faces into groups that touch no cell twice, by having each thread own a set of cells and loop over their faces redundantly, or by using atomic updates, each with its own cost. The arithmetic intensity of the face loop depends heavily on the flux scheme. A cheap scheme on a simple equation set behaves much like a stencil, with an intensity well below one flop per byte. An expensive scheme for a multi-species reacting flow, with many operations per face and relatively few bytes, can have a much higher intensity and approach the compute-bound regime. That variety is one reason performance advice for CFD codes is so often specific to the code: the same machine can be a good fit for one solver and a poor one for another, depending on where its dominant kernels sit relative to the ridge point. Counting for such a loop proceeds just as before. Count the bytes per face: two cells' state vectors, the face geometry, two indices, and the residual updates. Count the operations per face from the flux routine. Divide. Then measure and see how far the actual performance falls below the corresponding roof. If it falls far below, indirect access and poor cell ordering are the usual suspects, and renumbering the mesh is the usual first remedy. Using the model without fooling yourself The roofline is a model, and it can mislead in predictable ways. Three cautions are worth keeping in mind. First, the bytes that count are the ones actually moved between levels of the hierarchy, not the ones the source code appears to touch. A loop that reads an array twice may cost one memory pass or two, depending on whether the array stays in cache between the reads. Hardware performance counters, which count cache misses and memory transactions directly, are the only reliable way to know. A rough estimate made by hand is still valuable for setting expectations, but it should be checked. Second, the attainable bandwidth is not the advertised bandwidth. The figure on a datasheet is a theoretical peak; a well-written streaming kernel typically sustains a large fraction of it but rarely all of it, and a poorly vectorised or poorly parallelised one may get much less. The honest roof is the one measured with something like STREAM on the actual hardware, using the same number of cores or threads the application will use. On a CPU, a single core usually cannot saturate the memory bandwidth of a socket; several cores working together are needed. On a GPU, enough threads must be in flight to hide memory latency, which is why GPU codes need very large amounts of parallelism to reach the bandwidth roof at all. Third, the model describes throughput, not latency. A kernel that chases pointers through a linked data structure may move very few bytes and still run slowly, because each load must wait for the previous one to finish. Such kernels sit far below the roofline, and their remedy is to restructure the data so that accesses become independent and predictable, not to reduce the byte count. With those cautions, the model gives a practitioner something precious: a number for how far from ideal a kernel is. If a stencil sweep is already running at eighty per cent of the measured memory bandwidth, there is little to gain from tuning its arithmetic and nothing to gain from a faster processor with the same memory; the only further gains come from algorithmic change that reduces traffic, such as kernel fusion, temporal blocking, lower-precision storage, or a different discretisation with more work per point. If it is running at fifteen per cent, there is a fivefold improvement waiting to be found, probably in loop order, cache blocking, vectorisation or the layout of the data in memory. Why intensity favours some algorithms over others An important consequence of the memory wall is that it changes which numerical methods are attractive. For decades, numerical analysts compared methods by counting operations: a method that reached a given accuracy in fewer operations was considered better. On a memory-bound machine, operations are nearly free and bytes are what cost, so a method that does more arithmetic per grid point but reaches the same accuracy on a coarser grid, or with fewer sweeps over memory, can be faster even though its operation count is higher. This is one reason high-order methods have gained ground in CFD. Discontinuous Galerkin and spectral-element methods represent the solution inside each element by a polynomial of relatively high degree, which involves dense small-matrix operations on data that stays in cache, and so achieves much higher arithmetic intensity than low-order finite-volume schemes. Several of the atmospheric models that have moved to GPUs use spectral-element dynamical cores for exactly this reason. Matrix-free implementations of finite-element operators, which recompute element matrices on the fly rather than storing an assembled sparse matrix, follow the same logic: they trade cheap arithmetic for expensive memory traffic. It is also why reduced precision is so attractive. Storing a field in single precision rather than double halves the bytes moved for every sweep, which for a memory-bound kernel can nearly double the speed, independent of whether the arithmetic itself is any faster. The weather case study in chapter eight shows how one major forecasting centre turned that observation into an operational change. The roofline, then, is not only a diagnostic tool. It is a way of seeing the whole field. Every technique in the rest of this book, from domain decomposition to GPU data residency to parallel I/O, can be read as an attempt to raise the effective intensity of a computation at some level of the hierarchy: to do more useful work for every byte that has to travel. Chapter 3: The Message-Passing Model When a program's data no longer fits on one node, or its work can no longer be done fast enough by one node, the program must be divided among many. Because nodes do not share memory, the pieces must cooperate by sending each other messages. For more than three decades the dominant way of writing such programs has been the Message Passing Interface, MPI. It has outlived several generations of hardware, several waves of predictions of its demise, and a long list of more elegant competitors. Understanding why it has lasted, and what its model actually guarantees, is the foundation for everything else in parallel scientific computing. A standard, not a product MPI is a specification, not a piece of software. It was drawn up by the MPI Forum, a collaboration of vendors, national laboratories and universities that first met in the early 1990s, when every parallel computer manufacturer had its own incompatible message-passing library and scientific codes had to be rewritten for each new machine. The first version of the standard appeared in 1994. MPI-2, in 1997, added one-sided communication, parallel file input and output, and dynamic process management. MPI-3, in 2012, added non-blocking collective operations and substantially revised one-sided communication. MPI-4, in 2021, added persistent collectives, partitioned communication designed for multithreaded and accelerator-driven codes, and a model of sessions that relaxes the requirement for a single global initialisation. MPI-5.0, voted in by the Forum in June 2025, adds a standard application binary interface, so that a program compiled against one MPI library can in principle run with another without recompilation, a long-standing practical nuisance for anyone distributing software for clusters. Implementations of the standard come from open-source projects and from vendors. MPICH, which originated at Argonne National Laboratory, and Open MPI, a community project, are the two main open-source lines; many vendor libraries, such as HPE's Cray MPICH and Intel MPI, derive from MPICH and add optimisations for particular interconnects. MVAPICH, from Ohio State University, focuses on InfiniBand and related networks. A user rarely chooses among these directly; the system's administrators provide the one tuned for the machine. Because MPI is a library, not a language, it can be called from C, C++ and Fortran, and through bindings from Python, Julia and others. The mpi4py package has made MPI accessible to a large population of Python programmers, and it is often the most painless way to learn the concepts. Processes, ranks and communicators An MPI program is launched as a fixed number of processes, typically with a command such as mpirun -n 1024 ./solver or the system scheduler's equivalent. Every process runs the same executable, though each may take a different path through it. This style is called single program, multiple data. Processes do not share variables; each has its own private memory, even when several of them happen to run on the same node. Each process is identified by its rank, an integer from zero to one less than the total number of processes. Ranks are defined relative to a communicator, an object that represents a group of processes and a private communication context among them. Every program starts with a communicator containing all processes, called MPI_COMM_WORLD. From it, programs can create smaller communicators: one per node, one per row of a process grid, one for the processes handling the ocean in a coupled climate model and another for those handling the atmosphere. A message sent in one communicator can never be received in another, which is what allows independently written libraries to use MPI inside the same program without their messages getting mixed up. The opening of almost every MPI program looks like this: MPI_Init(&argc, &argv); int rank, size; MPI_Comm_rank(MPI_COMM_WORLD, &rank); MPI_Comm_size(MPI_COMM_WORLD, &size); After these calls, each process knows who it is and how many companions it has, and it can use that knowledge to decide which part of the problem to own. A process with rank 5 out of 64 might take the sixth slab of a 64-slab domain. From that point, the program's correctness depends on every process agreeing, without being told, on how the problem has been divided. Point-to-point communication The most basic operation is a message from one process to another. The sender calls MPI_Send with a buffer, a count, a datatype, a destination rank, a tag and a communicator. The receiver calls MPI_Recv with a buffer, a count, a datatype, a source rank, a tag and the same communicator. The message is delivered when a send and a receive match on communicator, source, destination and tag. The receiver may use wildcards to accept a message from any source or with any tag, at the cost of some performance and some predictability. MPI guarantees that messages between a given pair of processes in the same communicator are non-overtaking: if process A sends two messages to process B that both match the same receive, B receives them in the order A sent them. There is no such guarantee across different senders. If processes A and C both send to B, the order in which B receives them depends on timing. The datatype argument is more important than it looks. MPI does not simply move bytes; it moves typed elements, which allows it to convert between representations on heterogeneous systems and, more usefully today, to describe non-contiguous data. A derived datatype can describe, for instance, one column of a row-major matrix, or one face of a three-dimensional block, as a strided pattern in memory. The library can then send that face without the programmer first copying it into a contiguous buffer, and on some hardware it can gather the pieces directly as it sends. Whether this is faster than manual packing depends on the implementation, and it is worth measuring, but derived datatypes make code clearer either way. Blocking, buffering and the anatomy of a deadlock The semantics of MPI_Send trip up nearly every newcomer. The standard says that a blocking send returns when the send buffer can safely be reused. It does not say the message has been received, or even that the matching receive has been posted. An implementation is free to copy a small message into an internal buffer and return immediately; it is also free to wait until the receiver is ready and then transfer the data directly. In practice, most implementations do the former for small messages, using what is called an eager protocol, and the latter for large ones, using a rendezvous protocol, with the threshold between them depending on the library and the network. This leads to a classic bug. Suppose two processes each need to exchange a block of data with the other, and each calls MPI_Send to the other and then MPI_Recv from the other. For small messages this usually works, because both sends complete eagerly into buffers. For large messages both sends wait for a matching receive that neither process will post until its send returns, and the program hangs forever. The code is incorrect in both cases; it simply fails to show its incorrectness until the problem size grows. A program that runs correctly on a laptop with a test case and hangs on the supercomputer with the production case is often exhibiting exactly this behaviour. There are several cures. The processes can be ordered so that one sends first while the other receives first, which works for pairs but becomes tangled for larger patterns. The combined call MPI_Sendrecv performs a send and a receive together and lets the library order them safely. The most general cure, and the one that also improves performance, is to use non-blocking communication. Non-blocking communication The non-blocking calls MPI_Isend and MPI_Irecv initiate an operation and return immediately with a request handle. The program may then do other work and later call MPI_Wait or MPI_Waitall on the requests, which return when the operations are complete. Between the initiation and the completion, the program must not modify a buffer being sent or read a buffer being received. MPI_Request req[4]; MPI_Irecv(ghost_left, n, MPI_DOUBLE, left, 0, comm, &req[0]); MPI_Irecv(ghost_right, n, MPI_DOUBLE, right, 1, comm, &req[1]); MPI_Isend(edge_left, n, MPI_DOUBLE, left, 1, comm, &req[2]); MPI_Isend(edge_right, n, MPI_DOUBLE, right, 0, comm, &req[3]); compute_interior(); MPI_Waitall(4, req, MPI_STATUSES_IGNORE); compute_boundary(); This fragment, the skeleton of a one-dimensional halo exchange, shows the two benefits at once. Because all the receives are posted before any send must complete, there is no ordering that can deadlock. And because the interior computation, which does not depend on neighbour data, happens between initiation and completion, the communication may overlap with useful work. Chapter five develops this pattern in full. The word "may" deserves emphasis. Whether communication actually progresses while the program is computing depends on the implementation and the hardware. Some networks can move data entirely in hardware once a transfer is started. In other cases, the MPI library needs the process to call into it periodically to advance the protocol, for instance to respond to a rendezvous handshake; if the process is busy in a long computational loop, the transfer may not begin until the wait call. This problem of asynchronous progress is one of the most common reasons that apparently overlapped code shows no overlap at all. Remedies include calling MPI_Test occasionally inside long loops, enabling a progress thread if the library offers one, or relying on hardware offload where it exists. Persistent, one-sided and partitioned communication Many simulations perform the same communication pattern at every time step: the same neighbours, the same buffer addresses, the same sizes. Persistent requests, created once with MPI_Send_init and MPI_Recv_init and then started repeatedly, let the library do its setup work once, which can reduce overhead for small, frequent messages. MPI-4 extended the idea to collective operations. One-sided communication, sometimes called remote memory access, takes a different approach. Each process exposes a region of its memory as a window, and other processes can put data into it or get data from it without the target process making a matching call. Synchronisation is handled separately, with fences or with finer-grained locks and flushes. On networks with good remote direct memory access support, this can reduce overhead, and it suits irregular patterns in which a process does not know in advance who will send it data. It is also harder to reason about, because the program must make sure that a put is complete and visible before the target reads the data, and the synchronisation rules are subtle. Partitioned communication, new in MPI-4, addresses a pattern that has become common with multithreaded and GPU-accelerated codes: a large buffer is filled in parts by many threads, and it would be wasteful either to wait until every thread has finished before sending, or to send many small separate messages. The program declares a send buffer divided into partitions, and each thread marks its partition ready when done; the library may transmit partitions as they become ready. Getting it right: the bugs MPI invites The freedom MPI gives the programmer is also a freedom to make errors that are hard to find. Some are simple mismatches. A receive that specifies a count smaller than the incoming message produces a truncation error; one that specifies the wrong tag or source simply never matches, and the program hangs at the next wait. A send and receive that disagree on datatype may transfer the right number of bytes but interpret them as the wrong kind of number, producing garbage that propagates silently through the simulation. Collective operations must be called by every process in the communicator in the same order; a process that skips a collective because of a conditional that evaluates differently on different ranks will deadlock the whole program or, worse, match its next collective with the others' current one and continue with corrupted data. Non-blocking communication adds a further class of error. A program that modifies a send buffer before the corresponding wait has completed may send the old values, the new values, or a mixture, depending on timing and on whether the library has already copied the data. A program that reads a receive buffer before the wait may see stale data. Such bugs often appear only intermittently and only at scale, because they depend on how quickly messages happen to be delivered, which is the worst possible property for a bug to have. There is tooling for these problems. Correctness checkers such as MUST, developed at RWTH Aachen and the Technical University of Dresden, intercept every MPI call through the profiling interface and check for mismatched types, leaked requests, buffer reuse and potential deadlocks, including those that would only occur under a different message protocol. Parallel debuggers, such as Linaro DDT and Perforce's TotalView, can attach to thousands of processes, show where each one is stopped, and group processes with identical call stacks, which makes it quick to see, for instance, that one rank is waiting in a receive while all the others are waiting in an allreduce. A habit of running every new piece of communication code under a correctness checker at small scale, before running it at large scale, saves a great deal of expensive allocation time. A second habit is to make the communication structure of the program as regular and explicit as the problem allows. Code that computes its list of neighbours, buffer sizes and tags once, at start-up, in one well-tested routine, and then reuses them every step, has far fewer opportunities for mismatch than code that constructs messages ad hoc inside physics routines. This is one reason mature codes concentrate their MPI calls in a small communication layer, often a few thousand lines, which the rest of the code calls through higher-level operations such as "exchange halos for these fields" or "sum this quantity globally". The pattern isolates the hardest parallel logic in one place and makes it possible to change the communication strategy, for instance to switch to neighbourhood collectives or to GPU-aware transfers, without touching the science. Why MPI has lasted MPI is frequently criticised as low-level. The programmer must decide explicitly which process owns which data, compute which neighbours to talk to, pack and unpack messages, and keep track of request handles. Higher-level alternatives have repeatedly promised to free scientists from this bookkeeping: languages with partitioned global address spaces such as Unified Parallel C, Coarray Fortran and Chapel; task-based runtimes such as Charm++, Legion and HPX; and, in data analytics, frameworks such as Spark. Several of these are excellent tools for particular problems, and some, notably Charm++ in the molecular dynamics code NAMD, underpin major applications. Yet MPI remains the default, for reasons that follow directly from the argument of this book. Because data movement dominates performance, a programming model that makes data movement explicit lets the programmer see and control the most important cost. A message in MPI is a visible event in the source code; its size, its partner and its timing can be read off the page. Models that hide communication behind array syntax or a runtime scheduler can achieve excellent performance, but when they do not, the programmer often has little insight into why. MPI's explicitness is a burden while writing code and an asset while tuning it. The other reason is ecosystem. Decades of numerical libraries, including PETSc, Trilinos, hypre and ScaLAPACK, are built on MPI. Every supercomputer vendor ships a tuned MPI library on day one, and procurement contracts often specify MPI performance targets. A scientific code written with MPI in 2000 still compiles and runs on a 2026 exascale system. Few other pieces of software infrastructure can say as much. What has changed is how MPI is used. It is now rarely the only programming model in a code. Within a node, threads or GPU kernels do the arithmetic; MPI moves data between nodes, and increasingly between GPUs. The chapters that follow treat MPI in that role: as the layer responsible for the most expensive journeys data makes. Chapter 4: Collectives, Synchronisation and the Limits of Scaling Point-to-point messages connect pairs of processes. A great deal of parallel computation, however, requires all the processes in a group to take part in the same operation at the same moment: to agree on the size of the next time step, to sum a residual across the whole domain, to broadcast a configuration, or to redistribute data from one decomposition to another. These collective operations are where the cost of parallelism shows itself most plainly, because they couple every process to every other. Understanding their cost, and the wider theory of how parallel performance changes with the number of processors, is what allows a programmer to predict whether a code will scale before spending a large allocation finding out. A simple model of communication cost The standard first approximation of the time to send a message of n bytes between two processes is a fixed latency plus a per-byte cost: time equals alpha plus n times beta. Alpha captures everything that does not depend on the message size, including software overhead in the MPI library, the time for the network interface to start the transfer, and the propagation delay through the switches. Beta is the reciprocal of the bandwidth. This is sometimes called the Hockney model, after Roger Hockney, who used it to characterise early parallel machines, and it is crude but surprisingly predictive. The model's most important consequence is that there are two regimes. For small messages, alpha dominates, and sending ten small messages costs nearly ten times as much as sending one message ten times as large. For large messages, beta dominates, and the cost is essentially proportional to the data volume. On current interconnects, alpha is in the vicinity of a microsecond and the bandwidth is in the range of tens of gigabytes per second per link, so the crossover where the two terms are equal is at a message size of some kilobytes to tens of kilobytes. The practical rule follows: aggregate small messages wherever possible, and for large messages, concentrate on reducing the volume. More detailed models exist. The LogP model, introduced by David Culler and colleagues in 1993, separates latency from the processor overhead of sending and receiving and from the minimum gap between consecutive messages, which captures the fact that a processor cannot inject messages infinitely fast. Its extensions handle long messages more accurately. For most practical reasoning, the two-parameter model is enough, provided one remembers that alpha includes software costs that a busy or poorly configured node can inflate. How collectives are implemented A collective operation is implemented by the MPI library as a pattern of point-to-point messages, chosen to suit the message size and the number of processes. Consider the operation that matters most to simulation codes, MPI_Allreduce, which combines a value from every process with an operation such as sum or maximum and returns the result to all of them. A naive implementation would have every process send its value to rank zero, which sums them and sends the result back. With p processes this takes time proportional to p in both directions, and rank zero becomes a bottleneck. A tree-based implementation does much better: processes pair up and combine values, then pairs of pairs combine, and so on, so that the reduction completes in about log₂ p rounds, followed by a broadcast down the tree in another log₂ p rounds. For small messages, where latency dominates, logarithmic depth is what matters. An algorithm called recursive doubling achieves the same result in about log₂ p rounds total, by having each process exchange its partial result with a partner at distance one, then two, then four, and so on, so that after the last round every process holds the full sum. For large messages, the bandwidth term dominates, and tree algorithms are wasteful because every round sends the whole vector. Here the preferred approach splits the vector into p pieces. In a reduce-scatter phase, each process ends up responsible for one fully reduced piece, after passing pieces around a ring or exchanging them by recursive halving. In an allgather phase, the reduced pieces are distributed so that every process has all of them. The total data each process sends is about twice the vector size regardless of p, which is close to the theoretical minimum. This combination is often associated with Rolf Rabenseifner, who analysed it in the early 2000s, and the ring variant has become famous in deep learning, where large gradient vectors are all-reduced across many GPUs at every training step. Table 3 summarises the trade-off in the two-parameter model, for p processes and a vector of n bytes, ignoring the arithmetic cost of the reduction itself. Table 3. Approximate cost of allreduce algorithms in the latency-bandwidth model (p processes, n bytes; derived, illustrative). Algorithm Latency term Bandwidth term Best suited to Reduce to root, then broadcast (binary tree) about 2 log₂ p × alpha about 2 log₂ p × n × beta small vectors Recursive doubling about log₂ p × alpha about log₂ p × n × beta small vectors, p a power of two Ring reduce-scatter plus allgather about 2(p − 1) × alpha about 2n × beta very large vectors Recursive halving plus doubling (Rabenseifner) about 2 log₂ p × alpha about 2n × beta medium to large vectors Real libraries go further. They detect the hierarchy of the machine, doing a fast reduction among processes on the same node through shared memory, then a reduction among one representative per node over the network, then a broadcast back within each node. They may offload reductions to the network switches, as NVIDIA's SHARP technology does on InfiniBand. They choose algorithms by message size using tables tuned at installation time. The programmer seldom needs to know which algorithm is chosen, but should know that the cost of a small allreduce grows roughly logarithmically with the number of processes and that the cost of a large one is roughly constant in p and proportional to n. The hidden cost: synchronisation The arithmetic of the latency-bandwidth model understates the true cost of collectives, because it assumes every process arrives at the collective at the same moment. In reality they arrive at different times, and the collective cannot complete until the last one arrives. A process that reaches an allreduce early simply waits. The time lost is not a property of the collective at all; it is the accumulated imbalance of everything that happened since the last synchronisation, made visible. Imbalance comes from many sources. The work may be divided unevenly: a CFD mesh partition with slightly more cells, or more cells of an expensive type, takes longer to update. The physics may vary across the domain: in a weather model, columns with active convection or complex cloud microphysics cost more than columns of clear air, and the expensive columns move as the weather moves. The hardware may vary: a processor running slightly slower because of its thermal state, or a node whose memory is slightly slower, holds everyone up. And the operating system may intervene. That last effect has a famous history. In 2003, Fabrizio Petrini, Darren Kerbyson and Scott Pakin published an analysis of the ASCI Q supercomputer at Los Alamos, titled "The Case of the Missing Supercomputer Performance", showing that a hydrodynamics code ran far below its predicted speed at scale because of operating-system noise: brief, unsynchronised interruptions from daemons and kernel activity on each node. Any one interruption was negligible, but in a code that synchronised frequently across thousands of processors, the probability that at least one processor was interrupted during each interval approached certainty, so every interval took as long as the slowest. Removing or consolidating the noisy activities recovered much of the lost performance. The lesson, that rare delays on individual processors become the common case at scale, recurs with every new generation of machine. Modern systems mitigate it with lightweight operating-system configurations, by reserving a core per node for system activity, and by pinning processes to cores. The practical moral is that the number of synchronisation points matters as much as the cost of each. Algorithms that need a global reduction at every iteration pay the imbalance tax every iteration. Krylov solvers and the cost of an inner product This point bites hardest in the iterative linear solvers at the heart of implicit CFD and of the pressure equations in many atmospheric and ocean models. The conjugate gradient method, the workhorse for symmetric positive-definite systems, needs two inner products per iteration, and each inner product across a distributed vector requires an allreduce of a single number. GMRES, used for non-symmetric systems, needs a growing number of inner products per iteration. On a few hundred processes these reductions are cheap. On tens of thousands, the latency of the reductions, plus the imbalance they expose, can come to dominate the iteration time, even though each reduction moves only eight bytes. Numerical analysts have responded with communication-avoiding and pipelined variants of these methods. Pipelined conjugate gradient, described by Pieter Ghysels and Wim Vanroose in 2014, rearranges the algorithm so that the global reduction for one iteration can be overlapped with the sparse matrix-vector product of the next, using MPI's non-blocking MPI_Iallreduce. Other approaches perform several iterations' worth of matrix-vector products before a single combined reduction. These methods trade a little extra arithmetic, and sometimes some numerical stability, for fewer or hidden synchronisations, which on a modern machine is usually a good trade. They are now available in libraries such as PETSc. Preconditioners raise the same issue at a larger scale. Multigrid methods, which solve a problem on a hierarchy of coarser and coarser grids, are optimal in operation count, but on the coarsest grids there is very little work and a great deal of communication per unit of work. At scale, the coarse levels are often gathered onto fewer processes, or solved redundantly, to avoid having thousands of processes exchange tiny messages about tiny grids. Amdahl, Gustafson and what scaling means Two laws frame the theory of parallel speed-up, and both are frequently misquoted. Gene Amdahl's argument, made in 1967, is that if a fraction s of a program's work is inherently serial, the speed-up on any number of processors can never exceed one over s. A program that is ninety-nine per cent parallel can be made at most a hundred times faster. The argument is unanswerable for a fixed problem, and it governs what is called strong scaling: running the same problem on more processors to finish it sooner. In strong scaling, each processor's share of the work shrinks as processors are added, while some costs, such as the serial fraction, the latency of collectives, and the surface-to-volume ratio of each processor's subdomain discussed in the next chapter, shrink more slowly or not at all. Eventually adding processors makes the program slower. John Gustafson, writing in 1988 from his experience at Sandia National Laboratories, observed that scientists with a bigger machine rarely run the same problem faster. They run a bigger problem in the same time: a finer mesh, a longer simulation, a larger ensemble. If the parallel part of the work grows with the problem while the serial part stays fixed, the serial fraction shrinks as the machine grows, and the achievable speed-up grows nearly in proportion to the number of processors. This is weak scaling: holding the work per processor fixed while increasing both problem size and processor count. Ideal weak scaling means the time per step stays constant. The two laws do not contradict each other; they answer different questions. Weather services care intensely about strong scaling, because the deadline is fixed and the forecast model has a fixed resolution in any given operational configuration; they want that configuration to run faster so they can use the time for something else. Climate scientists care about both: strong scaling to complete century-long simulations within a reasonable time, weak scaling to increase resolution. CFD engineers designing a product often care about throughput, running many moderate simulations in parallel, which is a third regime in which each job scales only modestly and the machine is filled with many of them. A performance study should therefore state which kind of scaling it measured, the problem size, and the efficiency, which is the speed-up divided by the increase in processors. A plot showing a code scaling to a million cores is meaningless without knowing whether the problem grew with the machine. Weak scaling to large core counts is comparatively easy, and many codes achieve it. Strong scaling to the point where each processor holds only a few thousand grid points is hard, and it is where the techniques of the next chapters earn their keep. Estimating a scaling limit on paper The models in this chapter are useful mainly because they allow rough predictions before a large run is attempted. A worked example, with round illustrative numbers rather than measurements from any real code or machine, shows how. Suppose an explicit fluid solver on a structured grid of one billion cells takes, on a single CPU core, about one microsecond of computation per cell per time step, including all its stencils. Each step also requires one halo exchange with six neighbours, of one cell's worth of five double-precision variables per face cell, and one small allreduce to determine the next time step from a stability criterion. Assume the network has a latency of two microseconds per message, a bandwidth of ten gigabytes per second per process, and that a small allreduce costs about two microseconds times the base-two logarithm of the process count. On 1,000 processes, each owns a million cells, a cube a hundred cells on a side. Computation takes one second per step. The halo exchange moves six faces of ten thousand cells each, times forty bytes per cell, about 2.4 megabytes, which at ten gigabytes per second takes about a quarter of a millisecond, plus a negligible latency. The allreduce takes about twenty microseconds. Communication is well under one per cent of the step. On 125,000 processes, each owns eight thousand cells, a cube twenty cells on a side. Computation takes eight milliseconds per step. The halo moves six faces of four hundred cells, about 96 kilobytes, taking roughly ten microseconds, plus about twelve microseconds of latency for the six messages. The allreduce takes about thirty-four microseconds. Communication is now under one per cent still, at about fifty-six microseconds against eight thousand. The model says this code should strong-scale almost perfectly across that range. In practice, it would not, and the reasons are instructive. The model has left out the imbalance tax at the allreduce, which is the variability in each process's eight milliseconds of computation; operating-system noise and uneven cache behaviour can easily make that variability a few per cent, which at this scale exceeds the whole communication cost. It has left out the packing and unpacking of halo buffers, which for small subdomains is a large share of the boundary work. It has assumed that computation per cell is the same for small subdomains as for large ones, whereas a twenty-cell cube has proportionally more boundary cells, whose updates may be handled by slower code paths, and different cache behaviour. And it has assumed that the network delivers the same bandwidth to every process when all of them communicate at once. The value of the exercise is not the prediction of near-perfect scaling, which is too optimistic, but the structure of the result. It shows that for this code, pure message costs are unlikely to be the limit; synchronisation, imbalance and boundary overheads are. That directs the measurement effort to the right place. A different code, say one with a global transpose or a Krylov solver with many inner products per step, would produce a very different paper estimate, and would direct attention elsewhere. Designing for fewer meetings The cumulative lesson of collectives and scaling is that a parallel program should be designed around minimising the number of moments at which every process must wait for every other. Each such moment costs a latency that grows with the logarithm of the process count and an imbalance penalty that grows with the variability of the work. Non-blocking collectives allow those moments to be overlapped with useful computation; algorithmic reformulations reduce their number; good load balancing reduces the waiting at each. None of these changes the arithmetic. All of them change how often data has to be brought together, and that, at scale, is what decides whether a code finishes on time. Chapter 5: Domain Decomposition and the Halo Exchange Almost every large simulation of a physical field, whether airflow over a wing, the circulation of the ocean, or the evolution of the atmosphere, is parallelised in the same basic way. The physical domain is cut into pieces, each piece is assigned to a process, and each process updates its own piece. Because the equations couple each point to its neighbours, the processes owning adjacent pieces must regularly exchange the values along their shared boundaries. This pattern, domain decomposition with halo exchange, is so common that it is worth studying in detail. It is where the abstract cost model of the previous chapter meets the geometry of the problem. Ghost cells Consider a stencil computation on a three-dimensional structured grid, split into blocks with one block per process. To update a point on the edge of its block, a process needs the values of neighbouring points that belong to an adjacent block. Rather than requesting each of these individually, the process keeps an extra layer of cells around its block, called ghost cells or the halo, which hold copies of the neighbours' boundary values. At the start of each step, every process sends its own boundary values to its neighbours and receives theirs into its halo. Then it can update all of its points using only local data. The width of the halo is set by the stencil. A seven-point stencil, which reaches one point in each direction, needs a halo one cell wide. A higher-order advection scheme that reaches three cells needs a halo three cells wide. If the stencil includes diagonal neighbours, the halo must include edge and corner regions, which requires either exchanges with diagonal neighbours or a sequence of exchanges along each axis in turn so that corner values propagate through the face exchanges. The structure of a halo exchange is the skeleton shown in chapter three, generalised to more dimensions: post receives from each neighbour into the appropriate part of the halo, pack and send the boundary faces, compute whatever does not depend on the halo, wait for completion, then compute the rest. Every grid-based code has a version of this routine, and its efficiency often determines how well the whole code scales. Surface and volume The fundamental economics of domain decomposition come from geometry. The computation a process performs is proportional to the volume of its subdomain, the number of cells it owns. The communication it performs is proportional to the surface area of its subdomain, the number of cells on its boundary times the halo width. As a problem is divided among more processes, each subdomain shrinks, and its surface shrinks more slowly than its volume. Some arithmetic makes the point. For a cubic subdomain of side n cells with a one-cell halo, the volume is n³ and the surface is about 6n². The ratio of communicated cells to computed cells is about 6/n. With n equal to 100, a million cells per process, that ratio is six per cent. With n equal to 20, eight thousand cells per process, it is thirty per cent. If each communicated value is also more expensive to handle than each computed value, as it typically is because it must be packed, sent and unpacked, the communication cost rises from a small overhead to a major part of each step. This is the geometric root of the strong-scaling limit: divide a fixed problem finely enough and the processes spend more time talking about their boundaries than working on their interiors. The shape of each subdomain matters too. Cutting a cube into slabs, so each process owns a full plane of the domain one or a few cells thick, is the easiest decomposition to program, but each slab has two enormous faces. Cutting it into pencils, long thin columns, reduces the surface for the same volume. Cutting it into roughly cubic blocks minimises the surface for a given volume and therefore minimises communication. MPI provides helpers for this: MPI_Dims_create suggests a balanced factorisation of the process count into a grid of the requested dimension, and MPI_Cart_create builds a communicator with Cartesian topology so that each process can find its neighbours with MPI_Cart_shift. The library may also use the topology information to place neighbouring ranks on nearby nodes, though in practice much of the benefit comes from the launch configuration and the scheduler. Weather and climate models usually decompose only in the horizontal. A column of the atmosphere, from the surface to the top of the model, is kept on one process, because the physical parameterisations for radiation, convection and the boundary layer operate vertically within each column and would otherwise require constant communication. The horizontal decomposition is then two-dimensional, and the vertical extent of each subdomain is fixed at the full number of model levels. This shapes the arithmetic: each process exchanges halos only along its horizontal edges, but each exchanged column contains a hundred or more levels of many variables. Unstructured meshes and graph partitioning Structured grids are easy to divide into blocks. Most engineering CFD, however, uses unstructured meshes that follow complicated geometry and vary in resolution: fine cells in boundary layers near walls and in wakes, coarse cells in the far field. For these meshes, the decomposition problem becomes a graph-partitioning problem. Each cell is a vertex; each shared face is an edge. The goal is to divide the vertices into p parts with nearly equal total weight, where the weight reflects computational cost, while minimising the number of edges cut, which reflects communication. This problem is computationally hard to solve exactly, but excellent heuristics have existed since the 1990s. The multilevel approach, popularised by George Karypis and Vipin Kumar in the METIS software, repeatedly coarsens the graph by merging adjacent vertices, partitions the small coarse graph, and then projects the partition back to the fine graph while refining it locally. ParMETIS performs the same process in parallel for meshes too large to partition on one node, and Scotch, from the French research institute Inria, offers similar capabilities and is widely used, including as a default option in the open-source CFD package OpenFOAM. Zoltan, from Sandia, provides a toolkit of partitioning and load-balancing methods behind a common interface. An alternative, cheaper approach uses space-filling curves, such as the Hilbert or Morton curves, which visit every cell of a domain in an order that keeps nearby cells close together along the curve. Cutting the curve into p equal-weight segments gives a decomposition with reasonable locality, computed in a fraction of the time a graph partitioner takes. Space-filling curves are popular in adaptive mesh refinement codes, where the decomposition must be recomputed frequently as the mesh changes, and in some weather and climate models that use them to balance work across irregular sets of grid blocks. The output of a partitioner is not just an assignment of cells to processes. The code must also build, for each process, a list of which of its cells must be sent to which neighbour and which halo cells are to be received from where. These communication lists are usually built once at start-up and reused every step, which is an ideal case for persistent requests or neighbourhood collectives. Neighbourhood collectives MPI-3 introduced a family of operations designed specifically for this kind of sparse, fixed communication pattern. A program first creates a communicator with a distributed graph topology, declaring for each process which ranks it sends to and which it receives from. It can then call MPI_Neighbor_alltoallv, which in a single call sends a different block of data to each of its declared neighbours and receives a block from each. This expresses the whole halo exchange as one operation, which is clearer than a hand-written loop of sends and receives and gives the library the chance to optimise the pattern as a whole, for instance by scheduling messages to avoid congestion. A non-blocking variant allows the exchange to be overlapped with computation. In practice the performance advantage over well-written point-to-point code varies between implementations, but the clarity advantage is consistent. Overlapping communication with computation The most powerful optimisation of the halo exchange is to hide it behind computation. The idea is simple: the interior cells of a subdomain, those far enough from the boundary that their stencil does not reach into the halo, can be updated without waiting for neighbour data. So a process initiates its halo exchange, updates its interior, waits for the exchange to complete, and then updates its boundary cells. If the interior update takes longer than the exchange, the communication is entirely hidden, and the scaling penalty of the halo largely disappears. Achieving this in practice requires several things to go right. The interior must be large enough to provide cover, which is exactly what fails at the extreme of strong scaling, when subdomains are small and almost every cell is a boundary cell. The MPI library must progress the communication while the process computes, as discussed in chapter three. The code must be structured so that interior and boundary loops are separate, which complicates the implementation and can reduce cache efficiency by splitting what was one sweep into two. And on GPUs, where kernels are launched asynchronously and communication may involve the host, the orchestration involves streams and events, discussed in chapter seven. When overlap works, the gains are substantial. When it does not, a common reason is that the boundary computation was not actually independent, or that packing the send buffers is itself expensive and was left on the critical path. Profiling with a tool that shows the timeline of computation and communication on each process, rather than just totals, is the way to tell. Deeper halos and communication avoidance An alternative strategy trades extra computation for fewer messages. If a process keeps a halo two layers deep instead of one, it can perform two time steps of a one-cell stencil before exchanging again: in the first step, it updates its own cells and the inner layer of its halo, using the outer layer; in the second, it updates only its own cells. It does some redundant work on cells that a neighbour also updates, and each exchange moves more data, but it halves the number of exchanges and therefore halves the latency cost. For small subdomains where latency dominates, this can be a net win. The idea generalises to k layers and k steps, and to more complex schemes, and it belongs to a broader research programme on communication-avoiding algorithms that has produced versions of many linear-algebra and stencil methods designed for machines where messages are expensive. Weather and ocean models use a related trick for the multiple stages of a Runge-Kutta time integration scheme, or for the several tracer fields advected each step: rather than exchanging the halo of each field separately, they pack several fields into one message, reducing the number of messages and therefore the total latency. Load balance as a moving target A decomposition that divides cells equally does not necessarily divide work equally. In CFD, cells near a moving shock or in a region undergoing chemical reaction may require more iterations of a nonlinear solver. In adaptive mesh refinement, the mesh itself changes as features develop, so a balanced decomposition at the start becomes unbalanced over time. In weather models, the cost of the physics varies with the weather: clear skies are cheap, thunderstorms are expensive, and day and night differ because radiation calculations are more expensive in sunlit columns in some schemes. Codes deal with this in several ways. Static weights, assigned to cells according to their expected cost and fed to the partitioner, handle predictable variation. Periodic repartitioning, in which the code measures the actual time spent per cell, recomputes the decomposition, and migrates data to match, handles slow drift; space-filling-curve methods make this cheap. Over-decomposition, in which each process owns several small blocks rather than one large one, allows blocks to be moved between processes without recomputing a global partition, and is the basis of task-based approaches such as Charm++. For the physics in weather models, some codes go further and redistribute columns among processes for the physics step alone, using a separate, physics-balanced decomposition, and then return them for the dynamics. Each of these strategies costs data movement to reduce waiting. Whether the trade is worthwhile depends on how unbalanced the work becomes and how often. The right answer, here as throughout, comes from measurement rather than intuition. A worked example: a global atmospheric grid To see how these considerations combine in a real-sized problem, consider a global atmospheric model with a horizontal grid spacing of about five kilometres. The surface of the Earth is about 510 million square kilometres, so the grid has roughly twenty million columns. Suppose each column has a hundred vertical levels and the dynamical core carries ten prognostic variables per cell in double precision, so each cell holds eighty bytes and each column, across its hundred levels, about eight kilobytes. These are illustrative figures chosen for easy arithmetic, not the specification of any particular model. Divide the twenty million columns among 4,000 GPUs, five thousand columns each, as a roughly square patch about seventy columns on a side. Each patch has four edges of about seventy columns. A halo three columns wide, as a typical higher-order advection scheme might need, means each patch receives about four times seventy times three, or 840 columns of halo data per exchange, plus corners. At eight kilobytes per column that is about 6.7 megabytes per exchange. If the model performs several halo exchanges per time step, for different stages of its time integration and for different groups of variables, the total per step might be a few tens of megabytes per GPU, which a modern interconnect delivers in a few milliseconds. Now notice the ratio. The patch owns five thousand columns; its halo is 840. The halo is about seventeen per cent of the owned data, which is substantial but tolerable, and the GPU has five thousand columns of a hundred levels each, half a million cells, on which to keep its threads busy while the exchange proceeds. Double the GPU count to 8,000, and each patch shrinks to about fifty columns on a side, with a halo of about six hundred columns against 2,500 owned: twenty-four per cent. The interior available to hide communication has halved, while the halo has shrunk only by about thirty per cent. Double again to 16,000, and the patch is about thirty-five columns on a side, the halo about 420 columns against 1,250, a third. Each GPU now has only 125,000 cells, which may be too few to keep it fully occupied. This is the practical shape of the strong-scaling limit for GPU weather models. It explains why such models aim for a particular number of columns per GPU and why kilometre-scale global simulations need machines with thousands of GPUs: not because each GPU is slow, but because each GPU must own enough of the planet to work efficiently, and the planet at kilometre scale is very large. It also explains the attention paid to halo width. A scheme that needs a three-column halo rather than a one-column halo triples the communication volume, and numerical methods that achieve the same accuracy with narrower stencils have a real advantage at scale. What the pattern teaches Domain decomposition makes the controlling idea of this book concrete. The work of the simulation is fixed by the physics and the mesh. What the programmer controls is how the data is divided, and that division determines how much data must move between processes, how often, and how much each process must wait for others. A good decomposition minimises surface for a given volume, balances the work rather than the cell count, groups messages to reduce latency, and arranges computation so that messages travel while useful work continues. None of these choices touches the arithmetic. All of them can change the run time by large factors at scale. Chapter 6: Inside the Node: Caches, NUMA and Hybrid Parallelism A parallel code can have a flawless decomposition and a perfectly overlapped halo exchange and still run at a small fraction of what the hardware allows, because each process uses its own share of the machine badly. The previous chapters treated the process as a unit that computes and communicates. This chapter looks inside that unit, at the layers of memory between a core and the DRAM, and at how the arrangement of data and loops decides whether the hardware's elaborate machinery for hiding memory latency helps or is defeated. It also considers how the processes and threads of a program should be arranged on a node, which has become a central question now that nodes have more than a hundred cores. Layout is destiny The first and largest decision is how data is laid out in memory. A fluid-dynamics code storing several variables per cell, such as density, three components of momentum, and energy, can store them in two basic ways. An array of structures keeps all the variables for one cell together, followed by all the variables for the next cell. A structure of arrays keeps all the densities in one array, all the x-momenta in another, and so on. Which is better depends on how the data is used. If a kernel uses all five variables of each cell together, an array of structures brings them into cache in one line. If a kernel uses only density, an array of structures wastes four-fifths of every cache line it loads, because the other variables come along uninvited. A structure of arrays loads only what is needed, and it also gives the vector units what they want: consecutive values of the same variable in consecutive memory locations, which can be loaded into a vector register in one instruction and processed in parallel. For most simulation codes, where different kernels use different subsets of variables and where vectorisation matters, structures of arrays or a hybrid, blocks of a few cells stored in structure-of-arrays order, perform better. GPU codes are even more sensitive, for reasons discussed in the next chapter. For multidimensional arrays, the order of indices matters in the same way. In C, the last index varies fastest in memory; in Fortran, the first does. A loop nest that walks memory in the wrong order, striding across large distances on its innermost loop, touches a new cache line on almost every access and defeats prefetching. Reordering the loops so that the innermost loop runs along the fastest-varying index can change performance by large factors with no change in the operations performed. Compilers can sometimes interchange loops automatically, but only when they can prove it is safe, and in large scientific codes with pointer aliasing and complex control flow, they often cannot. On unstructured meshes, the equivalent issue is the numbering of cells. If cells that are neighbours in the mesh are stored far apart in memory, a loop over faces that reads the two cells on either side of each face accesses memory almost randomly. Renumbering the cells so that neighbours have nearby indices, for example with the reverse Cuthill-McKee algorithm or a space-filling curve, can substantially improve cache reuse. Mesh-generation tools do not always produce good orderings, so renumbering is a standard pre-processing step in serious unstructured codes. Making caches work The roofline analysis of chapter two showed that a stencil sweep's intensity depends on whether neighbouring values remain in cache long enough to be reused. Cache blocking, also called tiling, is the technique for ensuring that they do. Instead of sweeping the whole grid along each axis, the code divides the grid into tiles small enough that a tile's working set fits in cache, and sweeps each tile completely before moving to the next. For a three-dimensional stencil, blocking in the two faster-varying dimensions so that a few planes of a tile fit in the second- or third-level cache is usually enough to recover most of the reuse. The right tile size depends on the cache sizes, the number of arrays the kernel touches, and the number of cores sharing each cache, so it is usually found by experiment or by an autotuning tool that tries a range of sizes. Some code-generation frameworks for stencils, such as Devito, which generates optimised finite-difference code from a high-level symbolic description and has been used in seismic imaging, perform this tuning automatically. Beyond spatial blocking lies temporal blocking, mentioned in chapter two: performing several time steps on a tile before moving on. Because each time step depends on neighbouring values from the previous step, the tiles must overlap or be shaped as trapezoids or diamonds in space-time, and the bookkeeping is substantial. When it works, it can raise the effective arithmetic intensity well beyond what a single sweep allows. It is more commonly found in research codes and specialised frameworks than in large production models, where the complexity of the physics makes it hard to apply. A simpler and more widely applicable technique is kernel fusion. Large simulation codes are often written as a sequence of loops over the grid, each computing one term: one for advection, one for diffusion, one for the pressure gradient, and so on. Each loop streams the relevant fields through memory. If two consecutive loops read the same fields, fusing them into one loop means the fields are loaded once instead of twice, nearly halving the memory traffic for that pair. The software-engineering cost is that fused loops are harder to read and maintain, which is one reason the weather and climate community has invested in domain-specific languages that let scientists write loops separately while a compiler fuses them, a story chapter eight returns to. Vectorisation Every modern CPU core contains vector units that apply the same operation to several values at once: four or eight doubles in one instruction with the AVX2 and AVX-512 extensions on x86 processors, and a variable width with the Scalable Vector Extension on Arm processors such as Fujitsu's A64FX, which powers Fugaku, and NVIDIA's Grace. A loop that is not vectorised can use only a fraction of the core's arithmetic capability. For a memory-bound kernel this matters less than it might seem, since the arithmetic was not the bottleneck, but vectorisation also affects how efficiently the core issues loads, and unvectorised code often cannot generate enough memory requests to saturate the bandwidth. Compilers vectorise loops automatically when they can prove that iterations are independent and the memory access pattern is regular. They fail when loops contain function calls they cannot inline, conditionals that vary from iteration to iteration, indirect addressing, or potential aliasing between arrays. Compiler optimisation reports, which list each loop and whether it was vectorised and why not, are one of the most underused tools in scientific computing. Reading them for the hottest loops in a code, and then restructuring those loops or adding hints such as OpenMP's simd directive, is usually a quicker route to performance than any hand-written intrinsics. NUMA and placement On a node with several NUMA domains, the placement of data and threads becomes a performance question. Linux, by default, places each page of memory in the domain of the core that first writes to it, a policy known as first touch. A common and costly mistake follows from it: a program allocates and initialises a large array in a serial section on one thread, so all its pages land in one NUMA domain, and then processes it in a parallel loop on threads spread across all domains. Most of those threads now access remote memory, and the one domain's memory controllers become a bottleneck. The fix is to initialise data in parallel using the same distribution of work among threads that the computation will use, so that each thread's portion lands in its own domain. Placement of processes and threads matters equally. If the operating system is free to migrate threads between cores, a thread may move away from the memory it initialised or from the cache that holds its working set. Binding, or pinning, each MPI process to a set of cores within one NUMA domain, and each thread to a single core, prevents this. Job launchers on supercomputers, such as Slurm's srun, provide options for binding, and the right settings for a given node layout are among the first things a new user of a system should learn from its documentation. On GPU nodes, binding matters further, because each GPU is attached to a particular CPU socket or NUMA domain and to a particular network interface, and a process driving a GPU should run on cores close to it. MPI everywhere, or MPI plus threads A node with 128 or more cores can be programmed in two basic ways. The first is to run one MPI process per core, treating the node as many independent processors that happen to share a box. The second is to run a few MPI processes per node, perhaps one per NUMA domain or one per GPU, each spawning threads, typically with OpenMP, to use the cores within it. Pure MPI has real advantages. It is simple, needing only one programming model. Each process's data is private, which avoids the subtle bugs of shared-memory programming, and processes naturally respect NUMA boundaries if pinned correctly. Many codes perform best this way on CPU systems. Hybrid MPI plus threads has its own advantages, and they grow with the core count. Fewer MPI processes means fewer, larger subdomains, which by the surface-to-volume argument of chapter five reduces the total halo size, both in communication and in memory. The memory saving can be decisive: each process carries its own copy of some data, such as lookup tables, mesh metadata and MPI library buffers, and with hundreds of processes per node the duplication can consume a substantial share of the node's memory. Fewer processes also means fewer participants in each collective, which reduces latency. And a hybrid code can sometimes balance load dynamically among threads within a process, which MPI processes cannot do among themselves without migrating data. The difficulties of hybrid programming are real too. Threads sharing data need care to avoid race conditions. The parts of the code that run serially between parallel regions, often including communication, leave most cores idle unless the code is carefully structured, which is Amdahl's law at the node level. And the interaction between threads and MPI must be declared: a program initialises MPI with MPI_Init_thread and requests a level of thread support, from MPI_THREAD_FUNNELED, where only the main thread makes MPI calls, to MPI_THREAD_MULTIPLE, where any thread may call MPI at any time. The higher levels are more flexible but have historically carried overhead in some libraries, because the library must protect its internal state with locks. Most hybrid codes use the funnelled model, with one thread handling communication while the others compute, or with all threads joining to compute after communication completes. A third option sits between the two. MPI-3 introduced shared-memory windows, created with MPI_Win_allocate_shared, which let MPI processes on the same node allocate a region of memory that all of them can access directly with ordinary loads and stores. A code can keep its pure-MPI structure but store large read-only tables once per node instead of once per process, or exchange halo data with on-node neighbours by direct copying instead of messages. Good MPI libraries already use shared memory internally for on-node messages, so the gain for communication is often modest, but the memory saving for duplicated data can be large. There is no universal answer to which model is best. It depends on the code's memory footprint, the cost of its serial sections, the node architecture, and the quality of the MPI library's on-node communication. The sensible practice is to make the number of processes and threads per process a run-time choice, and measure. Precision as a lever on bandwidth The final lever at the node level is the size of each number. Double precision, sixty-four bits, has been the default in scientific computing for decades, largely out of caution: rounding errors accumulate over millions of operations, and many algorithms, such as the solution of ill-conditioned linear systems, genuinely need the extra digits. But many quantities in a simulation are known only to a few significant figures to begin with, and for a memory-bound kernel the cost of each variable is proportional to its size. Storing a field in single precision halves its memory traffic and its footprint in cache and memory. The numerical analysis of where reduced precision is safe has become an active field. Mixed-precision iterative refinement, which solves a linear system in low precision and then corrects the solution using residuals computed in high precision, can recover full double-precision accuracy while doing most of the work in single or even half precision. In fluid dynamics and weather models, the experience of recent years, described more fully in chapter eight, is that large parts of the computation can run in single precision without degrading the results, provided a few sensitive operations, such as accumulations of small increments into large values, are kept in double. The benefit is not that single-precision arithmetic is faster, though on GPUs it often is, but that it moves half as many bytes. A tuning session, step by step The order in which these techniques are applied matters, and a sketch of a typical tuning session on a CPU node shows why. Imagine a finite-difference diffusion solver, written cleanly but without attention to performance, running on one dual-socket node with one MPI process per core. A profile shows that one routine, the stencil update, accounts for most of the run time, and a quick measurement against a STREAM run on the same node shows that the routine achieves well under half of the attainable memory bandwidth. The first step is to check that the loop order matches the memory layout. In this hypothetical case the loops are nested so that the innermost loop runs along the slowest-varying index, an easy mistake when code is translated between C and Fortran. Swapping the loops changes nothing in the arithmetic but brings the bandwidth achieved much closer to the measured roof. The second step is to read the compiler's vectorisation report for the loop. It reports that the loop was not vectorised because the compiler could not rule out that the input and output arrays overlap in memory. Declaring the pointers as non-aliasing, with the restrict qualifier in C, allows vectorisation. For a memory-bound kernel, the gain from this step is smaller than from the first, but the vectorised loop issues memory requests more efficiently and edges closer to the roof. The third step is to check what the hardware counters say about memory traffic. If the traffic per grid point is higher than the best-case estimate from chapter two, neighbours are being evicted before they are reused, and cache blocking is warranted. Blocking in the two faster dimensions brings the traffic down towards the best case, and performance rises in proportion. The fourth step is to look at the code around the hot loop. If the solver also computes a separate diagnostic over the same field every step, fusing the diagnostic into the stencil loop removes a whole extra pass over memory. If the field is stored in double precision but the problem's accuracy requirements permit single precision for storage, halving its size halves the remaining traffic. At the end of such a session, which might take a few days for a well-understood kernel, the routine could plausibly be running several times faster than at the start, entirely through changes to how data moves. None of the steps required a deep understanding of the physics, and none required changing the numerical method. They required knowing where the roof was, measuring how far below it the code ran, and applying the techniques in order of expected benefit. The same session on a GPU would follow the same logic with different specifics: coalescing before occupancy, occupancy before scratchpad tiling, and data residency above all. The node as the unit of craftsmanship The techniques in this chapter are unglamorous. They involve reading compiler reports, rearranging loops, renumbering meshes and getting binding flags right. They are also where the largest performance gains in many real codes are found, often several-fold on a single node, and those gains multiply across every node of a parallel run. They share the logic of the book's controlling idea. Layout, blocking, fusion, NUMA-aware initialisation and reduced precision all reduce the number of bytes that travel, or the distance they travel, between memory and the arithmetic units. A code whose processes each use their node well needs fewer nodes, sends fewer messages, and scales from a stronger base. Chapter 7: GPU Clusters: Accelerators and the New Balance Fifteen years ago, a supercomputer with graphics processors was an experiment. Today, nearly every system near the top of the rankings outside China relies on accelerators for the bulk of its floating-point capability, and the procurement plans of national laboratories, weather centres and cloud providers assume they will continue to. GPUs did not change the fundamental argument of this book. They sharpened it. An accelerator offers far more arithmetic and far more memory bandwidth than a CPU socket, but it demands far more parallelism to use either, and it adds new distances to the hierarchy: between the GPU and its host, between GPUs in the same node, and between GPUs in different nodes. Programming a GPU cluster well is largely a matter of keeping data on the accelerator and moving it between accelerators by the shortest available route. How a GPU differs A CPU core is designed to run one thread of instructions as fast as possible, with deep pipelines, large caches, branch prediction and out-of-order execution to hide the latency of memory. A GPU takes the opposite approach. It contains many simpler processing units grouped into streaming multiprocessors, as NVIDIA calls them, or compute units, in AMD's terminology, and it hides latency not by cleverness but by numbers. Each multiprocessor keeps many groups of threads resident at once, and when one group stalls waiting for memory, it switches instantly to another that is ready. The groups execute in lockstep: thirty-two threads form a warp on NVIDIA hardware, and on AMD's data-centre GPUs sixty-four threads form a wavefront. All threads in a group execute the same instruction at the same time on different data. Three consequences follow for the programmer. First, a GPU needs a great deal of parallel work to be efficient: not hundreds of independent tasks but hundreds of thousands, so that each multiprocessor has enough groups in flight to hide memory latency. A fluid-dynamics kernel that loops over a million cells, with one thread per cell, provides that easily; one that loops over a few thousand cells on a heavily strong-scaled subdomain may not. Second, threads in a group that take different branches of a conditional are serialised, with some threads idle while the others run, so conditionals that vary from cell to cell, common in physics parameterisations, cost more on a GPU than on a CPU. Third, and most important for the argument of this book, memory accesses by threads in a group are combined, or coalesced, into as few memory transactions as possible when the threads access adjacent addresses. If thread i reads element i of an array, the whole group's reads are served by a few wide transactions. If thread i reads element i of a structure in an array of structures, the reads are scattered, and the effective bandwidth falls sharply. This is why the structure-of-arrays layout discussed in the previous chapter is close to mandatory on GPUs. Each multiprocessor also has a small scratchpad memory, called shared memory on NVIDIA hardware and the local data share on AMD's, which the programmer controls explicitly. A stencil kernel can load a tile of the grid into this scratchpad once, then have all threads in a block read their neighbours from it, rather than from the high-bandwidth memory. This is cache blocking done by hand, and it is one of the standard optimisations for GPU stencils. Data residency The first rule of GPU computing in a large simulation is that data should live on the GPU and stay there. On systems in which the GPU is a separate device connected to its host CPU by PCI Express, copying data across that link is much slower than the GPU's access to its own memory. A code that copies its fields to the GPU, runs a kernel, and copies the results back at every time step will typically be slower than the same code running on the CPU alone, because the copying dominates. Early GPU ports of large codes often followed exactly this pattern, offloading a few expensive kernels while leaving the rest of the code and its data on the CPU, and they were often disappointing for exactly this reason. The successful ports moved the entire time step to the GPU, including the parts that were individually cheap, so that the fields could remain resident for the whole simulation and cross the host link only for input, output and occasional diagnostics. That requires porting much more code, often the entire model, and it is why GPU adoption in large scientific applications has taken years rather than months. Unified memory, offered by both NVIDIA and AMD, lets the CPU and GPU use the same pointers, with the system migrating pages between host and device memory on demand. It simplifies porting considerably, but on discrete GPUs the migrations are not free, and a code that relies on them without care can spend much of its time moving pages back and forth. The tightly coupled designs now arriving, AMD's MI300A in El Capitan, where the CPU and GPU share the same physical high-bandwidth memory, and NVIDIA's Grace Hopper, where a coherent high-bandwidth link joins CPU and GPU memories, reduce or remove this penalty. They make it practical to leave parts of a code on the CPU without paying a copy cost, which lowers the barrier to incremental porting. They do not remove the deeper requirement that each processor's data be close to where it is used. Many GPUs in one node A typical GPU node contains four to eight accelerators, connected to one another by a faster fabric than the host link: NVIDIA's NVLink, and AMD's Infinity Fabric. Within a node, data moving from one GPU to another can travel directly over this fabric without passing through the CPU's memory, at a bandwidth much higher than the network that joins nodes. The hierarchy of distances therefore gains another rung. Communication between GPUs on the same node is relatively cheap; communication between GPUs on different nodes crosses the interconnect and is much more expensive. This has practical consequences for domain decomposition. A code that assigns one MPI process to each GPU, which is the most common arrangement, should place the processes that own neighbouring subdomains on the same node where possible, so that most halo exchanges stay on the fast intra-node fabric. With a structured grid, this can be arranged by mapping the process grid onto nodes in blocks; with an unstructured mesh, a hierarchical partitioner can first divide the mesh among nodes and then among the GPUs within each node. GPU-aware MPI and the network When data resides in GPU memory and must travel to a GPU on another node, the naive route copies it from the source GPU to host memory, sends it with MPI from host to host, and copies it from host memory to the destination GPU. Each copy adds latency and consumes bandwidth. GPU-aware MPI libraries, sometimes called CUDA-aware or ROCm-aware, accept pointers to GPU memory directly in their send and receive calls and choose the best route themselves. With technologies such as NVIDIA's GPUDirect RDMA, and equivalent support in AMD's software stack and HPE's Slingshot network, the network interface can read from and write to GPU memory directly, bypassing host memory entirely. On large GPU systems, network interfaces are attached close to the GPUs, and on several current designs there is one network interface per GPU or per pair of GPUs, so that each accelerator has its own direct path to the interconnect. Using GPU-aware MPI is usually a matter of passing device pointers to the familiar MPI calls, and the gain over staging through the host can be large. It requires the MPI library to be built with GPU support and the code to be careful about synchronisation: before a buffer computed by a GPU kernel can be sent, the kernel must have finished writing it. Because GPU kernels run asynchronously with respect to the CPU, the CPU must explicitly wait for the relevant stream of GPU work to complete before calling MPI. That wait is a synchronisation point that can stall the pipeline, and avoiding it has become an active area of development. Stream-triggered and GPU-initiated communication, in which the GPU itself enqueues the send once its kernel completes without a round trip through the CPU, are being developed by vendors and MPI implementers for exactly this reason. For collective operations among GPUs, especially the large allreduce operations of machine-learning training, vendors provide specialised libraries: NVIDIA's NCCL and AMD's RCCL. These implement collectives with kernels that run on the GPUs themselves and exploit the topology of the intra-node fabric and the network. Many scientific codes that combine simulation with machine learning now use MPI for their halo exchanges and these libraries for their training collectives. Overlap on the GPU The overlap of communication with computation described in chapter five becomes more intricate on a GPU. The GPU executes work submitted to streams, ordered queues of kernels and copies; work in different streams may run concurrently. A typical overlapped time step submits a kernel to pack the boundary values into send buffers on one stream, and the interior update on another. The CPU waits for the pack kernel, starts the GPU-aware MPI exchange, then waits for the messages to arrive, and submits the unpack and boundary-update kernels. Meanwhile the interior kernel continues on the GPU. If everything is arranged correctly, the GPU is never idle and the network transfer is hidden under the interior update. The difficulty is that the GPU's interior update is now very fast, because the GPU is very fast, while the network is not correspondingly faster. A subdomain whose interior took long enough to hide the halo exchange on a CPU may finish its interior on a GPU before the exchange is done. Strong scaling on GPUs therefore runs out sooner, in terms of subdomain size, than on CPUs: each GPU must hold a larger subdomain, often millions of cells, to be efficient, both to keep its many threads busy and to provide enough interior work to cover the communication. This is the new balance in concrete form. The accelerator raised the speed of computation inside the node by a large factor, and raised the relative cost of everything outside it. Writing portable GPU code The GPU market is divided among vendors with different native programming models. NVIDIA's CUDA is the most mature and widely used. AMD's HIP is deliberately similar to CUDA, and tools exist to translate one to the other. Intel promotes SYCL, an open standard from the Khronos Group based on modern C++, through its oneAPI toolkit. A code written in CUDA runs only on NVIDIA hardware, which matters a great deal to laboratories whose last three systems came from three different vendors. Several approaches offer portability. Directive-based models, OpenACC and the device-offload features of OpenMP, let programmers annotate existing loops, often in Fortran, with directives telling the compiler to run them on the accelerator and which data to move. They are attractive for large legacy codes, because they leave the source largely readable to its scientific authors. C++ performance-portability libraries, notably Kokkos, developed at Sandia, and RAJA, developed at Lawrence Livermore, express parallel loops and multidimensional arrays as C++ abstractions that compile to CUDA, HIP, SYCL or CPU threads, and let the library choose the memory layout appropriate to each architecture. Standard-language parallelism, through the parallel algorithms of C++17 and later and the do concurrent construct of Fortran, is increasingly supported by compilers for offload as well. Python codes can reach GPUs through libraries such as CuPy, Numba and JAX. Table 4 sets out the main options on the criteria that usually decide among them. Table 4. Main approaches to programming GPU clusters (qualitative comparison). Approach Examples Vendor portability Typical use Native kernel languages CUDA, HIP Low; HIP covers AMD and NVIDIA Hand-tuned hot kernels, libraries Open C++ standard SYCL Moderate to high, depends on compiler support New C++ codes, Intel-based systems Compiler directives OpenACC, OpenMP offload Moderate; compiler support varies Large Fortran and C legacy codes C++ abstraction libraries Kokkos, RAJA High Large C++ applications at national laboratories Domain-specific languages GT4Py, Devito, PSyclone High within the domain Stencil codes in weather, ocean, seismic No approach is free. Directives depend on compiler maturity, which has varied considerably between vendors and over time. Abstraction libraries require a C++ code base and some learning. Domain-specific languages, discussed in the next chapter, require the community to accept a new way of writing code. In every case, the hardest part of a GPU port is rarely the syntax. It is restructuring the data layout for coalescing, keeping data resident, exposing enough parallelism, and reducing branching in physics code, which are the same changes whichever model is used. Deciding whether to port For an organisation with a working CPU code, the decision to port to GPUs is an investment decision, and it is often framed badly. The tempting comparison is of a single node: a GPU node runs the code some factor faster than a CPU node, so the port is worthwhile if the factor is large. That comparison leaves out most of what matters. The first thing it leaves out is the cost of the port itself. For a large code, a complete GPU port that keeps data resident can take several person-years of effort from people who understand both the science and the hardware, and those people are scarce. The second is maintenance. A code that must run on CPUs and on GPUs from more than one vendor needs either a portability layer, with its own learning curve and dependencies, or several code paths that must be kept consistent as the science evolves. The third is the strong-scaling behaviour described earlier: because each GPU needs a large subdomain, a problem that ran on many CPU nodes may run on fewer GPU nodes, but it may not run faster in absolute terms if it cannot use enough GPUs efficiently. A team with a fixed deadline and a fixed problem size needs to know the time to solution, not the throughput per node. The fourth, increasingly important, is energy. The relevant comparison for a computing centre is often energy to solution: the total electricity consumed to produce a given result. GPUs are typically far more energy-efficient for workloads that suit them, which is why the most energy-efficient systems on the Green500 list are accelerated. For workloads that do not suit them, such as codes with little parallelism or heavy branching, the advantage shrinks or disappears. The fifth is the hardware landscape. National facilities have moved decisively towards accelerated systems, and many no longer procure large CPU-only partitions. A code that cannot use GPUs may find itself confined to a shrinking share of the available computing, which can be a stronger argument for porting than any single benchmark. The counterexample of Fugaku and of the CPU-only LineShine system shows that CPU designs with high-bandwidth memory remain viable at the top end, but most researchers do not get to choose their centre's architecture. A sensible approach starts with measurement of where the time goes, then a small prototype of the dominant kernel on the target hardware, measured against its roofline, then an estimate of how much of the code must move to keep data resident. If the dominant kernels are bandwidth-bound and highly parallel, the high-bandwidth memory of a GPU will speed them up roughly in proportion to its bandwidth advantage, and that gives a realistic upper bound on the gain before any porting effort is committed. A note on CFD In computational fluid dynamics, GPU adoption has followed this pattern closely. Codes based on high-order methods with dense element-local operations, such as the spectral-element solver NekRS, developed at Argonne and descended from the long-lived Nek5000, have been designed for GPUs from the outset and use them efficiently. Explicit compressible-flow solvers, whose work is dominated by local stencil operations, port relatively easily. Implicit solvers with complex preconditioners and unstructured meshes are harder, because sparse linear algebra is memory-bound and irregular, and because the algebraic multigrid methods that work well on CPUs need careful redesign for GPUs; libraries such as hypre, PETSc and NVIDIA's AmgX now provide GPU implementations. Commercial CFD vendors have released GPU solvers for parts of their product lines, and engineering users are increasingly offered the choice. The common thread is that the codes that do well on GPUs are those whose data can stay resident, whose kernels expose abundant parallelism, and whose communication can be hidden behind sufficiently large subdomains. The balance, restated A GPU cluster is a CPU cluster with the gradient of the hierarchy made steeper. The inner layers, high-bandwidth memory and on-chip scratchpads, are faster than ever. The outer layers, the host link where it still exists, the network and the filesystem, are relatively farther away than ever. Every successful GPU code has responded the same way: by keeping data on the device, by arranging it for coalesced access, by routing communication along the fastest paths the node offers, and by making each GPU's share of the problem large enough that the far layers are visited rarely. Chapter 8: Forecasting at Scale: Weather and Climate Models Numerical weather prediction is one of the oldest and most demanding customers of supercomputing, and one of the most instructive. Its history runs from Lewis Fry Richardson's hand calculation of a six-hour forecast during the First World War, which took him weeks and produced a wildly wrong answer, through the first computer forecast on ENIAC in 1950, to the global ensembles of today. The accuracy of forecasts has improved steadily over that period: a review by Peter Bauer, Alan Thorpe and Gilbert Brunet in Nature in 2015 described a "quiet revolution" in which skill had improved by roughly a day of lead time per decade. That improvement came from better observations, better data assimilation and better physics, and from computers able to run finer grids and larger ensembles within the forecast deadline. This chapter traces how the weather and climate community has met the challenges described in the preceding chapters, because few other fields have had to confront all of them at once. The shape of the problem An atmospheric model divides into two main components. The dynamical core solves the equations of fluid motion for a compressible or nearly compressible atmosphere on a rotating sphere: the transport of momentum, heat and moisture by the wind, and the propagation of pressure and gravity waves. The physical parameterisations represent everything that happens at scales smaller than a grid cell or that the fluid equations do not describe: radiation, cloud formation and precipitation, convection, turbulence in the boundary layer, and the exchange of heat and moisture with land and ocean. Climate models couple the atmosphere to models of the ocean, sea ice and land surface, each with its own grid and its own parallel decomposition, exchanging fluxes through a coupler at regular intervals. The two atmospheric components stress the machine differently. The dynamical core is a set of stencil computations with horizontal halo exchanges, sometimes with global communication, and it is memory-bound in the way chapter two described. The physics operates column by column, with no horizontal communication at all, but with large amounts of branching logic, lookup tables and nonlinear calculations, and its cost varies with the weather. A forecast model must do both every time step, under a deadline. The pole and the transpose The geometry of the sphere has shaped atmospheric models for decades. A regular latitude-longitude grid is easy to program, but its meridians converge at the poles, so cells near the poles become very narrow. Explicit time-stepping schemes must take steps short enough that information cannot cross a cell in one step, and the narrow polar cells force short steps everywhere. One classical way around this, adopted by the European Centre for Medium-Range Weather Forecasts in its Integrated Forecasting System, is the spectral transform method. Fields are represented as sums of spherical harmonics, smooth global functions on the sphere, in which horizontal derivatives can be computed exactly; nonlinear terms and physics are computed on a grid; and at each time step the model transforms the fields from grid space to spectral space and back, using fast Fourier transforms along latitude circles and Legendre transforms along meridians. Combined with semi-implicit, semi-Lagrangian time-stepping, which permits long time steps, this approach has proved accurate and efficient for decades. Its parallel cost is a global data transposition. A Fourier transform along a latitude circle needs all the longitudes of that circle on one process; a Legendre transform needs all the latitudes for a given wavenumber. Moving between these representations requires an all-to-all exchange in which every process sends a portion of its data to many others, which in MPI terms is a call to MPI_Alltoallv or an equivalent pattern within subgroups. Unlike a halo exchange, whose volume per process shrinks as more processes are added, a transpose moves a large fraction of the whole data set across the network at every step. As machines have grown, the transposes have become an increasing share of the cost, and they are a major reason the community has explored alternative grids. The alternatives replace the latitude-longitude grid with a quasi-uniform grid covering the sphere without singular points. The icosahedral grid used by ICON, developed by the German Weather Service and the Max Planck Institute for Meteorology, is built by subdividing the faces of an icosahedron projected onto the sphere. The cubed-sphere grid, used by the FV3 dynamical core in the United States and by the LFRic model being developed by the UK Met Office to replace its long-serving Unified Model, projects the six faces of a cube onto the sphere. The MPAS model from the National Center for Atmospheric Research uses an unstructured Voronoi mesh that can be refined over a region of interest. These grids have no poles and require only local communication, at the price of more complicated numerics and data structures, and they scale more comfortably to large process counts. ECMWF, for its part, has continued to develop the spectral model while researching alternatives, including a finite-volume dynamical core, and has invested in making its transforms faster and more scalable, including on GPUs. Precision as a forecast improvement The most striking demonstration of the bandwidth argument in weather prediction came from reducing numerical precision. Research led by Tim Palmer at Oxford and colleagues, and work by Filip Váňa and others at ECMWF published in 2017, showed that the Integrated Forecasting System could run in single precision with no meaningful loss of forecast skill, provided a small number of sensitive calculations were kept in double precision. Since the cost of a memory-bound model is roughly proportional to the bytes it moves, halving the size of most variables reduced the cost of a forecast by a large fraction. ECMWF's choice of what to do with the saving is the instructive part. Rather than simply running the same forecast more cheaply, the centre reinvested the saved computing in more resolution and more vertical levels, and single precision became part of its operational forecasts in 2021. In 2023 the centre raised the horizontal resolution of its ensemble forecasts, the fifty-one-member set of perturbed forecasts used to quantify uncertainty, to about nine kilometres, the same as its single high-resolution forecast. That kind of change, in which an improvement in data movement is converted directly into better science, is the pattern this book has argued for throughout. The weather did not care whether the arithmetic was done with sixty-four bits; the memory system did. The research has continued below single precision. Half-precision arithmetic, with only sixteen bits, offers another halving of bandwidth and access to the very fast matrix units that GPUs provide for machine learning, but its narrow range and few significant digits make it suitable only for carefully chosen parts of a model, such as some components of the spectral transforms or parts of the radiation scheme. Stochastic rounding, which rounds randomly up or down in proportion to the discarded fraction so that errors do not accumulate systematically, is being studied as a way to make low precision safer. Moving the atmosphere onto GPUs Weather models posed a severe test for GPUs because of their size and structure. A typical model contains hundreds of thousands to millions of lines of Fortran, written over decades by scientists whose primary expertise is atmospheric physics. The physics code is full of conditionals, column loops and lookup tables. The dynamical core is spread across many separate loops, each streaming fields through memory. Porting such a code requires moving everything to the GPU for the data-residency reasons of chapter seven, and that means touching nearly all of it. The first operational weather model to run on GPUs was the COSMO regional model at MeteoSwiss, the Swiss national weather service, which went into service on a GPU-based system at the Swiss National Supercomputing Centre in 2016. The port combined two strategies that remain the main options today. The dynamical core was rewritten using a C++ domain-specific library, called STELLA, in which stencils were expressed abstractly and the library generated efficient code for CPUs or GPUs. The physics was ported with OpenACC directives added to the existing Fortran. MeteoSwiss has since moved to running the ICON model on GPUs on the centre's Alps system, building on a multi-year community effort to port ICON with OpenACC. Domain-specific languages have become a significant theme in the field. GT4Py, developed at the Swiss centre and ETH Zurich, lets developers write stencil computations in a restricted form of Python, from which it generates optimised code for different back-ends, applying fusion, tiling and layout transformations automatically. It was used, notably, to rewrite the FV3 dynamical core in Python, and it underpins efforts to port parts of ICON. PSyclone, developed at the UK's Science and Technology Facilities Council, takes a different approach: scientists write Fortran in a constrained style that separates the science from the parallelisation, and PSyclone transforms it, inserting the halo exchanges, directives and loop optimisations appropriate for each target machine. It is central to LFRic and has been applied to the NEMO ocean model. The motivation for both is the separation of concerns: atmospheric scientists should be able to write numerical methods without encoding the data layout and memory hierarchy of a particular machine, and computational scientists should be able to optimise data movement for a new architecture without rewriting the science. In the United States, the Energy Exascale Earth System Model project of the Department of Energy took the C++ route. Its cloud-resolving atmosphere component, known as SCREAM, was written in C++ with Kokkos so that it could run on the GPUs of the department's exascale machines. In 2023 a team running SCREAM on Frontier at a global resolution of about three kilometres received the first ACM Gordon Bell Prize for Climate Modelling, for showing that a global cloud-resolving model could run fast enough to make multi-year simulations practical. At that resolution, deep convection, the towering storm clouds that dominate tropical weather, begins to be represented explicitly rather than through a parameterisation, removing one of the largest sources of uncertainty in climate projections. It is a scientific capability made possible only by the combination of a large GPU system and a code designed around its memory hierarchy. Coupled models and the problem of many components A climate model is not one program but several, coupled together. The atmosphere, the ocean, the sea ice, the land surface, and sometimes ice sheets, atmospheric chemistry and ocean biogeochemistry are each represented by a component with its own grid, its own time step and often its own development team. The ocean grid is typically finer than the atmospheric grid in the horizontal, to resolve eddies and boundary currents, and it is masked by continents, so many of its grid points are land and do no work. The atmosphere may take a step of a few minutes, the ocean longer, and the components exchange fluxes of heat, water and momentum at their shared surface at a coupling interval of perhaps an hour. This creates a parallel-computing problem of its own: how to divide the machine among components. Components can run sequentially on the same processes, each taking its turn, which wastes no processes but means the total time per coupling interval is the sum of the component times. They can run concurrently on disjoint sets of processes, each component with its own communicator split from the world communicator, which allows them to compute at the same time but means one component may sit idle waiting for the other's fluxes if the division is unbalanced. Getting the division right is a load-balancing exercise performed by trial runs, and when the atmosphere moves to GPUs while the ocean remains on CPUs, as has happened in some projects, the balance must be found across different kinds of hardware. The coupling itself involves regridding: interpolating fields from one component's grid to another's while conserving fluxes. Because the two grids are decomposed differently across processes, regridding requires each process to exchange data with the processes of the other component whose subdomains overlap its own, a sparse irregular communication pattern computed once from the geometry of the grids. Coupling software such as OASIS, developed at CERFACS in France and widely used in European climate models, and the Earth System Modeling Framework, developed in the United States, handle the bookkeeping, computing the interpolation weights and the communication patterns so that each component sees only its own decomposition. Coupling also illustrates the deadline pressure that runs through the whole field. A climate projection of a century, at a resolution where the model achieves, say, one simulated year per day of wall-clock time, takes more than three months of continuous running for a single member of a single scenario. Climate scientists talk about throughput in simulated years per day, and a model that cannot reach a few simulated years per day at a given resolution is impractical for multi-century experiments regardless of how well it scales in a benchmark. That arithmetic is why the strong-scaling limit, and not just the size of the machine, sets the highest resolution at which climate can be studied. Ensembles, I/O and the data deluge A forecast is no longer a single model run. Ensembles of dozens of perturbed forecasts are run to estimate uncertainty, and each member produces output: three-dimensional fields of many variables at many time steps. The volume of data a modern forecasting centre produces each day is enormous, and much of it must be post-processed, archived and delivered to users within tight deadlines. For many centres, writing and processing output has become as challenging as computing it. The standard response is to separate input and output from computation. Rather than having every compute process write its own part of each field to the filesystem, models designate a set of dedicated I/O server processes. Compute processes send their output to these servers with MPI, often asynchronously, and resume computing; the servers aggregate, compress, and write the data, or pass it directly to post-processing. The XIOS server developed in France is widely used in European climate models; ECMWF has developed its own systems for its forecast output. Chapter nine returns to the general principles. The underlying logic is again about distances: the filesystem is the farthest layer in the hierarchy, and the cheapest way to deal with it is to visit it less often, in larger, well-organised chunks, using processes whose only job is to do so. Machine learning enters the forecast The largest recent change in weather prediction has come from an unexpected direction. Beginning around 2022, machine-learned forecasting models, trained on decades of ECMWF's reanalysis data, began to match and in some measures exceed the skill of traditional physics-based models for medium-range forecasts. Huawei's Pangu-Weather and Google DeepMind's GraphCast, published in Nature and Science respectively in 2023, were among the most prominent. Once trained, such a model produces a ten-day forecast in a minute or so on a single GPU, compared with the large fraction of a supercomputer that a physics-based forecast requires. ECMWF developed its own machine-learned system, the Artificial Intelligence Forecasting System, and made it operational alongside its physics-based forecasts in 2025. This does not make supercomputing irrelevant to weather; it changes what the supercomputer is for. The machine-learned models depend on reanalyses, which are produced by physics-based models and data assimilation running on supercomputers. Their initial conditions still come from the data assimilation system of a traditional model. Training them requires large GPU clusters and the collective-communication patterns of chapter seven. And physics-based models remain the reference for extreme and unprecedented events, for climate projections beyond the range of the training data, and for scientific understanding. The likely future, which forecasting centres are already exploring, is hybrid: physics-based models at high resolution for reanalysis, research and some operational products, machine-learned models for rapid large ensembles, and machine-learned components embedded within physics models to replace expensive parameterisations. Digital twins of the Earth The European Union's Destination Earth initiative, launched in 2022 and run by ECMWF, the European Space Agency and EUMETSAT, aims to build what it calls digital twins of the Earth system: very high-resolution models, running continuously on EuroHPC supercomputers, that users can interrogate interactively to explore scenarios, such as the effect of a heatwave on a region's water supply or of a storm on its infrastructure. The ambition requires running kilometre-scale global models on GPU systems spread across several European centres, streaming their output to users rather than storing it all, and coupling them with machine-learned tools for rapid analysis. It brings every theme of this book into one programme: strong scaling under deadlines, GPU portability across vendors, reduced precision, I/O at extreme volume, and the integration of simulation with machine learning. The weather and climate community's experience offers a summary of the field. Gains have come less from faster arithmetic than from moving less data, through single precision, quasi-uniform grids that avoid global transposes, data residency on accelerators, kernel fusion through domain-specific languages, and dedicated I/O servers. And the community has learned, sometimes painfully, that the software investment needed to exploit a new architecture is measured in years, which makes portability across architectures not a convenience but a strategic necessity. Chapter 9: Measuring, Writing and Surviving: Profiling, Parallel I/O and Resilience The preceding chapters have described what limits a parallel simulation's performance and how to design around those limits. This chapter deals with three activities that determine whether that design survives contact with a real machine: measuring what the code actually does, getting data in and out of the machine without crippling it, and surviving the failures that become routine at scale. They tend to be neglected because they are not where the science lives. They are frequently where the time goes. Measure before tuning The first rule of performance work is that intuition about where a large parallel code spends its time is usually wrong. A routine that looks expensive turns out to be cheap; a harmless-looking diagnostic computed every step turns out to dominate at scale; the halo exchange that was supposed to overlap with computation turns out to be waiting for a rendezvous handshake that never progresses. Measurement comes first, and it should answer three questions in order. How much of the run time is computation, how much is communication, and how much is waiting? Within the computation, which kernels dominate, and how close is each to its roofline? Within communication and waiting, which operations and which processes are responsible? Tools for answering these questions fall into two broad families. Profilers sample or count events and report aggregated totals: time per function, cache misses per loop, bytes sent per MPI call. They are cheap to run and give the overall picture. Tracers record a timestamped log of events on every process and let the user view a timeline, showing exactly when each process computed, sent, received and waited. They are expensive in data volume but indispensable for diagnosing load imbalance and failed overlap, which are invisible in aggregated totals. The ecosystem of tools is mature. Score-P, a community instrumentation infrastructure, feeds both the Scalasca tool, which automatically searches traces for wait-state patterns such as late senders and imbalanced collectives, and the Vampir trace visualiser. TAU, from the University of Oregon, and HPCToolkit, from Rice University, offer profiling and tracing with low overhead. Vendors provide their own: NVIDIA's Nsight Systems and Nsight Compute for GPU timelines and kernel analysis, AMD's ROCm profiling tools, Intel's VTune and Advisor, HPE's CrayPat, and Linaro's Forge suite, which grew out of the Allinea tools. Most of these build on two facilities defined by the MPI standard itself: the profiling interface, which allows a tool to intercept every MPI call by providing its own versions of the functions that then call the real ones, and the MPI_T tools interface, which exposes internal performance variables of the library such as queue lengths and protocol choices. A practical approach for a new code is to begin with a lightweight profile at two or three process counts, to see which costs grow as the process count grows, and then to trace a short run at the largest scale of interest, to see why. A cost that is constant per process in a weak-scaling study is usually benign; one that grows with the process count, typically a collective, a serial section, or an unbalanced kernel, is what will eventually stop the code scaling. Measurement also has to contend with variability. On a shared machine, a job's performance depends on where its nodes are placed in the network and what other jobs are doing. Timing results from a single run can be misleading, so serious performance studies repeat runs and report the spread as well as the mean. When the spread is large, it is itself a finding: it often points to sensitivity to network congestion or to a particular slow node, and it argues for designs that synchronise less often. The filesystem is far away Parallel input and output is where many codes that scale perfectly in computation fall apart. The parallel filesystem described in chapter one offers high aggregate bandwidth, but only to access patterns that suit it: large, contiguous, well-aligned requests spread across many storage servers. It performs badly on the patterns that come most naturally to a parallel program: many processes each writing small, scattered pieces, and many processes each creating, opening and closing their own files. The simplest strategy, one file per process, is easy to program and often fast at modest scale, because each process writes contiguously to its own file. At tens of thousands of processes it creates tens of thousands of files per output step, overwhelming the filesystem's metadata servers, which must track the existence, location and permissions of every file. It also leaves the user with a dataset that must be reassembled before it can be analysed and that can be read back only with the same decomposition. The opposite strategy, having every process send its data to one process that writes a single file, avoids the metadata problem but funnels all output through one node's bandwidth. The middle path is a single shared file, or a modest number of files, written in parallel by many processes. MPI-IO, part of the MPI standard since MPI-2, provides the mechanism. Each process describes, using derived datatypes, which parts of the file it owns, and the processes then write collectively. A collective write lets the MPI library apply an optimisation called two-phase I/O, or collective buffering: a subset of processes, the aggregators, gather data from the others so that each aggregator holds a large contiguous region of the file, and only the aggregators write, in large, aligned requests. This converts many small scattered writes into a few large ones, which is exactly what the filesystem wants. It is the same principle as message aggregation in the network, applied to storage. Most scientific codes do not use MPI-IO directly but through higher-level libraries that sit on top of it. HDF5 and NetCDF-4, which is built on HDF5, store self-describing multidimensional arrays with metadata, and support parallel writing through MPI-IO. Parallel-NetCDF serves the older NetCDF formats widely used in climate science. ADIOS2, developed under the US exascale programme, provides a flexible framework that can write to files in various formats or stream data directly to analysis programs running elsewhere, which supports the in-situ analysis discussed below. Using these libraries well still requires attention to the underlying filesystem. On Lustre, for example, the number of storage targets across which a file is striped and the size of each stripe must be set appropriately for large shared files; the default settings on many systems suit small files and can leave a large parallel write bottlenecked on a few servers. Writing less The most effective I/O optimisation is to write less. For a simulation whose output will be analysed later, that might mean writing only the variables and time steps that will actually be used, reducing precision for stored fields where scientifically justified, or applying compression. Lossless compression of floating-point fields gives modest ratios, because the low-order bits of measured or simulated quantities look like noise. Error-bounded lossy compressors, such as SZ and ZFP, both developed with US Department of Energy support, guarantee that each reconstructed value lies within a specified tolerance of the original and achieve much higher ratios. Weather and climate centres have studied which variables tolerate how much loss, with the aim of matching the stored precision to the information the data actually contains. A more radical approach is in-situ analysis and visualisation: analysing data while the simulation is running, on the same nodes or on nearby ones, and storing only the results. Tools such as ParaView Catalyst and VisIt's in-situ interfaces, and streaming through frameworks such as ADIOS2, make this practical. The I/O server model of the previous chapter is a step in the same direction. For the largest simulations, where storing every field at every time step would be impossible, in-situ processing is not an optimisation but the only feasible design. It also changes the scientific workflow, since analyses must be decided before the run rather than after, which is a real cost and one reason that adoption has been gradual. Failure as a normal event A supercomputer contains tens of thousands of nodes, hundreds of thousands of memory modules, thousands of network links and many thousands of storage devices. Even if each component is highly reliable, the probability that some component fails during a long job on a large fraction of the machine is substantial. Large jobs on large systems must therefore expect to be interrupted, and design for it. The standard defence is checkpoint and restart. Periodically, the application writes its full state to storage, and if the job fails, it is restarted from the most recent checkpoint, losing only the work done since. The question is how often to checkpoint. Checkpointing too rarely wastes a lot of work at each failure; checkpointing too often spends a lot of time writing checkpoints. A classic result, derived by John Young in 1974 and refined by John Daly in 2006, gives a first-order answer: the optimal interval between checkpoints is approximately the square root of twice the time to write a checkpoint multiplied by the mean time between failures. The formula shows why checkpoint cost matters so much at scale. As machines grow, the mean time between failures for a whole-machine job falls; if checkpoint cost does not fall too, a growing fraction of the machine's time is spent checkpointing and recovering. Several techniques reduce checkpoint cost. Application-level checkpointing, in which the code writes only the state it actually needs to restart rather than an image of its entire memory, typically produces much smaller checkpoints. Multilevel checkpointing, implemented in libraries such as SCR from Lawrence Livermore and VeloC from the US exascale programme, writes frequent checkpoints quickly to node-local storage or to a partner node's memory, and only occasionally drains them to the parallel filesystem, on the reasoning that most failures affect one node and can be recovered from a nearby copy. Asynchronous checkpointing writes the state in the background while computation continues. MPI itself has traditionally offered little help with failure: under the standard's default behaviour, if one process fails, the whole program is aborted. The User-Level Failure Mitigation proposal, developed over many years within the MPI Forum and implemented experimentally in Open MPI and MPICH, defines ways for surviving processes to detect a failure, agree on it, and rebuild a working communicator without the failed process, so that the application can recover in place. Adoption by applications has been limited, partly because recovery logic is complex to write and partly because checkpoint and restart remains adequate for most workloads, but the work has influenced how libraries and runtimes approach resilience. There is also a quieter kind of failure: silent data corruption, in which a hardware fault changes a value without any error being reported. Memory protected by error-correcting codes catches most such events in memory, but not all faults in all components are detectable. For most simulations, the practical defences are physical consistency checks, such as conservation of mass and energy, and comparison of key diagnostics against expectations. Reproducibility A related issue, which surprises many newcomers, is that parallel programs do not always produce bit-identical results from run to run. Floating-point addition is not associative: adding a set of numbers in a different order can change the last few digits of the sum. An MPI_Allreduce may combine contributions in an order that depends on the algorithm chosen, which can depend on the process count and sometimes on timing. Threaded reductions and GPU atomic operations introduce similar variation. For a chaotic system such as the atmosphere, tiny differences in the last bits grow over a forecast until two runs differ visibly. This is not an error in either run, but it complicates testing, debugging and regulatory or scientific requirements for reproducibility. Weather and climate centres, which need to verify that a code change has not altered results unintentionally, often require bit-reproducibility for a fixed configuration and enforce it by fixing the order of reductions, at some cost in performance. Some MPI libraries offer options for reproducible reductions. Reproducible summation algorithms, which produce the same result regardless of order, are an active research topic. The general lesson is that reproducibility, like performance, must be designed in; it does not arise by accident in a parallel program. Measuring energy A final kind of measurement has moved from the margins to the centre of the field: energy. When a large system draws tens of megawatts, the electricity consumed by a single large simulation campaign is a significant cost, and computing centres increasingly account for it. Many systems now report energy usage per job, drawing on power sensors in nodes and accelerators, and some charge allocations partly in energy rather than node-hours. Energy measurement changes some optimisation decisions. Because moving data costs more energy than computing on it, the optimisations that reduce data movement generally reduce energy as well, often by more than they reduce time. But there are trade-offs. Running a processor at a lower clock frequency reduces its power substantially, and for a memory-bound code, whose speed is set by memory bandwidth rather than by the clock, the run time may barely change. Several centres have experimented with capping GPU or CPU frequencies for memory-bound workloads, gaining energy savings for a small loss of speed. Conversely, spreading a job over more nodes to finish it sooner may cost more total energy if the extra nodes spend much of their time waiting on communication. The relevant figure is energy to solution, and it is measured the same way as everything else in this chapter: by running the code, recording what it consumes, and comparing configurations rather than assuming. The operational discipline These three activities share a character. They are where the abstract design of a parallel code meets the messy reality of a shared, failure-prone machine with a finite filesystem. They reward the same habits: measuring rather than assuming, aggregating many small operations into a few large ones, keeping data close and visiting the far layers of the hierarchy rarely, and designing for the common case at scale, which is very different from the common case on a workstation. A code that ignores them can compute brilliantly and still fail to produce a usable result on time. Conclusion The argument of this book has been that a supercomputer is best understood as a hierarchy of distances, and that the work of making a simulation run well on one is the work of managing where data lives and how it travels. That argument has been made layer by layer: the roofline for a single kernel, the message for a pair of processes, the collective for a whole communicator, the halo for a decomposed domain, the cache and the NUMA domain within a node, the high-bandwidth memory and the fabric of a GPU cluster, the transposes and I/O servers of a weather model, and the checkpoints and parallel files that connect a computation to storage. It is worth drawing out what follows from the argument as a whole, because several of its consequences cut against habits that remain common in scientific computing. Count bytes before counting flops The first consequence is that the instinct to measure a computation by its operation count is usually misplaced. Scientists and engineers are trained to compare algorithms by how many operations they require, and the machines they use are marketed by how many operations they can perform. Yet for the great majority of simulation codes, the arithmetic is almost free, and the cost lies in bringing operands to the arithmetic units. A new member of a computational team who learns to estimate the bytes moved per operation in a hot loop, and to compare it against the measured balance of the machine, will understand the code's performance better than one who knows the peak speed of every processor on the market. This has implications for the choice of numerical methods. A method that does more arithmetic per grid point, but moves fewer bytes for a given accuracy, is likely to win on current and future hardware. High-order discretisations, matrix-free operators, communication-avoiding Krylov solvers and reduced-precision storage all exploit this, and all have gained ground for that reason. Numerical analysts and computational scientists who once worked largely apart now need to work together, because the best algorithm depends on the machine. Treat communication as a design decision, not an implementation detail The second consequence concerns MPI. It is common to regard the parallelisation of a code as a technical chore applied after the science is written: add some halo exchanges, a few reductions, and the job is done. The chapters on collectives and domain decomposition suggest otherwise. How a domain is divided, how wide its halos are, how often processes synchronise, which communication is overlapped with which computation, and how many small messages are aggregated into large ones, are choices that determine whether the code will scale, and they interact with the numerical method. A time-stepping scheme that requires a global reduction at every stage, or an advection scheme with a wide stencil, has a parallel cost that should be weighed when the scheme is chosen, not discovered when the code is run on ten thousand processes. MPI's explicitness, which makes it laborious to use, is what makes these decisions visible. The lasting value of the message-passing model is that it forces the programmer to state where data goes. Higher-level abstractions are welcome where they preserve that visibility, as domain-specific languages for stencils do, and dangerous where they hide it. Design for the steepening gradient The third consequence follows from the trend described in the first chapter and confirmed by the GPU chapter: the hierarchy of distances is getting steeper. Arithmetic grows faster than memory bandwidth, memory bandwidth faster than network bandwidth, network bandwidth faster than storage bandwidth. Each generation of hardware moves the outer layers relatively farther from the centre. A code that is balanced today will be less balanced on the next machine. The practical response is to build codes that can adapt. That means separating the description of the science from the decisions about data layout, parallel decomposition and target hardware, so that the latter can change without rewriting the former. The weather and climate community's investment in domain-specific languages and source-to-source tools is the most systematic attempt to do this, and its experience suggests both the promise and the cost: such tools take years to mature and require a community willing to adopt them, but they make it possible to move a large model to a new architecture in a fraction of the time that a hand port would take. For smaller teams, performance-portability libraries such as Kokkos and RAJA, or carefully used directive-based models, offer a similar separation at lower cost. It also means treating portability across vendors as a requirement. The large facilities of the last decade have been built by different vendors with different accelerators, and the top of the June 2026 list included systems based on AMD, Intel and NVIDIA accelerators and a CPU-only design. A code written for one vendor's proprietary programming model is a bet on that vendor's future share of the market, and history suggests that such bets are risky over the twenty or thirty years that a major scientific code typically lives. What remains unsettled Several questions are genuinely open. The first is the future role of machine learning in simulation. Machine-learned weather models have already changed operational forecasting, and machine-learned surrogates are being developed for turbulence closures, cloud parameterisations and other expensive components of physics models. They have very different computational profiles from traditional simulation: dense matrix operations with high arithmetic intensity, well suited to the specialised matrix units in modern GPUs. If substantial parts of simulation migrate to learned components, the balance of the machines that serve science may shift towards that profile, and the memory-bound picture painted in this book will apply to a smaller share of the work. It is not yet clear how far this will go, how reliably learned models extrapolate to conditions outside their training data, or how to verify them to the standards that safety-critical engineering and climate projection require. The second open question is how far reduced precision can go. Single precision has proved safe for large parts of weather and fluid models. Half precision and other narrow formats, which would again halve data movement and unlock the fastest hardware units, remain limited to carefully chosen components. The numerical analysis needed to use them widely and safely is still being developed. The third concerns the machines themselves. Tightly coupled CPU-GPU designs with shared memory, such as those in El Capitan and JUPITER, reduce one of the most painful distances in the hierarchy. Further integration, including memory stacked directly on processors and optical interconnects that reduce the energy cost of moving data between chips and nodes, may continue that trend. Whether these developments will flatten the hierarchy appreciably, or merely slow its steepening, will shape the next decade of parallel programming. What a reader should do differently For someone who writes or maintains a parallel simulation code, the practical message is to measure before tuning, and to measure the right things: the fraction of time spent computing, communicating and waiting; the bytes moved by each hot kernel against the roofline; the growth of each cost with process count. Tuning effort should go to whatever the measurements reveal, and it usually reveals data movement rather than arithmetic. For someone choosing a method or designing a new code, the message is to consider data movement from the start: how the domain will be divided, how much each process will exchange, how often everyone must synchronise, and whether the dominant kernels can be arranged to reuse the data they fetch. These decisions are cheap at the design stage and very expensive to change later. For someone who funds, procures or depends on these codes, whether a research manager, a computing centre or a national weather service, the message is that hardware alone does not deliver performance. The most expensive machines in the world will run a poorly structured code at a small fraction of their capability. Investment in software, and in the people who understand both the science and the memory hierarchy, is what converts a machine's peak into results. The weather community's conversion of a precision change into a resolution increase, and the climate community's achievement of global cloud-resolving simulation on an exascale GPU system, were software achievements as much as hardware ones. The supercomputers of 2026 can perform more than a quintillion operations per second. Almost none of the simulations they run come close to that number, and they never will, because the operations are not the hard part. The hard part is the journey each number makes from where it is stored to where it is needed. The people who understand that journey are the ones who make these machines useful. Further Reading Foundations and textbooks Georg Hager and Gerhard Wellein, Introduction to High Performance Computing for Scientists and Engineers (CRC Press, 2010).** A practical, performance-oriented introduction to node-level optimisation, memory hierarchies, OpenMP and MPI, written by practitioners at a German computing centre. Its treatment of balance and memory-bound code remains one of the clearest available. Victor Eijkhout, The Art of HPC (freely available online).** A set of open textbooks from the Texas Advanced Computing Center covering the science of scientific computing and parallel programming in MPI and OpenMP, with extensive examples. A good companion for readers who want to try the ideas in this book in code. John L. Hennessy and David A. Patterson, Computer Architecture: A Quantitative Approach (6th edition, Morgan Kaufmann, 2017).** The standard text on processor and memory-system design, for readers who want to understand why caches, vector units and GPUs are built as they are. David B. Kirk and Wen-mei W. Hwu (with Izzat El Hajj), Programming Massively Parallel Processors (4th edition, Morgan Kaufmann, 2022).** A widely used introduction to GPU architecture and CUDA programming, with chapters on tiling, memory coalescing and parallel patterns. MPI William Gropp, Ewing Lusk and Anthony Skjellum, Using MPI: Portable Parallel Programming with the Message-Passing Interface (3rd edition, MIT Press, 2014).** The standard introduction by members of the MPI Forum, covering point-to-point and collective communication and the design of parallel programs. William Gropp, Torsten Hoefler, Rajeev Thakur and Ewing Lusk, Using Advanced MPI: Modern Features of the Message-Passing Interface (MIT Press, 2014).** Covers one-sided communication, neighbourhood collectives, shared-memory windows, hybrid programming and parallel I/O. MPI Forum, MPI: A Message-Passing Interface Standard, Version 5.0 (2025).** The authoritative specification, freely available from the MPI Forum's website. Dense, but the definitive answer to questions about semantics. Rajeev Thakur, Rolf Rabenseifner and William Gropp, "Optimization of Collective Communication Operations in MPICH", International Journal of High Performance Computing Applications 19, no. 1 (2005).** The classic account of how collective algorithms are chosen by message size and process count. Performance models and scaling Samuel Williams, Andrew Waterman and David Patterson, "Roofline: An Insightful Visual Performance Model for Multicore Architectures", Communications of the ACM 52, no. 4 (2009).** The paper that introduced the roofline model. John L. Gustafson, "Reevaluating Amdahl's Law", Communications of the ACM 31, no. 5 (1988).** The short paper that set out the case for weak scaling. Fabrizio Petrini, Darren J. Kerbyson and Scott Pakin, "The Case of the Missing Supercomputer Performance: Achieving Optimal Performance on the 8,192 Processors of ASCI Q", Proceedings of the ACM/IEEE Conference on Supercomputing (2003).** A detective story about operating-system noise, and a lasting lesson about rare events at scale. Pieter Ghysels and Wim Vanroose, "Hiding Global Synchronization Latency in the Preconditioned Conjugate Gradient Algorithm", Parallel Computing 40, no. 7 (2014).** An accessible example of reformulating a numerical method to reduce the cost of synchronisation. George Karypis and Vipin Kumar, "A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs", SIAM Journal on Scientific Computing 20, no. 1 (1998).** The algorithmic basis of METIS and of most modern mesh partitioning. John T. Daly, "A Higher Order Estimate of the Optimum Checkpoint Interval for Restart Dumps", Future Generation Computer Systems 22, no. 3 (2006).** The refinement of Young's checkpoint-interval formula. Weather and climate Peter Bauer, Alan Thorpe and Gilbert Brunet, "The Quiet Revolution of Numerical Weather Prediction", Nature 525 (2015).** A concise review of how forecast skill has improved and what computing has contributed. Filip Váňa and colleagues, "Single Precision in Weather Forecasting Models: An Evaluation with the IFS", Monthly Weather Review 145 (2017).** The evidence that a major global forecast model could run in single precision without loss of skill. Remi Lam and colleagues, "Learning Skillful Medium-Range Global Weather Forecasting", Science 382 (2023), and Kaifeng Bi and colleagues, "Accurate Medium-Range Global Weather Forecasting with 3D Neural Networks", Nature 619 (2023).** The GraphCast and Pangu-Weather papers, which mark the arrival of machine-learned medium-range forecasting. The TOP500 list (top500.org). Published each June and November since 1993, with the HPCG and Green500 rankings alongside. The lists and their accompanying highlights are the best single source for the current state of the largest machines. Hashtags: #HighPerformanceComputing #HPC #Supercomputing #Supercomputers #SupercomputerArchitecture #ParallelComputing #ParallelProgramming #DistributedComputing #MessagePassingInterface #MPI #MPIProgramming #MPICollectives #NonBlockingCommunication #PointToPointCommunication #DomainDecomposition #HaloExchange #GhostCells #ParallelAlgorithms #ParallelScaling #StrongScaling #WeakScaling #AmdahlsLaw #GustafsonsLaw #ExascaleComputing #ExascaleSupercomputers #GPUComputing #GPUClusters #GPUAcceleration #HybridParallelism #OpenMP #CUDA #MemoryHierarchy #MemoryBandwidth #MemoryOptimization #CacheOptimization #NUMA #RooflineModel #ArithmeticIntensity #DataMovement #CommunicationOptimization #LoadBalancing #HighSpeedInterconnects #InfiniBand #ParallelIO #ScientificComputing #ComputationalFluidDynamics #NumericalWeatherPrediction #ClimateModeling #PerformanceProfiling #HPCOptimization
Latest Book Releases:










































